diff --git a/.agents/skills/trellis-before-dev/SKILL.md b/.agents/skills/trellis-before-dev/SKILL.md new file mode 100644 index 0000000..096f8bf --- /dev/null +++ b/.agents/skills/trellis-before-dev/SKILL.md @@ -0,0 +1,40 @@ +--- +name: trellis-before-dev +description: "Discovers and injects project-specific coding guidelines from .trellis/spec/ before implementation begins. Reads spec indexes, pre-development checklists, and shared thinking guides for the target package. Use when starting a new coding task, before writing any code, switching to a different package, or needing to refresh project conventions and standards." +--- + +Read the relevant development guidelines before starting your task. + +Execute these steps: + +1. **Read current task artifacts**: + - `prd.md` for requirements and acceptance criteria + - `design.md` if present for technical design + - `implement.md` if present for execution order and validation plan + +2. **Discover packages and their spec layers**: + ```bash + python ./.trellis/scripts/get_context.py --mode packages + ``` + +3. **Identify which specs apply** to your task based on: + - Which package you're modifying (e.g., `cli/`, `docs-site/`) + - What type of work (backend, frontend, unit-test, docs, etc.) + - Any spec/research paths referenced by the task artifacts + +4. **Read the spec index** for each relevant module: + ```bash + cat .trellis/spec///index.md + ``` + Follow the **"Pre-Development Checklist"** section in the index. + +5. **Read the specific guideline files** listed in the Pre-Development Checklist that are relevant to your task. The index is NOT the goal — it points you to the actual guideline files (e.g., `error-handling.md`, `conventions.md`, `mock-strategies.md`). Read those files to understand the coding standards and patterns. + +6. **Always read shared guides**: + ```bash + cat .trellis/spec/guides/index.md + ``` + +7. Understand the coding standards and patterns you need to follow, then proceed with your development plan. + +This step is **mandatory** before writing any code. diff --git a/.agents/skills/trellis-brainstorm/SKILL.md b/.agents/skills/trellis-brainstorm/SKILL.md new file mode 100644 index 0000000..bd7aeb4 --- /dev/null +++ b/.agents/skills/trellis-brainstorm/SKILL.md @@ -0,0 +1,112 @@ +--- +name: trellis-brainstorm +description: "Guides collaborative requirements discovery before implementation. Creates task directory, seeds PRD, asks high-value questions one at a time, researches technical choices, and converges on MVP scope. Use when requirements are unclear, there are multiple valid approaches, or the user describes a new feature or complex task." +--- + +# Trellis Brainstorm + +## Non-Negotiable Interview Contract + +Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer. + +Ask the questions one at a time. + +## Non-Negotiable Evidence Rule + +If a question can be answered by exploring the codebase, explore the codebase instead. + +This is mandatory. Before asking the user a question, first check whether the answer is already available in code, tests, configs, docs, existing specs, or task history. + +Do not ask the user to confirm facts that the repository can answer. Ask only for product intent, preference, scope, risk tolerance, or decisions that remain ambiguous after inspection. + +--- + +Use this skill during Phase 1 planning to turn the user's request into clear requirements and planning artifacts. + +## Preconditions + +Use this skill only after task-creation consent has been given and the user is ready to enter Trellis planning. + +If no task exists yet, create one: + +```bash +TASK_DIR=$(python ./.trellis/scripts/task.py create "" --slug ) +``` + +Use a concise title from the user's request. Use a slug without a date prefix. `task.py create` adds the `MM-DD-` directory prefix automatically. + +`task.py create` creates the default `prd.md`. Update that file with the current understanding before asking follow-up questions. + +## Planning Flow + +1. Capture the user's request and initial known facts in `prd.md`. +2. Inspect available evidence before asking questions: + - code, tests, fixtures, and configs + - README files, docs, existing specs, and domain notes + - related Trellis tasks, research files, and session history when present +3. Separate what you found into: + - confirmed facts + - product intent still needed from the user + - scope or risk decisions still needed from the user + - likely out-of-scope items +4. Ask the single highest-value remaining question. +5. Include your recommended answer with the question. +6. After each user answer, update `prd.md` before continuing. +7. For complex tasks, create or update `design.md` and `implement.md` before implementation starts. + +Do not invent a project-specific product/spec hierarchy. If the repository already has product, domain, or spec docs, use them. If it does not, proceed with the evidence that exists. + +## Question Rules + +Ask only one question per message. + +Each question must include: + +- the decision needed +- why the answer matters +- your recommended answer +- the trade-off if the user chooses differently + +Do not ask process questions such as whether to search, inspect files, or continue brainstorming. Do the evidence work directly. Ask the user only when the remaining issue is a product decision, preference, scope boundary, or risk tolerance choice. + +## Artifact Rules + +`prd.md` records requirements and acceptance: + +- goal and user value +- confirmed facts +- requirements +- acceptance criteria +- out of scope +- open questions that still block planning + +`design.md` records technical design for complex tasks: + +- architecture and boundaries +- data flow and contracts +- compatibility and migration notes +- important trade-offs +- operational or rollback considerations + +`implement.md` records execution planning for complex tasks: + +- ordered implementation checklist +- validation commands +- risky files or rollback points +- follow-up checks before `task.py start` + +Lightweight tasks may have only `prd.md`. Complex tasks must have `prd.md`, `design.md`, and `implement.md` before `task.py start`. + +`implement.md` is not a replacement for `implement.jsonl`. Use JSONL files only for manifest-style spec and research references when the task needs them. + +## Quality Bar + +Before declaring planning ready: + +- `prd.md` contains testable acceptance criteria. +- Repository-answerable questions have already been answered through inspection. +- Remaining open questions are genuinely about user intent or scope. +- Complex tasks have `design.md` and `implement.md`. +- The user has reviewed the final planning artifacts or explicitly approved proceeding. + +Do not start implementation until the user approves or asks for implementation. diff --git a/.agents/skills/trellis-break-loop/SKILL.md b/.agents/skills/trellis-break-loop/SKILL.md new file mode 100644 index 0000000..ef2b50c --- /dev/null +++ b/.agents/skills/trellis-break-loop/SKILL.md @@ -0,0 +1,130 @@ +--- +name: trellis-break-loop +description: "Deep bug analysis to break the fix-forget-repeat cycle. Analyzes root cause category, why fixes failed, prevention mechanisms, and captures knowledge into specs. Use after fixing a bug to prevent the same class of bugs." +--- + +# Break the Loop - Deep Bug Analysis + +When debug is complete, use this for deep analysis to break the "fix bug -> forget -> repeat" cycle. + +--- + +## Analysis Framework + +Analyze the bug you just fixed from these 5 dimensions: + +### 1. Root Cause Category + +Which category does this bug belong to? + +| Category | Characteristics | Example | +|----------|-----------------|---------| +| **A. Missing Spec** | No documentation on how to do it | New feature without checklist | +| **B. Cross-Layer Contract** | Interface between layers unclear | API returns different format than expected | +| **C. Change Propagation Failure** | Changed one place, missed others | Changed function signature, missed call sites | +| **D. Test Coverage Gap** | Unit test passes, integration fails | Works alone, breaks when combined | +| **E. Implicit Assumption** | Code relies on undocumented assumption | Timestamp seconds vs milliseconds | + +### 2. Why Fixes Failed (if applicable) + +If you tried multiple fixes before succeeding, analyze each failure: + +- **Surface Fix**: Fixed symptom, not root cause +- **Incomplete Scope**: Found root cause, didn't cover all cases +- **Tool Limitation**: grep missed it, type check wasn't strict +- **Mental Model**: Kept looking in same layer, didn't think cross-layer + +### 3. Prevention Mechanisms + +What mechanisms would prevent this from happening again? + +| Type | Description | Example | +|------|-------------|---------| +| **Documentation** | Write it down so people know | Update thinking guide | +| **Architecture** | Make the error impossible structurally | Type-safe wrappers | +| **Compile-time** | Strict type checking, no escape hatches | Signature change causes compile error | +| **Runtime** | Monitoring, alerts, scans | Detect orphan entities | +| **Test Coverage** | E2E tests, integration tests | Verify full flow | +| **Code Review** | Checklist, PR template | "Did you check X?" | + +### 4. Systematic Expansion + +What broader problems does this bug reveal? + +- **Similar Issues**: Where else might this problem exist? +- **Design Flaw**: Is there a fundamental architecture issue? +- **Process Flaw**: Is there a development process improvement? +- **Knowledge Gap**: Is the team missing some understanding? + +### 5. Knowledge Capture + +Solidify insights into the system: + +- [ ] Update `.trellis/spec/guides/` thinking guides +- [ ] Update relevant `.trellis/spec/` docs +- [ ] Create issue record (if applicable) +- [ ] Create feature ticket for root fix +- [ ] Update check guidelines if needed + +--- + +## Output Format + +Please output analysis in this format: + +```markdown +## Bug Analysis: [Short Description] + +### 1. Root Cause Category +- **Category**: [A/B/C/D/E] - [Category Name] +- **Specific Cause**: [Detailed description] + +### 2. Why Fixes Failed (if applicable) +1. [First attempt]: [Why it failed] +2. [Second attempt]: [Why it failed] +... + +### 3. Prevention Mechanisms +| Priority | Mechanism | Specific Action | Status | +|----------|-----------|-----------------|--------| +| P0 | ... | ... | TODO/DONE | + +### 4. Systematic Expansion +- **Similar Issues**: [List places with similar problems] +- **Design Improvement**: [Architecture-level suggestions] +- **Process Improvement**: [Development process suggestions] + +### 5. Knowledge Capture +- [ ] [Documents to update / tickets to create] +``` + +--- + +## Core Philosophy + +> **The value of debugging is not in fixing the bug, but in making this class of bugs never happen again.** + +Three levels of insight: +1. **Tactical**: How to fix THIS bug +2. **Strategic**: How to prevent THIS CLASS of bugs +3. **Philosophical**: How to expand thinking patterns + +30 minutes of analysis saves 30 hours of future debugging. + +--- + +## After Analysis: Immediate Actions + +**IMPORTANT**: After completing the analysis above, you MUST immediately: + +1. **Update spec/guides** - Don't just list TODOs, actually update the relevant files: + - If it's a cross-platform issue → update `cross-platform-thinking-guide.md` + - If it's a cross-layer issue → update `cross-layer-thinking-guide.md` + - If it's a code reuse issue → update `code-reuse-thinking-guide.md` + - If it's domain-specific → update `backend/*.md` or `frontend/*.md` + +2. **Sync templates** - After updating `.trellis/spec/`, sync to `src/templates/markdown/spec/` + +3. **Commit the spec updates** - This is the primary output, not just the analysis text + +> **The analysis is worthless if it stays in chat. The value is in the updated specs.** diff --git a/.agents/skills/trellis-check/SKILL.md b/.agents/skills/trellis-check/SKILL.md new file mode 100644 index 0000000..856ee0d --- /dev/null +++ b/.agents/skills/trellis-check/SKILL.md @@ -0,0 +1,98 @@ +--- +name: trellis-check +description: "Comprehensive quality verification: spec compliance, lint, type-check, tests, cross-layer data flow, code reuse, and consistency checks. Use when code is written and needs quality verification, before committing changes, or to catch context drift during long sessions." +--- + +# Code Quality Check + +Comprehensive quality verification for recently written code. Combines spec compliance, cross-layer safety, and pre-commit checks. + +--- + +## Step 1: Identify What Changed + +```bash +git diff --name-only HEAD +git status +``` + +## Step 2: Read Task Artifacts and Applicable Specs + +Read the current task artifacts in order: + +- `prd.md` +- `design.md` if present +- `implement.md` if present + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +For each changed package/layer, read the spec index and follow its **Quality Check** section: + +```bash +cat .trellis/spec///index.md +``` + +Read the specific guideline files referenced — the index is a pointer, not the goal. + +## Step 3: Run Project Checks + +Run the project's lint, type-check, and test commands. Fix any failures before proceeding. + +## Step 4: Review Against Checklist + +### Code Quality + +- [ ] Linter passes? +- [ ] Type checker passes (if applicable)? +- [ ] Tests pass? +- [ ] No debug logging left in? +- [ ] No suppressed warnings or type-safety bypasses? + +### Test Coverage + +- [ ] New function → unit test added? +- [ ] Bug fix → regression test added? +- [ ] Changed behavior → existing tests updated? + +### Spec Sync + +- [ ] Does `.trellis/spec/` need updates? (new patterns, conventions, lessons learned) + +> "If I fixed a bug or discovered something non-obvious, should I document it so future me won't hit the same issue?" → If YES, update the relevant spec doc. + +## Step 5: Cross-Layer Dimensions (if applicable) + +Skip this step if your change is confined to a single layer. + +### A. Data Flow (changes touch 3+ layers) + +- [ ] Read flow traces correctly: Storage → Service → API → UI +- [ ] Write flow traces correctly: UI → API → Service → Storage +- [ ] Types/schemas correctly passed between layers? +- [ ] Errors properly propagated to caller? + +### B. Code Reuse (modifying constants, creating utilities) + +- [ ] Searched for existing similar code before creating new? + ```bash + grep -r "pattern" src/ + ``` +- [ ] If 2+ places define same value → extracted to shared constant? +- [ ] After batch modification, all occurrences updated? + +### C. Import/Dependency (creating new files) + +- [ ] Correct import paths (relative vs absolute)? +- [ ] No circular dependencies? + +### D. Same-Layer Consistency + +- [ ] Other places using the same concept are consistent? + +--- + +## Step 6: Report and Fix + +Report violations found and fix them directly. Re-run project checks after fixes. diff --git a/.agents/skills/trellis-continue/SKILL.md b/.agents/skills/trellis-continue/SKILL.md new file mode 100644 index 0000000..a83bb07 --- /dev/null +++ b/.agents/skills/trellis-continue/SKILL.md @@ -0,0 +1,61 @@ +--- +name: trellis-continue +description: "Resume work on the current task. Loads the workflow Phase Index, figures out which phase/step to pick up at, then pulls the step-level detail via get_context.py --mode phase. Use when coming back to an in-progress task and you need to know what to do next." +--- + +# Continue Current Task + +Resume work on the current task — pick up at the right phase/step in `.trellis/workflow.md`. + +--- + +## Step 1: Load Current Context + +```bash +python ./.trellis/scripts/get_context.py +``` + +Confirms: current task, git state, recent commits. + +## Step 2: Load the Phase Index + +```bash +python ./.trellis/scripts/get_context.py --mode phase +``` + +Shows the Phase Index (Plan / Execute / Finish) with routing + skill mapping. + +## Step 3: Decide Where You Are + +`get_context.py` shows the active task's `status` field. Route by `status` + artifact presence. This command replaces the user needing to remember the Trellis flow; it does not itself approve implementation. + +- `status=planning` + no `prd.md` → **1.1** (load `trellis-brainstorm`) +- `status=planning` + `prd.md` only → decide whether the task is lightweight or complex. Lightweight can move to **1.4** review; complex returns to **1.1** to add `design.md` + `implement.md`. +- `status=planning` + complex artifacts complete + sub-agent jsonl not curated (only the seed `_example` row) → **1.3** +- `status=planning` + required artifacts complete + required jsonl curated or inline mode → **1.4** (ask for start review; only run `task.py start` after user confirms) +- `status=in_progress` + implementation not started → **2.1** +- `status=in_progress` + implementation done, not yet checked → **2.2** +- `status=in_progress` + check passed → **3.1** +- `status=completed` (rare; usually archived immediately) → archive flow + +Phase rules (full detail in `.trellis/workflow.md`): + +1. Run steps **in order** within a phase — `[required]` steps must not be skipped +2. `[once]` steps are already done if the required output exists. `prd.md` alone can be enough only for lightweight tasks; complex tasks also need `design.md` and `implement.md`. +3. You may go back to an earlier phase if discoveries require it + +## Step 4: Load the Specific Step + +Once you know which step to resume at: + +```bash +python ./.trellis/scripts/get_context.py --mode phase --step --platform codex +``` + +Follow the loaded instructions. After each `[required]` step completes, move to the next. + +--- + +## Reference + +Full workflow and detailed phase steps live in `.trellis/workflow.md`. This command is only an entry point — the canonical guidance is there. diff --git a/.agents/skills/trellis-finish-work/SKILL.md b/.agents/skills/trellis-finish-work/SKILL.md new file mode 100644 index 0000000..02ad8f1 --- /dev/null +++ b/.agents/skills/trellis-finish-work/SKILL.md @@ -0,0 +1,71 @@ +--- +name: trellis-finish-work +description: "Wrap up the current session: verify quality gate passed, remind user to commit, archive completed tasks, and record session progress to the developer journal. Use when done coding and ready to end the session." +--- + +# Finish Work + +Wrap up the current session: archive the active task (and any other completed-but-unarchived tasks the user wants to clean up) and record the session journal. Code commits are NOT done here — those happen in workflow Phase 3.4 before you invoke this command. + +## Step 1: Survey current state + +```bash +python ./.trellis/scripts/get_context.py --mode record +``` + +This prints: + +- **My active tasks** — review whether any besides the current one are actually done (code merged, AC met) and should be archived this round. +- **Git status** — quick visual on what's dirty. +- **Recent commits** — you'll need their hashes in Step 4 for `--commit`. + +If `--mode record` surfaces other completed tasks not tied to the current session, surface them to the user with a one-shot confirmation: "These N tasks look done — archive them too in this round? [y/N]". Default is no; the current active task is always archived in Step 3 regardless. + +## Step 2: Sanity check — classify dirty paths + +Run: + +```bash +git status --porcelain +``` + +Filter out paths under `.trellis/workspace/` and `.trellis/tasks/` — those are managed by `add_session.py` and `task.py archive` auto-commits and will appear dirty as part of this skill's own work. + +For each remaining dirty path, decide whether it belongs to **the current task** or to **other parallel work** (e.g., another terminal window editing the same repo). Heuristics: + +- Paths referenced in the current task's `prd.md` / `implement.jsonl` / `check.jsonl` → current task +- Paths in code areas matching the task's stated scope, or that you remember editing this session → current task +- Paths in unrelated areas you have no recollection of touching this session → other parallel work + +Then route: + +- **Any remaining path looks like current-task work** — bail out with: + > "Working tree has uncommitted code changes from this task: ``. Return to workflow Phase 3.4 to commit them before running ``finish-work` (Trellis command)`." + + Do NOT run `git commit` here. Do NOT prompt the user to commit. The user goes back to Phase 3.4 and the AI drives the batched commit there. +- **All remaining paths look unrelated** (other parallel-window work) — report them once and continue to Step 3: + > "FYI, dirty files outside this task's scope — leaving them for the other window: ``." +- **Genuinely unsure** — ask the user once: "Are `` this task's work I forgot to commit, or another window's? (commit / ignore)" — then route per their answer. + +## Step 3: Archive task(s) + +```bash +python ./.trellis/scripts/task.py archive +``` + +At minimum: the current active task (if any). Plus any extra tasks the user confirmed in Step 1. Each archive produces a `chore(task): archive ...` commit via the script's auto-commit. + +If there is no active task and the user did not confirm any cleanup archives, skip this step. + +## Step 4: Record session journal + +```bash +python ./.trellis/scripts/add_session.py \ + --title "Session Title" \ + --commit "hash1,hash2" \ + --summary "Brief summary" +``` + +Use the work-commit hashes produced in Phase 3.4 (visible in Step 1's `Recent commits` list, or via `git log --oneline`) for `--commit`. Do not include the archive commit hashes from Step 3. This produces a `chore: record journal` commit. + +Final git log order: `` → `chore(task): archive ...` (one or more) → `chore: record journal`. diff --git a/.agents/skills/trellis-meta/SKILL.md b/.agents/skills/trellis-meta/SKILL.md new file mode 100644 index 0000000..590bfac --- /dev/null +++ b/.agents/skills/trellis-meta/SKILL.md @@ -0,0 +1,73 @@ +--- +name: trellis-meta +description: "Understand and customize the local Trellis architecture inside a user project. Use when modifying .trellis plus platform hooks, settings, agents, skills, commands, prompts, or workflows generated by trellis init." +--- + +# Trellis Meta + +This skill is for local Trellis users who have already run `trellis init` in a project. After reading it, an AI should understand the Trellis architecture, operating model, and customization entry points inside that user project, then modify the generated `.trellis/` and platform directory files according to the user's request. + +The default operating scope is local files in the user project: + +- `.trellis/`: workflow, config, tasks, spec, workspace, scripts, and runtime state. +- Platform directories: `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, `.kiro/`, `.gemini/`, `.qoder/`, `.codebuddy/`, `.github/`, `.factory/`, `.pi/`, `.kilocode/`, `.agent/`, `.windsurf/`, and similar directories. +- Shared skill layer: `.agents/skills/`. + +Do not assume the user has the Trellis source repository. Do not default to modifying the global npm install directory or `node_modules`. + +## How To Use + +1. Read `references/local-architecture/overview.md` first to establish the local Trellis system model. +2. If the request involves a specific AI tool, read `references/platform-files/platform-map.md` and the relevant platform file notes. +3. If the user wants to change behavior, read `references/customize-local/overview.md`, then open the specific customization topic. +4. Before editing, read the actual files in the user project and treat local content as authoritative. + +## References + +### Local Architecture + +- `references/local-architecture/overview.md`: The three-layer local Trellis architecture and customization principles. +- `references/local-architecture/generated-files.md`: Files generated by `trellis init` and their customization boundaries. +- `references/local-architecture/workflow.md`: Phases, routing, and workflow-state blocks in `.trellis/workflow.md`. +- `references/local-architecture/task-system.md`: Task directories, active tasks, JSONL context, and task runtime. +- `references/local-architecture/spec-system.md`: How `.trellis/spec/` is organized and injected. +- `references/local-architecture/workspace-memory.md`: `.trellis/workspace/`, journals, and cross-session memory. +- `references/local-architecture/context-injection.md`: Hooks, sub-agent preludes, and context injection paths. + +### Platform Files + +- `references/platform-files/overview.md`: How shared `.trellis/` files relate to platform directories. +- `references/platform-files/platform-map.md`: Platform directories and paths for skills, agents, hooks, and extensions. +- `references/platform-files/hooks-and-settings.md`: How settings/config files, hooks, plugins, and extensions connect to Trellis. +- `references/platform-files/agents.md`: Local file responsibilities for `trellis-research`, `trellis-implement`, and `trellis-check`. +- `references/platform-files/skills-and-commands.md`: Differences between skills, commands, prompts, and workflows, plus how to change them. + +### Local Customization + +- `references/customize-local/overview.md`: Choose the right local customization entry point for the user's request. +- `references/customize-local/change-workflow.md`: Change phases, routing, next actions, and workflow-state. +- `references/customize-local/change-task-lifecycle.md`: Change task creation, status, archive behavior, and hooks. +- `references/customize-local/change-context-loading.md`: Change how tasks, specs, journals, and hook context are loaded. +- `references/customize-local/change-hooks.md`: Change platform hooks, settings, and shell session bridges. +- `references/customize-local/change-agents.md`: Change research, implement, and check agent behavior. +- `references/customize-local/change-skills-or-commands.md`: Add or modify local skills, commands, prompts, and workflows. +- `references/customize-local/change-spec-structure.md`: Adjust the project spec structure under `.trellis/spec/`. +- `references/customize-local/add-project-local-conventions.md`: Put team rules into project-local specs or local skills. + +## Current Rules + +- `.trellis/workflow.md` is the local workflow source of truth. +- `.trellis/config.yaml` is the project-level Trellis configuration and task hook configuration entry point. +- `.trellis/spec/` stores the user's project-specific coding conventions and design constraints. +- `.trellis/tasks/` stores task PRDs, technical notes, research files, and JSONL context. +- `.trellis/workspace/` stores developer journals and cross-session memory. +- Platform settings/config files decide which hooks, agents, skills, commands, prompts, and workflows actually run. +- `.trellis/.template-hashes.json` and `.trellis/.runtime/` are management/runtime state files. Confirm necessity before editing them. + +## Do Not + +- Do not treat Trellis upstream source code as the default target for local customization. +- Do not modify the global npm install directory or `node_modules/@mindfoldhq/trellis` to implement project needs. +- Do not overwrite user-modified local files with default templates. +- Do not put team-private project rules into the public `trellis-meta`; put project rules in `.trellis/spec/` or a project-local skill. +- Do not describe removed historical mechanisms as current Trellis behavior. diff --git a/.agents/skills/trellis-meta/references/customize-local/add-project-local-conventions.md b/.agents/skills/trellis-meta/references/customize-local/add-project-local-conventions.md new file mode 100644 index 0000000..d32ca2d --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/add-project-local-conventions.md @@ -0,0 +1,83 @@ +# Add Project-Local Conventions + +Often the user does not need to change Trellis mechanics; they need local AI to understand their team's conventions. In that case, prefer `.trellis/spec/` or a project-local skill instead of editing `trellis-meta`. + +## Where To Put Things + +| Content type | Location | +| --- | --- | +| Rules code must follow | `.trellis/spec//` | +| Cross-layer thinking methods | `.trellis/spec/guides/` | +| AI capability for a project-specific flow | Platform-local skill | +| One-off task material | `.trellis/tasks//` | +| Session summary | `.trellis/workspace//journal-N.md` | + +## Create A Project-Local Skill + +If the user wants AI to know "how this project customizes Trellis," create a local skill: + +```text +.claude/skills/trellis-local/ +└── SKILL.md +``` + +Example: + +```md +--- +name: trellis-local +description: "Project-local Trellis customizations for this repository. Use when changing this project's Trellis workflow, hooks, local agents, or team-specific conventions." +--- + +# Trellis Local + +## Local Scope + +This skill documents this repository's Trellis customizations only. + +## Custom Workflow Rules + +- ... + +## Local Hook Changes + +- ... + +## Local Agent Changes + +- ... +``` + +For multi-platform projects, place equivalent versions in other platform skill directories, or use `.agents/skills/` for platforms that support the shared layer. + +## Write To `.trellis/spec/` + +If the content is a coding convention, write it to spec. Examples: + +```text +.trellis/spec/backend/error-handling.md +.trellis/spec/frontend/components.md +.trellis/spec/guides/cross-platform-thinking-guide.md +``` + +After writing it, update the corresponding `index.md` so AI can find the new rule from the entry point. + +## Make The Current Task Use New Conventions + +After writing a spec, add it to the current task context: + +```bash +python ./.trellis/scripts/task.py add-context implement ".trellis/spec/backend/error-handling.md" "Error handling conventions" +python ./.trellis/scripts/task.py add-context check ".trellis/spec/backend/error-handling.md" "Review error handling" +``` + +## Do Not Store Project-Private Rules In `trellis-meta` + +`trellis-meta` is a public skill for understanding Trellis architecture and local customization entry points. Put project-private content in: + +- `.trellis/spec/` +- a project-local skill +- the current task +- workspace journal + +This prevents future updates to Trellis's built-in `trellis-meta` from overwriting the team's own conventions. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-agents.md b/.agents/skills/trellis-meta/references/customize-local/change-agents.md new file mode 100644 index 0000000..9b63531 --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-agents.md @@ -0,0 +1,54 @@ +# Change Local Agents + +When the user wants to change `trellis-research`, `trellis-implement`, or `trellis-check` behavior, edit platform agent files in the user project. + +## Read These Files First + +1. Target platform agent directory +2. `.trellis/workflow.md` Phase 2 / research routing +3. Current task `prd.md` +4. Current task `implement.jsonl` / `check.jsonl` +5. Relevant hook or agent prelude + +## Common Paths + +| Platform | Path | +| --- | --- | +| Claude Code | `.claude/agents/trellis-*.md` | +| Cursor | `.cursor/agents/trellis-*.md` | +| OpenCode | `.opencode/agents/trellis-*.md` | +| Codex | `.codex/agents/trellis-*.toml` | +| Kiro | `.kiro/agents/trellis-*.json` | +| Gemini CLI | `.gemini/agents/trellis-*.md` | +| Qoder | `.qoder/agents/trellis-*.md` | +| CodeBuddy | `.codebuddy/agents/trellis-*.md` | +| Factory Droid | `.factory/droids/trellis-*.md` | +| Pi Agent | `.pi/agents/trellis-*.md` | + +Use the actual paths in the user project as authoritative. + +## Common Needs + +| Need | Which agent to edit | +| --- | --- | +| Research must write files, not only reply in chat | `trellis-research` | +| Certain local specs must be read before implementation | `trellis-implement` + `implement.jsonl` configuration rules | +| Specific commands must run during checking | `trellis-check` | +| Agent must not modify certain directories | The corresponding agent's write boundary instructions | +| Agent output format must be fixed | The corresponding agent's final/reporting instructions | + +## Modification Principles + +1. **Preserve role boundaries**: research investigates and persists; implement writes implementation; check reviews and fixes. +2. **Do not hard-code project specs into agents**: long-term specs belong in `.trellis/spec/`; agents are responsible for reading them. +3. **Make read order explicit**: active task -> PRD -> info -> JSONL -> spec/research. +4. **Make write boundaries explicit**: which directories may be written and which may not. +5. **Synchronize across platforms**: when the user configured multiple platforms, decide whether to change only the current platform or all platform agents. + +## Agent Pull Platforms + +If an agent file contains a prelude for "read task/context after startup," do not remove those steps when editing. Otherwise the agent will work only from chat context and bypass Trellis's core mechanism. + +## Hook Push Platforms + +If context is injected by a hook, the agent file should still retain responsibility boundaries. Do not remove PRD/spec requirements from the agent just because a hook injects context. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-context-loading.md b/.agents/skills/trellis-meta/references/customize-local/change-context-loading.md new file mode 100644 index 0000000..83bcd63 --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-context-loading.md @@ -0,0 +1,84 @@ +# Change Local Context Loading + +Context loading determines when AI reads workflow, task, spec, research, workspace, and git status. Read this page when the user says "AI does not know the current task," "the agent did not read specs," or "there is too much/too little context." + +## Read These Files First + +1. `.trellis/workflow.md` +2. `.trellis/scripts/get_context.py` +3. `.trellis/scripts/common/session_context.py` +4. `.trellis/scripts/common/task_context.py` +5. `.trellis/scripts/common/active_task.py` +6. Current platform hooks or agent files +7. The current task's `implement.jsonl` / `check.jsonl` + +## Context Sources + +| Source | Purpose | +| --- | --- | +| `.trellis/workflow.md` | Workflow and next-action hints. | +| `.trellis/tasks//prd.md` | Current task requirements. | +| `.trellis/tasks//design.md` | Complex task technical design. | +| `.trellis/tasks//implement.md` | Complex task execution plan. | +| `.trellis/tasks//implement.jsonl` | Spec/research to read before implementation. | +| `.trellis/tasks//check.jsonl` | Spec/research to read during checking. | +| `.trellis/spec/` | Project specs. | +| `.trellis/workspace/` | Session records. | +| git status | Current working tree changes. | + +## Common Needs And Edit Points + +| Need | Edit point | +| --- | --- | +| Inject more/less information in new sessions | `session_context.py` or the platform `session-start` hook. | +| Change hints on each user input | `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The `inject-workflow-state` hook is parser-only and reads the block verbatim. | +| Agent did not read specs | Task JSONL, agent prelude, `inject-subagent-context` hook. | +| Active task is lost | `active_task.py` and platform session identity propagation. | +| Change JSONL validation rules | `task_context.py`. | + +## JSONL Rules + +`implement.jsonl` / `check.jsonl` are the key context loading interface: + +```jsonl +{"file": ".trellis/spec/backend/index.md", "reason": "Backend conventions"} +{"file": ".trellis/tasks/04-28-x/research/api.md", "reason": "API research"} +``` + +Include only spec/research files. Do not put code files that will be modified into these manifests; agents read code files themselves during implementation. + +## Change Session Context + +If the user wants every new session to see more project state, edit: + +- `.trellis/scripts/common/session_context.py` +- the corresponding platform `session-start` hook + +Context cannot grow without bound. Prefer injecting indexes and paths so the AI can read detailed files on demand. + +## Change Sub-Agent Context + +First determine which mode the platform uses: + +- hook push: edit the `inject-subagent-context` hook. +- agent pull: edit the read steps in the corresponding `trellis-implement` / `trellis-check` agent file. + +In both modes, make sure the agent ultimately reads: + +1. active task +2. the corresponding JSONL +3. spec/research referenced by the JSONL +4. `prd.md` +5. `design.md` if present +6. `implement.md` if present + +## Troubleshooting Order + +```bash +python ./.trellis/scripts/task.py current --source +python ./.trellis/scripts/task.py list-context +python ./.trellis/scripts/task.py validate +python ./.trellis/scripts/get_context.py --mode packages +``` + +Confirm the task and JSONL are correct before editing hooks/agents. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-hooks.md b/.agents/skills/trellis-meta/references/customize-local/change-hooks.md new file mode 100644 index 0000000..093a171 --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-hooks.md @@ -0,0 +1,57 @@ +# Change Local Hooks + +Hooks are the automation layer that connects a platform to Trellis. When the user wants to change "when context is injected," "how shell commands inherit a session," or "which files are read before an agent starts," hooks are usually the edit point. + +## Read These Files First + +1. Target platform settings/config, such as `.claude/settings.json`, `.codex/hooks.json`, `.cursor/hooks.json` +2. Target platform hooks directory +3. `.trellis/scripts/common/active_task.py` +4. `.trellis/scripts/common/session_context.py` +5. `.trellis/workflow.md` + +## Common Hook Types + +| Hook | Purpose | +| --- | --- | +| session-start | Injects a Trellis overview when a session starts, clears, or compacts. | +| workflow-state | Injects a state hint on each user input. | +| sub-agent context | Injects PRD/spec/research before an agent starts. | +| shell session bridge | Lets `task.py` commands in shell see the same session identity. | + +## Modification Steps + +1. Find the hook registration in settings/config. +2. Confirm the registered script path exists. +3. Read the hook script and identify inputs, outputs, and called `.trellis/scripts/`. +4. Modify hook behavior. +5. If the hook depends on workflow content, synchronize `.trellis/workflow.md`. + +## Example: Change New-Session Injection Content + +First find the session-start hook: + +```text +.claude/settings.json +.claude/hooks/session-start.py +``` + +If the hook ultimately calls `.trellis/scripts/get_context.py` or `session_context.py`, editing the local script is usually more robust than hard-coding content in the hook. + +## Example: Agent Did Not Read JSONL + +First confirm: + +```bash +python ./.trellis/scripts/task.py current --source +python ./.trellis/scripts/task.py validate +``` + +If the task and JSONL are correct, determine whether the platform uses hook push or agent pull. For hook push, edit `inject-subagent-context`; for agent pull, edit the agent file. + +## Notes + +- Settings handle registration, hook scripts handle behavior; inspect both together. +- Different platforms support different hook events. Do not directly copy another platform's settings. +- Hooks should read project-local `.trellis/`; they should not depend on Trellis upstream source paths. +- Hook failures should produce visible errors so AI does not silently lose context. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-skills-or-commands.md b/.agents/skills/trellis-meta/references/customize-local/change-skills-or-commands.md new file mode 100644 index 0000000..84590a1 --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-skills-or-commands.md @@ -0,0 +1,78 @@ +# Change Local Skills, Commands, Prompts, And Workflows + +When the user wants to change AI entry points, auto-trigger rules, or explicit command behavior, edit skills, commands, prompts, or workflows in local platform directories. + +## Read These Files First + +1. `.trellis/workflow.md` +2. Target platform skill/command/prompt/workflow directory +3. Related agent or hook files +4. Whether project rules already exist in `.trellis/spec/` + +## Which Entry Type To Choose + +| Goal | Recommendation | +| --- | --- | +| AI should automatically know a capability | Add or modify a skill. | +| User wants to trigger manually with a command | Add or modify a command/prompt/workflow. | +| Team project conventions | Prefer `.trellis/spec/` or a project-local skill. | +| Change Trellis flow semantics | Synchronize `.trellis/workflow.md`. | + +## Modify A Skill + +A skill is usually: + +```text +/ +├── SKILL.md +└── references/ +``` + +`SKILL.md` should be short and responsible for triggering/routing. Put long content in `references/` so AI can read it on demand. + +The frontmatter description should specify when to use the skill. Example: + +```yaml +description: "Use when customizing this project's deployment workflow and release checklist." +``` + +Do not write vague descriptions such as "helpful project skill"; they can trigger incorrectly. + +## Modify A Command/Prompt/Workflow + +Explicit entry points should state: + +- How the user triggers it. +- Which `.trellis/` files to read. +- Which scripts to run. +- How to report after completion. + +If a command only repeats workflow rules, prefer making it reference/read `.trellis/workflow.md` instead of maintaining a second copy of the flow. + +## Common Paths + +| Platform | Entry directories | +| --- | --- | +| Claude Code | `.claude/skills/`, `.claude/commands/` | +| Cursor | `.cursor/skills/`, `.cursor/commands/` | +| OpenCode | `.opencode/skills/`, `.opencode/commands/` | +| Codex | `.agents/skills/`, `.codex/skills/` | +| GitHub Copilot | `.github/skills/`, `.github/prompts/` | +| Kilo / Antigravity / Windsurf | workflows + skills | + +## Add A Project-Local Skill + +If the user wants to document team-private customizations, create a project-local skill, for example: + +```text +.claude/skills/project-trellis-local/ +└── SKILL.md +``` + +For multi-platform projects, add equivalent versions in each platform skill directory, or use `.agents/skills/` on platforms that support the shared layer. + +## Notes + +- Do not mix every platform's syntax into one file. +- Do not change only one platform entry point while claiming all platforms are supported. +- Do not hide long-term engineering conventions inside a command; write them to `.trellis/spec/`. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-spec-structure.md b/.agents/skills/trellis-meta/references/customize-local/change-spec-structure.md new file mode 100644 index 0000000..14e0cd8 --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-spec-structure.md @@ -0,0 +1,83 @@ +# Change Local Spec Structure + +When the user wants to change the engineering conventions AI follows, add new spec layers, or adjust monorepo package mapping, edit `.trellis/spec/` and `.trellis/config.yaml`. + +## Read These Files First + +1. `.trellis/config.yaml` +2. `.trellis/spec/` +3. `.trellis/workflow.md` planning artifact guidance and Phase 3.3 +4. Current task `implement.jsonl` / `check.jsonl` + +## Common Needs + +| Need | Edit location | +| --- | --- | +| Add backend/frontend/docs/test spec layer | `.trellis/spec//` or `.trellis/spec///` | +| Add shared thinking guides | `.trellis/spec/guides/` | +| Adjust monorepo packages | `packages` in `.trellis/config.yaml` | +| Change default package | `default_package` in `.trellis/config.yaml` | +| Control spec scanning scope | `spec_scope` in `.trellis/config.yaml` | +| Make a task read a new spec | Task `implement.jsonl` / `check.jsonl` | + +## Add A Spec Layer + +Single-repository example: + +```text +.trellis/spec/security/ +├── index.md +└── auth.md +``` + +Monorepo example: + +```text +.trellis/spec/webapp/security/ +├── index.md +└── auth.md +``` + +`index.md` should include: + +- What code this layer applies to. +- Pre-Development Checklist. +- Quality Check. +- Links to specific guideline files. + +## Update Context + +Adding a spec does not mean every task automatically reads it. The current task must reference it in JSONL: + +```bash +python ./.trellis/scripts/task.py add-context implement ".trellis/spec/webapp/security/index.md" "Security conventions" +python ./.trellis/scripts/task.py add-context check ".trellis/spec/webapp/security/index.md" "Security review rules" +``` + +## Change Monorepo Packages + +Example `.trellis/config.yaml`: + +```yaml +packages: + webapp: + path: apps/web + api: + path: apps/api +default_package: webapp +``` + +After editing, run: + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +Use this output to confirm AI can see the correct packages and spec layers. + +## Notes + +- Specs are user project conventions and can be changed according to project needs. +- Do not put temporary task information into specs; put temporary information in the task. +- Do not put long-term conventions only in agents or commands; preserve them in specs. +- After changing spec structure, check whether existing task JSONL files still point to files that exist. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-task-lifecycle.md b/.agents/skills/trellis-meta/references/customize-local/change-task-lifecycle.md new file mode 100644 index 0000000..208e0da --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-task-lifecycle.md @@ -0,0 +1,90 @@ +# Change Local Task Lifecycle + +Task lifecycle includes creation, start, context configuration, finish, archive, parent/child tasks, and lifecycle hooks. The default customization targets are `.trellis/tasks/`, `.trellis/config.yaml`, and `.trellis/scripts/`. + +## Read These Files First + +1. `.trellis/workflow.md` +2. `.trellis/config.yaml` +3. `.trellis/scripts/task.py` +4. `.trellis/scripts/common/task_store.py` +5. `.trellis/scripts/common/task_utils.py` +6. The current task's `.trellis/tasks//task.json` + +## Common Needs And Edit Points + +| Need | Edit point | +| --- | --- | +| Automatically sync an external system after task creation | `hooks.after_create` in `.trellis/config.yaml`. | +| Automatically update status after task start | `hooks.after_start` in `.trellis/config.yaml`. | +| Run a script after task finish | `hooks.after_finish` in `.trellis/config.yaml`. | +| Clean external resources after archive | `hooks.after_archive` in `.trellis/config.yaml`. | +| Change default task fields | `.trellis/scripts/common/task_store.py`. | +| Change task parsing/search | `.trellis/scripts/common/task_utils.py`. | +| Change active task behavior | `.trellis/scripts/common/active_task.py`. | + +## lifecycle hooks + +`.trellis/config.yaml` supports: + +```yaml +hooks: + after_create: + - "python .trellis/scripts/hooks/my_sync.py create" + after_start: + - "python .trellis/scripts/hooks/my_sync.py start" + after_finish: + - "python .trellis/scripts/hooks/my_sync.py finish" + after_archive: + - "python .trellis/scripts/hooks/my_sync.py archive" +``` + +Hook commands receive the `TASK_JSON_PATH` environment variable, pointing to the current task's `task.json`. Hook failures should usually warn, but not block the main task operation. + +## Change Task Fields + +If the user wants to add project-local fields, prefer putting them under `meta` in `task.json` to avoid breaking existing scripts' assumptions about standard fields. + +Example: + +```json +"meta": { + "linearIssue": "ENG-123", + "risk": "high" +} +``` + +If standard fields really need to change, inspect every local script that reads `task.json`. + +## Change Active Task + +Active task is session-level state stored in `.trellis/.runtime/sessions/`. Do not fall back to a global `.current-task` model. If the user wants to change active task behavior, edit: + +- `.trellis/scripts/common/active_task.py` +- platform hooks or shell session bridges +- active task descriptions in `.trellis/workflow.md` + +### `task.py create` Sets the Active Pointer + +`cmd_create` in `.trellis/scripts/common/task_store.py` calls `set_active_task` best-effort right after writing the new task directory. The behavior: + +- When the calling shell carries session identity (`TRELLIS_CONTEXT_ID` env var, or any platform-specific session env that `resolve_context_key` recognizes — see `active_task.py:_ENV_SESSION_KEYS`), the per-session pointer at `.trellis/.runtime/sessions/.json` is rewritten to point at the new task. The task's `status=planning` and `[workflow-state:planning]` fires on the very next `UserPromptSubmit`. +- When session identity is unavailable (raw CLI invocation outside an AI session, or a platform that doesn't propagate identity to shell), the task directory is still created and `status=planning` is still written, but the active pointer is left untouched. The user can attach the task later with `task.py start ` once they're back in an AI session. + +This makes `[workflow-state:planning]` the live breadcrumb during the brainstorm and JSONL curation work that follows `task.py create`. The pre-R7 behavior left the breadcrumb stuck on `no_task` until `task.py start`, so the planning block was effectively dead text. + +If you fork `task.py` to add a new creation path (e.g. an external import that bypasses `cmd_create`), audit whether your path also calls `set_active_task`. Without that call, your created tasks will not surface as active. The full status writer table is in `.trellis/spec/cli/backend/workflow-state-contract.md`. + +## Modification Steps + +1. Confirm the current task with `python ./.trellis/scripts/task.py current --source`. +2. Read the current task's `task.json` and confirm status and fields. +3. For configuration needs, edit `.trellis/config.yaml` first. +4. For script behavior needs, then edit `.trellis/scripts/`. +5. If the AI flow changed, synchronize `.trellis/workflow.md`. + +## Do Not + +- Do not directly edit `.trellis/.runtime/sessions/` to "fix" business state. +- Do not hard-code project-private fields into scripts; prefer `meta`. +- Do not default to asking the user to fork Trellis CLI. diff --git a/.agents/skills/trellis-meta/references/customize-local/change-workflow.md b/.agents/skills/trellis-meta/references/customize-local/change-workflow.md new file mode 100644 index 0000000..aa2e663 --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/change-workflow.md @@ -0,0 +1,65 @@ +# Change Local Workflow + +When the user wants to change Trellis phases, next-action hints, whether to create tasks, whether to use sub-agents, or when to check/wrap up, edit `.trellis/workflow.md` first. + +## Read These Files First + +1. `.trellis/workflow.md` +2. Entry files for the current platform, such as skills/commands/prompts/workflows +3. The current task's `task.json` and `prd.md` + +## Common Needs And Edit Points + +| Need | Edit point | +| --- | --- | +| Change phase names or phase order | `Phase Index` and the corresponding Phase sections. | +| Change whether to create a task when there is no task | `[workflow-state:no_task]` state block. | +| Change the next step during planning | Phase 1 and `[workflow-state:planning]`. | +| Change whether an agent is required during in_progress | Phase 2 and `[workflow-state:in_progress]`. | +| Change wrap-up after completion | Phase 3 and `[workflow-state:completed]`. | +| Change which skill a user intent triggers | `Skill Routing` table. | + +## Modification Steps + +1. Find the relevant section in `.trellis/workflow.md`. +2. When changing rules, keep explicit trigger conditions and next actions. +3. If adding or renaming a skill/agent, synchronize the corresponding files in platform directories. +4. Workflow-state changes only need an edit to the `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The hook is parser-only — it reads whatever you put in the block. Keep the opening and closing tags' STATUS strings identical (`[workflow-state:foo]…[/workflow-state:foo]`); mismatched STATUS pairs are silently dropped. +5. Make the AI reread `.trellis/workflow.md`; do not keep using rules from the old conversation. + +## Example: Relax Task Creation Requirements + +To change when task creation can be skipped, usually edit `[workflow-state:no_task]`: + +```md +[workflow-state:no_task] +Task is not required when the answer is a one-reply explanation, no files are changed, and no research is needed. +[/workflow-state:no_task] +``` + +If the formal Phase 1 flow also needs to change, synchronize the Phase 1 section. + +## Example: One Platform Does Not Use Sub-Agents + +If the user wants only one platform to avoid sub-agents, first confirm whether that platform has a separate group in the workflow. Then change Phase 2 routing for that platform group instead of deleting all `trellis-implement` / `trellis-check` instructions across platforms. + +## `/trellis:continue` Route Table + +`/trellis:continue` resumes a task by deciding which phase step to load next. The decision combines `task.json.status` with the presence of artifacts inside the task directory. The mapping is fixed in the command itself; forks that add custom statuses must extend both the workflow.md tag block and this table. + +| `status` | Artifact state | Resume at | +| --- | --- | --- | +| `planning` | `prd.md` missing | Phase 1.1 (load `trellis-brainstorm`) | +| `planning` | lightweight task with `prd.md` complete | ask for start review, then run `task.py start` | +| `planning` | complex task missing `design.md` or `implement.md` | complete missing planning artifacts | +| `planning` | complex task has `prd.md`, `design.md`, and `implement.md` | ask for start review, then run `task.py start` | +| `in_progress` | no implementation in conversation history | Phase 2.1 (`trellis-implement`) | +| `in_progress` | implementation done, no `trellis-check` run | Phase 2.2 (`trellis-check`) | +| `in_progress` | check passed | Phase 3.1 (verify quality + spec update) | +| `completed` | task is still in active tree | Phase 3.5 (run `/trellis:finish-work` to archive) | + +When you add a custom status (e.g. `in-review`), add a `[workflow-state:in-review]` block in `.trellis/workflow.md` for the per-turn breadcrumb AND extend this route table — usually by editing the `/trellis:continue` command file (`.{platform}/commands/trellis/continue.md` or equivalent) to add a row that decides where to resume from. Without the route entry, `/trellis:continue` will fall through to a default branch and the user will not land on the step you intended. + +## Notes + +`.trellis/workflow.md` is the local project workflow, not an immutable template. The user can adapt it to team habits. After editing it, platform entry files may still contain old descriptions, so inspect them too. diff --git a/.agents/skills/trellis-meta/references/customize-local/overview.md b/.agents/skills/trellis-meta/references/customize-local/overview.md new file mode 100644 index 0000000..ac16a4c --- /dev/null +++ b/.agents/skills/trellis-meta/references/customize-local/overview.md @@ -0,0 +1,55 @@ +# Local Customization Overview + +This directory is for local AI working in a user project where Trellis was installed through npm and `trellis init` has already been run. The AI should modify generated `.trellis/` and platform directories inside the project, not Trellis CLI upstream source code. + +## First Determine What The User Actually Wants To Change + +| User wording | Read first | +| --- | --- | +| "Change the Trellis flow / phases / next prompt" | `change-workflow.md` | +| "Change task creation, status, archive, or hooks" | `change-task-lifecycle.md` | +| "AI did not read context / change injected content" | `change-context-loading.md` | +| "A platform hook is not behaving as expected" | `change-hooks.md` | +| "Change implement/check/research agent behavior" | `change-agents.md` | +| "Add a skill/command/workflow/prompt" | `change-skills-or-commands.md` | +| "Adjust the project spec structure" | `change-spec-structure.md` | +| "Add team conventions and local notes" | `add-project-local-conventions.md` | + +## General Operation Order + +1. **Confirm platform and directories**: inspect which directories exist, such as `.claude/`, `.codex/`, `.cursor/`. +2. **Confirm the current active task**: run `python ./.trellis/scripts/task.py current --source`. +3. **Read the local source of truth**: prefer `.trellis/workflow.md`, `.trellis/config.yaml`, and relevant platform files. +4. **Modify narrowly**: edit only files related to the user's request. +5. **Synchronize semantics**: if a shared flow changes, check whether platform entry points also need changes; if a platform entry changes, check whether `.trellis/workflow.md` still agrees. + +## Local File Priority + +| Layer | Files | +| --- | --- | +| Workflow | `.trellis/workflow.md` | +| Project configuration | `.trellis/config.yaml` | +| Task material | `.trellis/tasks//` | +| Project specs | `.trellis/spec/` | +| Runtime scripts | `.trellis/scripts/` | +| Platform integration | `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, and similar directories | +| Shared skill | `.agents/skills/` | + +## Things Not To Do By Default + +- Do not edit the global npm install directory. +- Do not edit `node_modules/@mindfoldhq/trellis`. +- Do not assume the user has the Trellis GitHub repository. +- Do not overwrite local files already modified by the user with default templates. +- Do not put team project rules into public `trellis-meta`; project rules belong in `.trellis/spec/` or a local skill. + +## When To Inspect Upstream Source + +Switch to an upstream source-code perspective only when the user explicitly expresses one of these goals: + +- "I want to open a PR to Trellis" +- "I want to change npm package publish contents" +- "I want to fork Trellis" +- "I want to modify the generation logic for `trellis init/update`" + +Otherwise, default to modifying local Trellis files inside the user project. diff --git a/.agents/skills/trellis-meta/references/local-architecture/context-injection.md b/.agents/skills/trellis-meta/references/local-architecture/context-injection.md new file mode 100644 index 0000000..4a7517b --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/context-injection.md @@ -0,0 +1,68 @@ +# Local Context Injection System + +Trellis context injection aims to make AI read the right files at the right time instead of relying on model memory. In a user project, injection is implemented by `.trellis/` scripts together with platform hooks, agents, and skills. + +## Injected Context Types + +| Type | Source | Purpose | +| --- | --- | --- | +| session context | `.trellis/scripts/get_context.py` | Current developer, git status, active task, active tasks, journal, packages. | +| workflow context | `.trellis/workflow.md` | Current Trellis flow and next action. | +| spec context | `.trellis/spec/` + task JSONL | Specs that must be followed during implementation/checking. | +| task context | `.trellis/tasks//prd.md`, `design.md`, `implement.md`, `research/` | Current task requirements, design, execution plan, and research. | +| platform context | Platform hooks/settings/agents | Lets different AI tools read the files above through their own mechanisms. | + +## session-start + +Platforms with session-start support inject a Trellis overview when a session starts, clears, compacts, or receives a similar event. Injected content usually includes: + +- workflow summary. +- current task status. +- active tasks. +- spec index paths. +- developer identity and git status. + +If the user feels the AI does not know the current task in a new session, first check whether the platform's session-start hook or equivalent mechanism is installed and running. + +## workflow-state + +workflow-state is a lightweight hint injected around each user turn. Based on current task status, it selects a block from `.trellis/workflow.md`, such as `no_task`, `planning`, `in_progress`, or `completed`. + +If the user wants to change "what the AI should do next in a given state," edit the corresponding state block in `.trellis/workflow.md` first. + +## sub-agent context + +Implement and check agents need task context. Trellis has two loading modes: + +1. **hook push**: a platform hook injects jsonl-referenced files plus `prd.md`, `design.md` if present, and `implement.md` if present before the agent starts. +2. **agent pull**: the agent definition instructs the agent to read the active task, jsonl context, and task artifacts after startup. + +In both modes, JSONL files in the task directory are the manifest for spec/research context. Task artifacts are read separately in this order: `prd.md` -> `design.md if present` -> `implement.md if present`. + +## JSONL Reading Rules + +`implement.jsonl` and `check.jsonl` contain one JSON object per line: + +```jsonl +{"file": ".trellis/spec/backend/index.md", "reason": "Backend rules"} +``` + +Readers should skip seed rows without a `file` field. When configuring JSONL, the AI should include only spec/research files, not pre-register code files that will be modified. + +## Active Task And Context Key + +Active task state lives in `.trellis/.runtime/sessions/` and is isolated per session. Hooks try to resolve the context key from platform events, environment variables, transcript paths, or `TRELLIS_CONTEXT_ID`. + +If shell commands cannot see the same context key, `task.py current --source` may report no active task. In that case, check whether the platform passes session identity into the shell instead of hand-writing a global current-task file. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Change session-start injected content | The platform's `session-start` hook or plugin file. | +| Change per-turn workflow-state rules | `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The platform workflow-state hook parses these blocks verbatim and embeds no fallback text. | +| Change how sub-agents read context | Platform agent definitions, the `inject-subagent-context` hook, or agent preludes. | +| Change JSONL validation/display | `.trellis/scripts/common/task_context.py`. | +| Change active task resolution | `.trellis/scripts/common/active_task.py`. | + +When modifying context injection, verify two things: new sessions can see the correct task, and sub-agents can see the correct task artifacts/spec/research. diff --git a/.agents/skills/trellis-meta/references/local-architecture/generated-files.md b/.agents/skills/trellis-meta/references/local-architecture/generated-files.md new file mode 100644 index 0000000..66f832d --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/generated-files.md @@ -0,0 +1,80 @@ +# Local Files Generated After Init + +`trellis init` writes the Trellis runtime into the user project. Later, `trellis update` tries to update Trellis-managed template files, but it uses `.trellis/.template-hashes.json` to determine which files have already been modified by the user. + +This page only describes files that are visible and editable inside the user project. + +## `.trellis/` + +```text +.trellis/ +├── workflow.md +├── config.yaml +├── .developer +├── .version +├── .template-hashes.json +├── .runtime/ +├── scripts/ +├── spec/ +├── tasks/ +└── workspace/ +``` + +| Path | Usually editable? | Notes | +| --- | --- | --- | +| `.trellis/workflow.md` | Yes | Local workflow documentation and AI routing rules. | +| `.trellis/config.yaml` | Yes | Project configuration, hooks, packages, journal line limits, and related settings. | +| `.trellis/spec/` | Yes | Project specs, intended to be updated regularly by users and AI. | +| `.trellis/tasks/` | Yes | Task material and research artifacts, maintained by the task workflow. | +| `.trellis/workspace/` | Yes | Session records, usually written by `add_session.py`. | +| `.trellis/scripts/` | Carefully | Local runtime. It can be customized, but only after understanding the call chain. | +| `.trellis/.runtime/` | No | Runtime state, usually written automatically by hooks/scripts. | +| `.trellis/.developer` | Carefully | Current developer identity. | +| `.trellis/.version` | No | Trellis version record used by update/migration logic. | +| `.trellis/.template-hashes.json` | No | Template hash record. Do not hand-write business rules here. | + +## Platform Directories + +Different platforms generate different directories. Common categories: + +| Category | Example paths | Purpose | +| --- | --- | --- | +| hooks | `.claude/hooks/`, `.codex/hooks/`, `.cursor/hooks/` | Inject session context, workflow-state, and sub-agent context. | +| settings | `.claude/settings.json`, `.codex/hooks.json`, `.qoder/settings.json` | Tell the platform when to run hooks or plugins. | +| agents | `.claude/agents/`, `.codex/agents/`, `.kiro/agents/` | Define agents such as `trellis-research`, `trellis-implement`, and `trellis-check`. | +| skills | `.claude/skills/`, `.agents/skills/`, `.qoder/skills/` | Skills that auto-trigger or can be read by AI. | +| commands/prompts/workflows | `.cursor/commands/`, `.github/prompts/`, `.windsurf/workflows/` | Explicit user-invoked command or workflow entry points. | + +When modifying a platform directory, also confirm whether `.trellis/workflow.md` still describes the same flow. + +## Meaning Of Template Hashes + +`.trellis/.template-hashes.json` records the content hash from the last time Trellis wrote a template file. `trellis update` uses it to distinguish three cases: + +| Case | Update behavior | +| --- | --- | +| File was not modified by the user | It can be updated automatically. | +| File was modified by the user | Prompt the user to overwrite, keep, or generate `.new`. | +| File is no longer a current template | It may be deleted, renamed, or preserved according to migration rules. | + +When an AI customizes local Trellis files, it does not need to maintain hashes manually. It is normal for Trellis update to recognize the result as "modified by the user." + +## Local Customization Boundaries + +Editable by default: + +- `.trellis/workflow.md` +- `.trellis/config.yaml` +- `.trellis/spec/**` +- `.trellis/scripts/**` +- Platform hooks, settings, agents, skills, commands, prompts, and workflows + +Do not edit by default: + +- Global npm install directory +- `node_modules/@mindfoldhq/trellis` +- Trellis GitHub repository source code +- Concrete state files under `.trellis/.runtime/**` +- Hash contents inside `.trellis/.template-hashes.json` + +Switch to the Trellis CLI source-code perspective only when the user explicitly wants to contribute upstream. diff --git a/.agents/skills/trellis-meta/references/local-architecture/overview.md b/.agents/skills/trellis-meta/references/local-architecture/overview.md new file mode 100644 index 0000000..99c7f73 --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/overview.md @@ -0,0 +1,51 @@ +# Local Trellis Architecture Overview + +`trellis-meta` is for user projects that have already run `trellis init`. The user's machine usually has only the npm-installed `trellis` command plus the Trellis files generated inside the project; it may not have the Trellis CLI source code. + +Therefore, when an AI uses this skill, the default customization target is local files inside the user project: + +- `.trellis/`: workflow, tasks, specs, memory, scripts, and runtime state. +- Platform directories: `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, `.kiro/`, `.gemini/`, `.qoder/`, `.codebuddy/`, `.github/`, `.factory/`, `.pi/`, `.kilocode/`, `.agent/`, `.windsurf/`, and similar directories. +- Shared skill layer: `.agents/skills/`. + +Do not default to guiding the user to fork the Trellis CLI repository. Treat upstream source code as the operating target only when the user explicitly says they want to change Trellis upstream source, publish an npm package, or contribute a PR. + +## Local System Model + +Trellis provides three layers inside a user project: + +1. **Workflow layer**: `.trellis/workflow.md` defines phases, routing, next actions, and prompt blocks. +2. **Persistence layer**: `.trellis/tasks/`, `.trellis/spec/`, and `.trellis/workspace/` store tasks, specs, and session memory. +3. **Platform integration layer**: hooks, settings, agents, skills, commands, prompts, and workflows in platform directories connect the Trellis workflow to different AI tools. + +All three layers live inside the user project, so an AI can read and modify them directly. + +## Core Paths + +| Path | Purpose | +| --- | --- | +| `.trellis/workflow.md` | Workflow phases, skill routing, and workflow-state prompt blocks. | +| `.trellis/config.yaml` | Project configuration, task lifecycle hooks, monorepo package configuration, and journal configuration. | +| `.trellis/spec/` | The user's project-specific coding conventions and thinking guides. | +| `.trellis/tasks/` | Each task's PRD, technical notes, research files, and JSONL context. | +| `.trellis/workspace/` | Per-developer journals and cross-session memory. | +| `.trellis/scripts/` | Local Python runtime used by commands, hooks, and context injection. | +| `.trellis/.runtime/` | Session-level runtime state, such as the current task pointer. | +| `.trellis/.template-hashes.json` | Template hashes for Trellis-managed files, used by update to determine whether local files were modified by the user. | + +## AI Customization Principles + +1. **Find the local source of truth first**: Do not edit from memory. Read `.trellis/workflow.md`, `.trellis/config.yaml`, the relevant platform directory, and related task files first. +2. **Edit the user project, not the npm package cache**: Modify generated files inside the project, not `node_modules` or the global npm install directory. +3. **Keep platform files aligned with `.trellis/`**: If workflow routing changes, also check whether platform skills or commands still describe the same flow. +4. **Put project-specific rules in `.trellis/spec/` or a local skill**: Do not put team conventions into `trellis-meta`. +5. **Preserve user changes**: If a file was already modified locally, work from the current content instead of overwriting it with a default template. + +## How To Use This Directory + +- To understand which files exist after init, read `generated-files.md`. +- To change phases, routing, or next actions, read `workflow.md`. +- To change the task model, JSONL context, or active task behavior, read `task-system.md`. +- To change coding convention injection, read `spec-system.md`. +- To understand journals and cross-session memory, read `workspace-memory.md`. +- To change hooks or sub-agent context loading, read `context-injection.md`. diff --git a/.agents/skills/trellis-meta/references/local-architecture/spec-system.md b/.agents/skills/trellis-meta/references/local-architecture/spec-system.md new file mode 100644 index 0000000..1ff49f4 --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/spec-system.md @@ -0,0 +1,102 @@ +# Local Spec System + +`.trellis/spec/` is the user's project-specific engineering spec library. Trellis is not about making AI memorize conventions; it injects relevant specs or requires the AI to read them at the right time. + +## Directory Model + +A common single-repository structure: + +```text +.trellis/spec/ +├── backend/ +│ ├── index.md +│ └── ... +├── frontend/ +│ ├── index.md +│ └── ... +└── guides/ + ├── index.md + └── ... +``` + +A common monorepo structure: + +```text +.trellis/spec/ +├── cli/ +│ ├── backend/ +│ │ ├── index.md +│ │ └── ... +│ └── unit-test/ +│ ├── index.md +│ └── ... +├── docs-site/ +│ └── docs/ +│ ├── index.md +│ └── ... +└── guides/ + ├── index.md + └── ... +``` + +`index.md` is the entry point for each layer. It should list the Pre-Development Checklist and Quality Check. Specific guidelines live in other Markdown files in the same directory. + +## Package Configuration + +`.trellis/config.yaml` can declare packages: + +```yaml +packages: + cli: + path: packages/cli + docs-site: + path: docs-site + type: submodule +default_package: cli +``` + +The AI can run: + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +This command lists packages and spec layers for the current project. Use this output as the reference when configuring context JSONL. + +## How Specs Enter Tasks + +Before a task enters implementation, planning may write relevant specs into `implement.jsonl` / `check.jsonl` when the task needs spec or research context beyond the task artifacts: + +```jsonl +{"file": ".trellis/spec/cli/backend/index.md", "reason": "CLI backend conventions"} +{"file": ".trellis/spec/cli/unit-test/conventions.md", "reason": "Test expectations"} +``` + +Sub-agents or platform preludes read these JSONL files and load the referenced specs. On platforms without sub-agent support, the AI should read the relevant specs directly according to the workflow. + +## What Specs Should Contain + +Specs should contain executable engineering conventions for the project, not generic best practices: + +- Where files should live. +- How error handling should be expressed. +- Input/output contracts for APIs, hooks, and commands. +- Patterns that are forbidden. +- Cases that require tests. +- Project-specific pitfalls and how to avoid them. + +When the AI learns a new rule during implementation or debugging, it should update `.trellis/spec/` rather than only summarizing it in chat. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Add a new spec layer | `.trellis/spec///index.md` and corresponding guideline files. | +| Change monorepo spec mapping | `packages` / `default_package` / `spec_scope` in `.trellis/config.yaml`. | +| Change which specs AI reads before implementation | The task's `implement.jsonl`. | +| Change which specs AI reads during checking | The task's `check.jsonl`. | +| Change when specs should be updated | Phase 3.3 in `.trellis/workflow.md` and the `trellis-update-spec` skill. | + +## Boundaries + +`.trellis/spec/` is the user's project specification, not a permanent copy of Trellis built-in templates. The AI should encourage the user to update it according to the actual project code instead of treating Trellis default templates as immutable documents. diff --git a/.agents/skills/trellis-meta/references/local-architecture/task-system.md b/.agents/skills/trellis-meta/references/local-architecture/task-system.md new file mode 100644 index 0000000..9dfe5bb --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/task-system.md @@ -0,0 +1,130 @@ +# Local Task System + +The Trellis task system is stored entirely under `.trellis/tasks/` in the user project. Each task is a directory containing requirements, context, research, state, and relationship information. + +## Task Directory Structure + +```text +.trellis/tasks/ +├── 04-28-example-task/ +│ ├── task.json +│ ├── prd.md +│ ├── design.md +│ ├── implement.md +│ ├── implement.jsonl +│ ├── check.jsonl +│ └── research/ +└── archive/ + └── 2026-04/ +``` + +| File | Purpose | +| --- | --- | +| `task.json` | Task metadata: status, assignee, priority, branch, parent/child tasks, and similar fields. | +| `prd.md` | Requirements, constraints, and acceptance criteria. Lightweight tasks may be PRD-only. | +| `design.md` | Technical design for complex tasks: boundaries, contracts, data flow, compatibility, tradeoffs. | +| `implement.md` | Execution plan for complex tasks: ordered checklist, validation commands, review gates, rollback points. | +| `implement.jsonl` | List of spec/research files the implement agent must read first. | +| `check.jsonl` | List of spec/research files the check agent must read first. | +| `research/` | Research artifacts. Complex findings should not live only in chat. | + +## `task.json` + +`task.json` records task status and metadata. Common fields: + +| Field | Meaning | +| --- | --- | +| `id` / `name` / `title` | Task identity and title. | +| `status` | Status such as `planning`, `in_progress`, `review`, or `completed`. | +| `priority` | `P0`, `P1`, `P2`, `P3`. | +| `creator` / `assignee` | Creator and assignee. | +| `package` | Target package in a monorepo; may be empty. | +| `branch` / `base_branch` | Working branch and PR target branch. | +| `children` / `parent` | Parent/child task relationships. | +| `commit` / `pr_url` | Commit and PR information after completion. | +| `meta` | Extension fields. | + +## Parent / Child Task Trees + +Parent/child task relationships are for work structure. A parent task groups related deliverables under one source requirement set; it is not a dependency scheduler and does not replace the child task's own planning artifacts. + +Use a parent task when a request has multiple independently verifiable deliverables. The parent owns: + +- Source requirements and user-facing scope. +- The map of child tasks and their responsibility boundaries. +- Cross-child acceptance criteria and final integration review. + +Use child tasks for deliverables that can move through planning, implementation, check, and archive independently. If one child depends on another, write that dependency in the child `prd.md` / `implement.md`; do not rely on tree position to imply ordering. + +Create new children with: + +```bash +python ./.trellis/scripts/task.py create "" --slug --parent +``` + +Link or unlink existing tasks with: + +```bash +python ./.trellis/scripts/task.py add-subtask +python ./.trellis/scripts/task.py remove-subtask +``` + +`children` on the parent is a historical list. When a child is archived, Trellis keeps that child name in the parent so progress like `[2/3 done]` remains meaningful after completed children move to `archive/`. + +The AI should not treat phase numbers as task status. Task progress is mainly determined by `status`, artifact presence (`prd.md`, optional `design.md` / `implement.md`), whether JSONL context is configured for sub-agent mode, and the phase descriptions in `workflow.md`. + +## Active Task + +The user sees a "current task," but Trellis stores active task state per session. + +```text +.trellis/.runtime/sessions/.json +``` + +`task.py start` writes the task path into the runtime session file for the current session. `task.py current --source` shows the current task and where it came from. Different AI windows can point to different tasks without overwriting each other. + +If the platform or shell environment has no stable session identity, `task.py start` may be unable to set the active task. The AI should read the error, inspect the platform hook/session environment, and not fall back to a shared global pointer. + +## JSONL Context + +`implement.jsonl` and `check.jsonl` are context manifests for sub-agents to read first. They do not replace `implement.md`; `implement.md` is the human-readable execution plan. + +Format: + +```jsonl +{"file": ".trellis/spec/cli/backend/index.md", "reason": "Backend conventions"} +{"file": ".trellis/tasks/04-28-example/research/api.md", "reason": "API research"} +``` + +Rules: + +- Include spec and research files. +- Do not include code files that are about to be modified. +- Do not treat temporary conclusions in chat as the only context. +- Seed rows have no `file` field; they only prompt the AI to fill in real entries. + +## Common Commands + +```bash +python ./.trellis/scripts/task.py create "" --slug <slug> +python ./.trellis/scripts/task.py start <task> +python ./.trellis/scripts/task.py current --source +python ./.trellis/scripts/task.py add-context <task> implement <file> <reason> +python ./.trellis/scripts/task.py validate <task> +python ./.trellis/scripts/task.py finish +python ./.trellis/scripts/task.py archive <task> +``` + +When modifying the task system, the AI should prefer script commands to maintain structure. Edit JSON/Markdown directly only when scripts do not cover the need. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Change the default task template | `.trellis/scripts/common/task_store.py` and task creation instructions. | +| Change status semantics | `.trellis/workflow.md`, workflow-state hook logic, and task usage conventions. | +| Add task lifecycle actions | `hooks.after_*` in `.trellis/config.yaml`. | +| Change context rules | Planning artifact guidance in `.trellis/workflow.md` and related platform agent/hook instructions. | +| Change archive policy | `.trellis/scripts/common/task_store.py` / `task_utils.py`. | + +These are local files in the user project. Do not default to editing Trellis CLI source code unless the user wants to contribute upstream. diff --git a/.agents/skills/trellis-meta/references/local-architecture/workflow.md b/.agents/skills/trellis-meta/references/local-architecture/workflow.md new file mode 100644 index 0000000..f0659ff --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/workflow.md @@ -0,0 +1,75 @@ +# Local Workflow System + +`.trellis/workflow.md` is the Trellis workflow source of truth inside the user project. An AI does not need Trellis source code to understand how the current project should move tasks forward; this file is enough. + +## File Responsibilities + +`.trellis/workflow.md` has three responsibilities: + +1. **Explain workflow phases**: Plan, Execute, Finish. +2. **Define skill routing**: which skill or agent the AI should use when the user expresses a certain intent. +3. **Provide workflow-state prompt blocks**: hooks can inject the prompt block for the current state into the conversation. + +## Current Phase Model + +```text +Phase 1: Plan -> clarify what to build, produce prd.md and required research +Phase 2: Execute -> implement against the PRD and specs, then check +Phase 3: Finish -> final verification, preserve lessons, and wrap up +``` + +Each phase contains numbered steps, such as `1.3 Configure context`. These numbers are not runtime fields in `task.json`; they are workflow structure for AI and humans to read. + +## Skill Routing + +`workflow.md` separates routing by platform capability: + +- Platforms with sub-agent support: dispatch `trellis-implement` by default for implementation and `trellis-check` for checking. +- Platforms without sub-agent support: the main session reads skills such as `trellis-before-dev`, then executes directly. + +When changing local AI behavior, update the routing descriptions in `workflow.md` first, then check whether the corresponding platform skill, command, or agent files need to stay in sync. + +## Workflow-State Prompt Blocks + +The bottom of `workflow.md` can contain state blocks like this: + +```text +[workflow-state:no_task] +... +[/workflow-state:no_task] +``` + +Hooks choose the right block based on current task status and inject it into the conversation. Common states include: + +| State | Meaning | +| --- | --- | +| `no_task` | The current session has no active task. | +| `planning` | The task is still in requirements, research, or context configuration. | +| `in_progress` | The task has entered implementation and checking. | +| `completed` | The task is complete and waiting for wrap-up or archive. | + +If the user wants to change policies such as "whether to create a task when there is no task," "when task creation may be skipped," or "whether sub-agents are required," edit these state blocks and the routing table above them. + +## Local Modification Patterns + +Common changes: + +| Goal | Edit point | +| --- | --- | +| Add a phase | Update the Phase Index, phase body, routing, and state blocks. | +| Change task creation policy | Update the `no_task` state block and Phase 1 description. | +| Change the default implementation/check path | Update Phase 2 and skill routing. | +| Change the wrap-up flow | Update Phase 3 and `finish-work` related descriptions. Note the current split: Phase 3.4 = AI-driven code commits (batched, user-confirmed), Phase 3.5 = `/finish-work` (archive + record session). `/finish-work` refuses to run if the working tree is dirty. | +| Change platform differences | Update routing descriptions grouped by platform. | + +After editing, make the AI reread `.trellis/workflow.md`; do not assume the flow from the old conversation is still valid. + +## Relationship To Platform Files + +`workflow.md` is the semantic center of the local workflow, but each platform can also have its own entry files: + +- skills, such as `trellis-brainstorm` and `trellis-check`. +- commands/prompts/workflows, such as continue and finish-work. +- hooks, such as session-start or workflow-state injection. + +If only `workflow.md` changes, platform entry files may still contain old language. When the user wants to change "what the AI actually does," also inspect the relevant platform directory. diff --git a/.agents/skills/trellis-meta/references/local-architecture/workspace-memory.md b/.agents/skills/trellis-meta/references/local-architecture/workspace-memory.md new file mode 100644 index 0000000..92d29f4 --- /dev/null +++ b/.agents/skills/trellis-meta/references/local-architecture/workspace-memory.md @@ -0,0 +1,71 @@ +# Local Workspace Memory System + +`.trellis/workspace/` stores cross-session memory. Its purpose is to let AI and humans understand what happened before across different windows and different days. + +## Directory Structure + +```text +.trellis/workspace/ +├── index.md +└── <developer>/ + ├── index.md + ├── journal-1.md + └── journal-2.md +``` + +| File | Purpose | +| --- | --- | +| `.trellis/.developer` | Current developer identity. | +| `.trellis/workspace/index.md` | Global workspace overview. | +| `.trellis/workspace/<developer>/index.md` | Session index for a developer. | +| `.trellis/workspace/<developer>/journal-N.md` | Session journal. | + +## Developer Identity + +Run this the first time: + +```bash +python ./.trellis/scripts/init_developer.py <name> +``` + +This creates `.trellis/.developer` and the corresponding workspace directory. The AI should not change developer identity casually; if the identity is wrong, first confirm who is using the current project. + +## Journal + +`journal-N.md` records completed or partially completed work from each session. By default, each journal holds about 2000 lines; after that it rotates to the next file. + +Common command for recording a session: + +```bash +python ./.trellis/scripts/add_session.py \ + --title "Session title" \ + --summary "What changed" \ + --commit "abc1234" +``` + +Planning or review work without a commit can also be recorded by using `--no-commit` or an empty commit value. + +## Relationship Between Workspace Memory And Tasks + +| System | What it stores | +| --- | --- | +| `.trellis/tasks/` | Requirements, design, research, and state for a specific task. | +| `.trellis/workspace/` | Work records across tasks and sessions. | +| `.trellis/spec/` | Engineering knowledge preserved as long-term conventions. | + +If information is only useful for the current task, put it in the task directory. +If information describes what happened in the current session, put it in the workspace journal. +If information should be followed every time code is written in the future, put it in spec. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Change maximum journal lines | `max_journal_lines` in `.trellis/config.yaml`. | +| Change session auto-commit message | `session_commit_message` in `.trellis/config.yaml`. | +| Change session content format | `.trellis/scripts/add_session.py`. | +| Change how workspace is displayed in context | `.trellis/scripts/common/session_context.py`. | + +## AI Usage Rules + +The AI should not treat workspace as the only source of truth. When resuming a task, read the current task first, then use workspace for background. After a task is complete, record important process notes in workspace; if long-term rules emerged, update spec. diff --git a/.agents/skills/trellis-meta/references/platform-files/agents.md b/.agents/skills/trellis-meta/references/platform-files/agents.md new file mode 100644 index 0000000..3976987 --- /dev/null +++ b/.agents/skills/trellis-meta/references/platform-files/agents.md @@ -0,0 +1,80 @@ +# Agents + +Trellis agent files define specialized roles. Common Trellis agents in a user project are: + +- `trellis-research` +- `trellis-implement` +- `trellis-check` + +File locations and formats differ by platform, but responsibility boundaries should stay consistent. + +## Agent Responsibilities + +| Agent | Responsibility | +| --- | --- | +| `trellis-research` | Investigate the question and write findings into the current task's `research/`. | +| `trellis-implement` | Implement against `prd.md`, optional `design.md` / `implement.md`, `implement.jsonl`, and related spec/research. | +| `trellis-check` | Review changes, fix discovered issues, and run necessary checks. | + +Agent files should not become generic chat prompts. They should define input sources, write boundaries, whether code may be changed, and how results are reported. + +## Common Paths + +| Platform | Agent path | +| --- | --- | +| Claude Code | `.claude/agents/trellis-*.md` | +| Cursor | `.cursor/agents/trellis-*.md` | +| OpenCode | `.opencode/agents/trellis-*.md` | +| Codex | `.codex/agents/trellis-*.toml` | +| Kiro | `.kiro/agents/trellis-*.json` | +| Gemini CLI | `.gemini/agents/trellis-*.md` | +| Qoder | `.qoder/agents/trellis-*.md` | +| CodeBuddy | `.codebuddy/agents/trellis-*.md` | +| Factory Droid | `.factory/droids/trellis-*.md` | +| Pi Agent | `.pi/agents/trellis-*.md` | + +GitHub Copilot agent/prompt support is provided by a combination of directories such as `.github/agents/`, `.github/prompts/`, and `.github/skills/`; inspect the files actually generated in the user project. + +Main-session workflow platforms such as Kilo, Antigravity, and Windsurf may not have Trellis sub-agent files. They usually rely on workflows/skills to guide the main session. + +## Two Context Loading Modes + +### hook push + +The platform hook injects task context before the agent starts. The agent file itself can focus more on responsibilities and boundaries. + +Common on platforms that support agent hooks. + +### agent pull + +The agent file instructs the agent to read after startup: + +- `python ./.trellis/scripts/task.py current --source` +- `implement.jsonl` or `check.jsonl` +- spec/research files referenced by JSONL +- current task `prd.md` +- `design.md` if present +- `implement.md` if present + +This mode fits platforms whose hooks cannot reliably rewrite sub-agent prompts. + +## Local Change Scenarios + +| User need | Edit location | +| --- | --- | +| Implement agent must follow extra restrictions | The platform's `trellis-implement` agent file. | +| Check agent must run project-specific commands | `trellis-check` agent file, and `.trellis/spec/` if needed. | +| Research agent must output a fixed format | `trellis-research` agent file. | +| Agent cannot read task context | Agent prelude or `inject-subagent-context` hook. | +| Add a project-specific agent | Platform agent directory + related workflow/command/skill entry point. | + +## Modification Principles + +1. **Keep responsibilities single-purpose**. Do not mix research, implement, and check responsibilities into one agent. +2. **Specify the read order**. Agents must know to start from the active task, read jsonl/spec context, then read `prd.md`, `design.md` if present, and `implement.md` if present. +3. **Specify write boundaries**. Research usually only writes `research/`; implement can write code; check can fix issues. +4. **Keep semantics synchronized in multi-platform projects**. If the user configured Claude, Codex, and Cursor together, decide whether changes to one platform's agent also need to be applied to others. + +## Do Not Default To Editing Upstream Templates + +Local AI should default to modifying platform agent files inside the user project. Discuss upstream template source only when the user explicitly wants to contribute the change back to Trellis. diff --git a/.agents/skills/trellis-meta/references/platform-files/hooks-and-settings.md b/.agents/skills/trellis-meta/references/platform-files/hooks-and-settings.md new file mode 100644 index 0000000..94156a8 --- /dev/null +++ b/.agents/skills/trellis-meta/references/platform-files/hooks-and-settings.md @@ -0,0 +1,69 @@ +# Hooks And Settings + +Hooks/settings are the entry layer that connects a platform to Trellis. They decide which scripts, plugins, or extensions a platform runs for which events. + +## Settings Responsibilities + +settings/config files usually register: + +- session-start hook: injects a Trellis overview when a new session starts or context resets. +- workflow-state hook: parses `[workflow-state:STATUS]` blocks from `.trellis/workflow.md` and emits the body matching the current task `status` on each user input. Parser-only; the script does not embed fallback content. +- sub-agent context hook: injects task context when implementation/check/research agents start. +- shell/session bridge: lets shell commands see the same Trellis session identity. +- platform plugin or extension entry points. + +Common files: + +| Platform | settings/config | +| --- | --- | +| Claude Code | `.claude/settings.json` | +| Cursor | `.cursor/hooks.json` | +| Codex | `.codex/hooks.json`, `.codex/config.toml` | +| OpenCode | `.opencode/package.json`, `.opencode/plugins/*` | +| Kiro | `.kiro/hooks/` + platform config | +| Gemini CLI | `.gemini/settings.json` | +| Qoder | `.qoder/settings.json` | +| CodeBuddy | `.codebuddy/settings.json` | +| GitHub Copilot | `.github/copilot/hooks.json` | +| Factory Droid | `.factory/settings.json` | +| Pi Agent | `.pi/settings.json`, `.pi/extensions/trellis/` | + +Whether these files exist in a project depends on which `trellis init --<platform>` flags the user ran. + +## Hook Script Types + +| Script | Purpose | +| --- | --- | +| `session-start.py` | Generates session-start context. | +| `inject-workflow-state.py` | Parses `[workflow-state:STATUS]` blocks in `.trellis/workflow.md` and emits the body matching the current task status. Falls back to `Refer to workflow.md for current step.` when no matching block exists. | +| `inject-subagent-context.py` | Injects PRD, JSONL context, and related spec/research into sub-agents. | +| `inject-shell-session-context.py` | Lets shell commands inherit Trellis session identity. | + +Not every platform has every hook. Do not copy files from another platform just because a platform lacks a hook; first confirm whether that platform supports the corresponding event. + +## Local Change Scenarios + +| User need | Edit location | +| --- | --- | +| AI should see more/less context in a new session | Platform `session-start` hook. | +| Per-turn hint policy should change | `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The hook parses workflow.md verbatim — no script edit required. | +| Sub-agent cannot read PRD/spec | `inject-subagent-context` hook or agent prelude. | +| `task.py current` in shell has no active task | Shell/session bridge hook or platform environment variable configuration. | +| Disable an automatic injection | The corresponding hook registration in settings/config. | + +## Modification Principles + +1. **Settings wire things up; hooks define behavior**. If only the hook changes, the platform may never call it. If only settings change, behavior may not change. +2. **Confirm platform event names first**. Different platforms use different names for SessionStart, UserPromptSubmit, AgentSpawn, shell execution, and similar events. +3. **Hooks read local `.trellis/`, not upstream source**. `.trellis/scripts/` and `.trellis/workflow.md` in the user project are the default targets. +4. **Errors must be visible**. Hook failures should tell the user what was not injected instead of silently leaving the AI without context. + +## Troubleshooting Path + +If the user says "AI did not read Trellis state": + +1. Check whether the platform settings register the hook. +2. Check whether the hook file exists. +3. Manually run the `.trellis/scripts/get_context.py` or `task.py current --source` command that the hook depends on. +4. Check whether active task state exists in `.trellis/.runtime/sessions/`. +5. Check whether the platform shell passes session identity. diff --git a/.agents/skills/trellis-meta/references/platform-files/overview.md b/.agents/skills/trellis-meta/references/platform-files/overview.md new file mode 100644 index 0000000..60ae1df --- /dev/null +++ b/.agents/skills/trellis-meta/references/platform-files/overview.md @@ -0,0 +1,59 @@ +# Platform Files Overview + +Trellis connects the same local architecture to different AI tools. `.trellis/` stores the shared runtime; platform directories store adapter files that define how each AI tool enters Trellis. + +When a local AI modifies Trellis, it should distinguish two file categories first: + +- **Shared files**: `.trellis/workflow.md`, `.trellis/tasks/`, `.trellis/spec/`, `.trellis/scripts/`. +- **Platform files**: `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, `.kiro/`, `.gemini/`, `.qoder/`, `.codebuddy/`, `.github/`, `.factory/`, `.pi/`, `.kilocode/`, `.agent/`, `.windsurf/`, and similar directories. + +Platform files do not store business state. They let the corresponding AI tool read Trellis state, call Trellis scripts, and load Trellis skills/agents/hooks. + +## Platform File Categories + +| Category | Common paths | Purpose | +| --- | --- | --- | +| settings/config | `.claude/settings.json`, `.codex/hooks.json`, `.qoder/settings.json` | Register hooks, plugins, extensions, or platform behavior. | +| hooks/plugins/extensions | `.claude/hooks/`, `.opencode/plugins/`, `.pi/extensions/` | Inject context at session start, user input, agent startup, shell execution, and similar events. | +| agents | `.claude/agents/`, `.codex/agents/`, `.kiro/agents/` | Define `trellis-research`, `trellis-implement`, and `trellis-check`. | +| skills | `.claude/skills/`, `.agents/skills/`, `.qoder/skills/` | Capability descriptions that auto-trigger or can be read on demand. | +| commands/prompts/workflows | `.cursor/commands/`, `.github/prompts/`, `.windsurf/workflows/` | Entry points explicitly invoked by the user. | + +## Three Platform Integration Modes + +### 1. Hook / Extension Driven + +These platforms can trigger scripts or plugins on specific events and actively inject Trellis context into AI. + +Common capabilities: + +- session-start injection of a `.trellis/` overview. +- workflow-state hints for each user turn. +- PRD/spec/research injection when sub-agents start. +- Shell commands inheriting session identity. + +To change "when the AI knows what," inspect hooks/plugins/extensions and settings first. + +### 2. Agent Prelude / Pull-Based + +Some platforms cannot reliably let hooks rewrite sub-agent prompts, so the agent file itself instructs the agent to read the active task, PRD, and JSONL context after startup. + +To change how sub-agents load context, inspect the agent files themselves. + +### 3. Main-Session Workflow + +Some platforms do not have Trellis sub-agent or hook capabilities. They rely on workflows/skills/commands to guide the main-session AI to read files, run scripts, and move tasks forward. + +To change behavior, inspect platform workflows/skills/commands and `.trellis/workflow.md`. + +## Local Modification Order + +When the user asks to customize behavior for a platform, the AI should inspect files in this order: + +1. Read `.trellis/workflow.md` to confirm the shared flow. +2. Read the target platform's settings/config to see which hooks/agents/skills/commands are registered. +3. Read the target platform's agents/skills/commands/hooks. +4. Modify the local file closest to the user's need. +5. If the change affects the shared flow, synchronize `.trellis/workflow.md` or `.trellis/spec/`. + +Do not modify only platform files and forget the shared workflow. Do not modify only `.trellis/workflow.md` and forget that platform entry points may still contain old descriptions. diff --git a/.agents/skills/trellis-meta/references/platform-files/platform-map.md b/.agents/skills/trellis-meta/references/platform-files/platform-map.md new file mode 100644 index 0000000..b5576f4 --- /dev/null +++ b/.agents/skills/trellis-meta/references/platform-files/platform-map.md @@ -0,0 +1,74 @@ +# Platform File Map + +This page lists common Trellis file locations in a user project by platform. Whether a platform directory exists in an actual project depends on which `trellis init --<platform>` commands the user ran. + +## Matrix + +| Platform | CLI flag | Main directory | Skill directory | Agent directory | Hooks/extensions | +| --- | --- | --- | --- | --- | --- | +| Claude Code | `--claude` | `.claude/` | `.claude/skills/` | `.claude/agents/` | `.claude/hooks/` + `.claude/settings.json` | +| Cursor | `--cursor` | `.cursor/` | `.cursor/skills/` | `.cursor/agents/` | `.cursor/hooks.json` + `.cursor/hooks/` | +| OpenCode | `--opencode` | `.opencode/` | `.opencode/skills/` | `.opencode/agents/` | `.opencode/plugins/` | +| Codex | `--codex` | `.codex/` | `.agents/skills/` | `.codex/agents/` | `.codex/hooks/` + `.codex/hooks.json` | +| Kilo | `--kilo` | `.kilocode/` | `.kilocode/skills/` | Usually none | `.kilocode/workflows/` | +| Kiro | `--kiro` | `.kiro/` | `.kiro/skills/` | `.kiro/agents/` | `.kiro/hooks/` | +| Gemini CLI | `--gemini` | `.gemini/` | `.agents/skills/` | `.gemini/agents/` | `.gemini/settings.json` + `.gemini/hooks/` | +| Antigravity | `--antigravity` | `.agent/` | `.agent/skills/` | Usually none | `.agent/workflows/` | +| Windsurf | `--windsurf` | `.windsurf/` | `.windsurf/skills/` | Usually none | `.windsurf/workflows/` | +| Qoder | `--qoder` | `.qoder/` | `.qoder/skills/` | `.qoder/agents/` | `.qoder/hooks/` + `.qoder/settings.json` | +| CodeBuddy | `--codebuddy` | `.codebuddy/` | `.codebuddy/skills/` | `.codebuddy/agents/` | `.codebuddy/hooks/` + `.codebuddy/settings.json` | +| GitHub Copilot | `--copilot` | `.github/` | `.github/skills/` | `.github/agents/` | `.github/copilot/hooks/` + prompts | +| Factory Droid | `--droid` | `.factory/` | `.factory/skills/` | `.factory/droids/` | `.factory/hooks/` + settings | +| Pi Agent | `--pi` | `.pi/` | `.pi/skills/` | `.pi/agents/` | `.pi/extensions/trellis/` + `.pi/settings.json` | + +## Capability Groups + +### Trellis Sub-Agent Support + +These platforms usually have `trellis-research`, `trellis-implement`, and `trellis-check` files: + +- Claude Code +- Cursor +- OpenCode +- Codex +- Kiro +- Gemini CLI +- Qoder +- CodeBuddy +- GitHub Copilot +- Factory Droid +- Pi Agent + +When changing implementation/check/research behavior, look for the corresponding platform agent files first. + +### Main-Session Workflow Platforms + +These platforms rely more on workflows/skills to guide the main session: + +- Kilo +- Antigravity +- Windsurf + +When changing behavior, inspect workflows and skills first. Do not assume Trellis sub-agents exist. + +### Shared `.agents/skills/` + +Codex writes the shared `.agents/skills/` layer. Some tools that support agentskills.io can also read this directory. If the user wants multiple compatible tools to share one skill, consider `.agents/skills/` first, but do not assume every platform reads it. + +## Decision Rules When Modifying Platform Files + +1. User specified a platform: modify only that platform directory unless shared workflow/spec files must also change. +2. User says "all platforms should do this": synchronize equivalent entry points platform by platform; do not modify only one directory. +3. User only says "my AI": inspect the configuration directories that actually exist in the project and infer the current AI platform. +4. User wants project rules: prefer `.trellis/spec/` or a project-local skill. +5. User wants Trellis behavior: edit `.trellis/workflow.md` plus platform hooks/agents/skills/commands. + +## When Paths Differ + +Platform ecosystems change, and user projects may already be customized. If this table disagrees with local files, use the actual settings/config in the user project as authoritative: + +- Check the hook that settings registers. +- Check the script that a command/prompt/workflow points to. +- Judge behavior by the read rules currently written in the agent file. + +Do not delete a custom file just because it is not listed in this path table. diff --git a/.agents/skills/trellis-meta/references/platform-files/skills-and-commands.md b/.agents/skills/trellis-meta/references/platform-files/skills-and-commands.md new file mode 100644 index 0000000..816c666 --- /dev/null +++ b/.agents/skills/trellis-meta/references/platform-files/skills-and-commands.md @@ -0,0 +1,83 @@ +# Skills, Commands, Prompts, And Workflows + +Skills and commands are textual entry points for user interaction with Trellis. Different platforms use different names, but their core purpose is the same: tell the AI how to enter the Trellis flow when the user expresses a certain intent. + +## Conceptual Differences + +| Type | Trigger mode | Best for | +| --- | --- | --- | +| skill | AI auto-match or explicit user mention | Long-term capabilities, workflow rules, modification guides. | +| command | Explicit user invocation | Clear operation entry points such as continue and finish-work. | +| prompt | Explicit user invocation or platform selection | Similar to command, but in a platform prompt format. | +| workflow | Explicit user selection or platform auto-match | Guides the main session when no sub-agent/hook exists. | + +Trellis workflow skills usually share one semantic set: brainstorm, before-dev, check, update-spec, break-loop. Multi-file built-in skills such as `trellis-meta` use layered references. + +## Common Paths + +| Platform | Common entries | +| --- | --- | +| Claude Code | `.claude/skills/`, `.claude/commands/` | +| Cursor | `.cursor/skills/`, `.cursor/commands/` | +| OpenCode | `.opencode/skills/`, `.opencode/commands/` | +| Codex | `.agents/skills/`, `.codex/skills/` | +| Kilo | `.kilocode/skills/`, `.kilocode/workflows/` | +| Kiro | `.kiro/skills/` | +| Gemini CLI | `.agents/skills/`, `.gemini/commands/` | +| Antigravity | `.agent/skills/`, `.agent/workflows/` | +| Windsurf | `.windsurf/skills/`, `.windsurf/workflows/` | +| Qoder | `.qoder/skills/`, `.qoder/commands/` | +| CodeBuddy | `.codebuddy/skills/`, `.codebuddy/commands/` | +| GitHub Copilot | `.github/skills/`, `.github/prompts/` | +| Factory Droid | `.factory/skills/`, `.factory/commands/` | +| Pi Agent | `.pi/skills/` | + +In a user project, use the files actually generated by init as authoritative. + +## Skill Structure + +A common skill is a directory: + +```text +trellis-meta/ +├── SKILL.md +└── references/ +``` + +`SKILL.md` should tell the AI: + +- When to use this skill. +- Which reference to read first for the current task. +- What not to do. + +References hold longer explanations so the entry file does not contain everything. + +## Command/Prompt/Workflow Structure + +Commands, prompts, and workflows are usually single files. Their content should include: + +- When to use it. +- Which `.trellis/` files to read. +- Which scripts to run. +- How to report after completion. + +They should not store task state; task state belongs in `.trellis/tasks/` and `.trellis/.runtime/`. + +## Local Change Scenarios + +| User need | Edit location | +| --- | --- | +| Change AI auto-trigger rules | The corresponding skill's frontmatter description. | +| Change user command behavior | The corresponding command/prompt/workflow file. | +| Add a project-local skill | Platform skill directory, or shared `.agents/skills/`. | +| Let multiple platforms share one capability | Write equivalent skills in each platform skill directory, or use the `.agents/skills/` shared layer on platforms that support it. | +| Change finish/continue entry points | Platform commands/prompts/workflows. | + +## Modification Principles + +1. **Keep entry files short; references carry long content**. This matters especially for multi-file skills like `trellis-meta`. +2. **Make trigger descriptions specific**. A description that is too broad can mis-trigger; one that is too narrow may not trigger. +3. **Keep the same semantics consistent across platforms**. File formats can differ, but behavior descriptions should match. +4. **Put project-specific capabilities in local skills**. Do not put team-private flows into public `trellis-meta`. + +If the user only wants local AI to know one more project rule, usually create a project-local skill or update `.trellis/spec/` instead of changing a Trellis built-in workflow skill. diff --git a/.agents/skills/trellis-session-insight/SKILL.md b/.agents/skills/trellis-session-insight/SKILL.md new file mode 100644 index 0000000..3670739 --- /dev/null +++ b/.agents/skills/trellis-session-insight/SKILL.md @@ -0,0 +1,81 @@ +--- +name: trellis-session-insight +description: "Reach into past AI conversation history through the `trellis mem` CLI. Use whenever the user asks 'how did we solve X last time', 'have we discussed this before', 'what was the decision on X', 'remind me what we did in this task', '上次怎么解的', '之前讨论过吗', '想起一段对话', or when starting a brainstorm that overlaps prior work, debugging a familiar bug, continuing a task across sessions, or doing a finish-work review. Returns raw past dialogue; decide for the moment whether to update spec, append to task notes, quote inline in the answer, or just internalize." +--- + +# Trellis Session Insight + +This skill teaches an AI **how to call `trellis mem`** — the project's cross-session memory feedstock — and **when reaching for it is the right move**. + +It is intentionally a **capability skill, not a workflow**. There is no fixed output file, no required write-back step, no "always run after finish-work" rule. What to do with what `mem` returns is a judgement call made in the moment of the conversation. The skill exists so the AI knows the capability is there and can decide. + +## What `trellis mem` is + +A local CLI that indexes the user's past Claude Code and Codex conversation logs (the JSONL files each platform stores under `~/.claude/projects/` and `~/.codex/sessions/`) and lets you list, search, slice by Trellis task boundaries, and dump cleaned dialogue from them. OpenCode logs are not yet indexable (provider adapter pending) — when an OpenCode session is the obvious target, surface that limitation rather than guessing. + +Nothing in `mem` is uploaded. All reads are local. + +## When to reach for it + +The bar is "would a senior teammate ask 'didn't we already talk about this?'" — those are the moments. Some concrete patterns: + +- **Brainstorm rerun risk.** Starting a new task that touches an area the user has been in before, and you want to check whether a decision was already made — before re-asking the user. +- **Familiar-bug debugging.** The current bug pattern feels like one the user reported / fixed before. Pulling the relevant past session can save a full debugging loop. +- **Cross-session continuation.** The user resumes work after a gap and says "where were we" / "继续上次的" without being specific. +- **Decision retrieval.** The user references "the decision we made about X" but the decision lives in an old brainstorm, not in any `prd.md` / `spec/`. +- **Finish-work retrospective.** When the user explicitly asks for a wrap-up of what was decided / what hurt / what surprised them in this task — not as a forced step on every finish-work. +- **Pattern-spotting across past work.** The user asks "do I keep making the same mistake on X" / "我每次都踩这个坑吗" — search across sessions answers that. + +If none of these apply, don't call `mem`. It is a tool, not a ceremony. + +## When NOT to reach for it + +- The relevant context is already in the current turn, `prd.md`, `design.md`, recent `git log`, or the open files. `mem` is for stuff that has fallen out of immediate reach. +- The user is asking about a fact in the code, not a fact from a past conversation. `git log -p` / `grep` / reading the file directly is faster and more authoritative. +- You are in a sub-agent (`trellis-implement` / `trellis-check`) whose dispatch prompt already includes the curated `implement.jsonl` / `check.jsonl` context. Adding `mem` on top usually just clutters. +- The user has explicitly said "don't dig through history, just answer what I asked". + +## What to do with what `mem` returns + +Treat the output as **raw material**, not a deliverable. Once you have it, decide based on the live conversation: + +- **Quote inline in your reply** if a specific past exchange answers the user's current question — and cite the session-id / phase so the user can verify. +- **Update `<task>/prd.md` or `<task>/design.md`** if `mem` surfaced a load-bearing decision that should have been written down but wasn't. Surface the proposed edit to the user first. +- **Append to a task-local notes file** (e.g. `<task>/notes.md` or extending an existing one) if the finding belongs to the current task's record but doesn't fit the PRD. +- **Update `.trellis/spec/`** if the finding is a project-wide convention or gotcha that would help future tasks. Run the `trellis-update-spec` skill for that — `session-insight` ends at the discovery. +- **Just absorb it** for the next few turns and answer better, without writing anything. This is often the right move for one-off recall. + +Trellis does not prescribe a single destination. Forcing every recall into a fixed file makes the file grow into noise. Let the situation decide. + +## How to call it + +Full CLI reference is in `references/cli-quick-reference.md`. The 80% case is one of: + +```bash +# Find sessions whose contents mention a keyword (project-scope is default; +# add --global to search every project on this machine). +trellis mem search "<keyword>" + +# Dump dialogue from one session, optionally filtered by phase or keyword. +trellis mem extract <session-id> --phase brainstorm +trellis mem extract <session-id> --grep "<keyword>" + +# Drill into a session: top-N hit turns + surrounding context. +trellis mem context <session-id> --turns 3 --around 2 + +# When you do not know the session id yet, start with list + filter. +trellis mem list --task <task-dir> +trellis mem projects # → list active project cwds, then narrow +``` + +Phase slicing (`--phase brainstorm|implement|all`) cuts the session at `task.py create` and `task.py start` boundaries. For a finish-work review of the current task, `--phase brainstorm` recovers the planning discussion and `--phase implement` recovers the execution loop. Default is `all`. + +## Triggering patterns + +`references/triggering-patterns.md` lists more verbatim user phrasings (English + Chinese) that should make you think "reach for `mem`" — keep that handy when training instinct. + +## Out of scope + +- `mem` does not edit code or update files. Any write-back is your decision in the moment. +- `mem` is read-only on the platform JSONL stores. It does not push or sync to remote. +- This skill does not replace `trellis-update-spec` (which is the right tool for promoting a finding into project-wide guidance) or the platform-native task / spec workflow. diff --git a/.agents/skills/trellis-session-insight/references/cli-quick-reference.md b/.agents/skills/trellis-session-insight/references/cli-quick-reference.md new file mode 100644 index 0000000..3d5f95c --- /dev/null +++ b/.agents/skills/trellis-session-insight/references/cli-quick-reference.md @@ -0,0 +1,66 @@ +# `trellis mem` CLI Reference + +Full flag reference for the five subcommands. Pin this as the authoritative source — `trellis mem help` prints the same content at runtime, so anything here that drifts is a bug. + +## Subcommands + +| Command | Purpose | +|---|---| +| `list` | List sessions. Default subcommand when none is given. | +| `search <keyword>` | Find sessions whose contents match a keyword. | +| `context <session-id>` | Drill into one session: top-N hit turns + surrounding context. Pair with `--grep` for keyword anchoring. | +| `extract <session-id>` | Dump cleaned dialogue. Combine with `--phase` / `--grep` to slice. | +| `projects` | List active project `cwd` values with session counts. Use this to discover which `--cwd` to pass to other subcommands. | + +## Flags (apply where meaningful) + +| Flag | Subcommands | Meaning | +|---|---|---| +| `--platform claude\|codex\|opencode\|all` | all | Default `all`. OpenCode adapter is currently a stub on `0.6.0-beta.*` — see "Caveats" below. | +| `--since YYYY-MM-DD` | list / search | Inclusive lower date bound. | +| `--until YYYY-MM-DD` | list / search | Inclusive upper date bound. | +| `--global` | list / search | Include sessions from every project on this machine. Default is the current project `cwd`. | +| `--cwd <path>` | list / search | Force a specific project cwd instead of inferring from where you are. | +| `--limit N` | list / search | Cap output rows. Default `50`. | +| `--grep KW` | extract / context | Filter turns by keyword. Multi-token AND when whitespace-separated. | +| `--phase brainstorm\|implement\|all` | extract | Slice session by Trellis task boundaries. `brainstorm` = `[task.py create, task.py start)`. `implement` = `[task.py start, task.py finish)` window. Default `all`. | +| `--turns N` | context | Number of hit turns to return. Default `3`. | +| `--around N` | context | Surrounding turns to include per hit. Default `1`. | +| `--max-chars N` | context | Total character budget. Default `6000` (~1500 tokens). | +| `--include-children` | search / context | Merge OpenCode sub-agent sessions into their parent session. | +| `--json` | all | Emit machine-parseable JSON instead of human-readable output. | +| `--task <task-dir>` | list | Narrow to sessions whose context-key resolved to a given task directory (uses `.trellis/.runtime/sessions/*.json`). | + +## Common one-liners + +```bash +# What past sessions discussed "deadlock" anywhere on this machine? +trellis mem search "deadlock" --global --limit 20 + +# Inside a specific session, surface the top 5 turns that mention "lock contention" +# plus 2 turns of surrounding context. +trellis mem context 5842592d --grep "lock contention" --turns 5 --around 2 + +# Recover the brainstorm window for a session — useful when continuing a task +# the user started a week ago. +trellis mem extract 5842592d --phase brainstorm + +# List every project this machine has Trellis sessions for, with counts. +trellis mem projects +``` + +## Output shapes + +- **Default human output** (no `--json`): wrapped to a terminal, with session ids highlighted and turn markers visible. Suitable to read inline but messy to paste into a markdown file. +- **`--json`**: stable schema, safe to parse and process. When piping `mem` output into a follow-up step (e.g. summarizing for a Lessons section), prefer `--json`. + +## Caveats + +- **OpenCode adapter is a stub on `0.6.0-beta.*`.** When `--platform` resolves to OpenCode (or `all` and OpenCode would be included), `mem` prints a one-line "reader unavailable" notice and continues with the other platforms. Don't promise OpenCode coverage in your reply until the adapter ships. +- **`--phase` slicing depends on `task.py create` / `task.py start` invocations appearing in the recorded bash calls of the session.** Sessions where the user ran `task.py` from a different terminal — outside the recorded AI loop — will not have phase boundaries. `--phase all` is the safe fallback. +- **`mem` indexes platform JSONL files directly.** If the user has cleared their Claude / Codex session storage, `mem` cannot recover what is no longer on disk. +- **`mem` is read-only.** No remote sync, no edits to platform JSONL. Any write you do based on `mem` findings is your own follow-up call into the editing tools available to you. + +## When you need more than this reference + +Run `trellis mem help` in the user's shell. The runtime help is authoritative and will be ahead of this reference during fast-moving beta releases. diff --git a/.agents/skills/trellis-session-insight/references/triggering-patterns.md b/.agents/skills/trellis-session-insight/references/triggering-patterns.md new file mode 100644 index 0000000..66021ca --- /dev/null +++ b/.agents/skills/trellis-session-insight/references/triggering-patterns.md @@ -0,0 +1,93 @@ +# Triggering Patterns + +Verbatim user phrasings that should make an AI reach for `trellis mem`. Calibrate instinct against these — if a user message hits one of these patterns and you do not reach for `mem`, you probably missed an obvious recall. + +Patterns are grouped by the *intent* behind the phrasing, not the surface words. The same intent shows up in different languages and registers. + +## Past-solution recall + +The user is asking "how did we (or I) solve this before". Past dialogue holds the answer; the codebase shows the result but not the reasoning. + +- "How did we solve this last time?" +- "What did we end up doing about X?" +- "We dealt with this once already, didn't we?" +- "上次怎么解的?" +- "之前是怎么搞定 X 的?" +- "我记得以前修过类似的" + +Reach: `trellis mem search "<symptom keyword>" --global --limit 10`, then `context` into the hit that looks closest. + +## Decision retrieval + +The user is referencing a decision that lives in old dialogue, not in any committed file. Look in brainstorm windows. + +- "What was the decision on X?" +- "Did we decide to use Postgres or SQLite?" +- "The rationale for choosing X over Y was…?" +- "我们当时为啥选了 X 而不是 Y?" +- "关于 X 我们之前是怎么定的?" +- "之前讨论过 X 的方案吗?" + +Reach: `trellis mem search "<decision keyword>"` to find the session, then `extract <id> --phase brainstorm` to recover the discussion. + +## Cross-session continuation + +The user resumed work after a gap and the context is implicit. + +- "Where were we?" +- "Continue from last time." +- "Pick up where we left off." +- "继续上次的" +- "我们上次做到哪了" +- "接着昨天那个任务" + +Reach: `trellis mem list --task <current-task-dir>` to find the most recent sessions tied to the active task, then `extract` the last one. + +## Familiar-bug debugging + +The current bug feels like one already seen. Past sessions probably hold the resolution path. + +- "I feel like I've hit this before." +- "Doesn't this look like that bug from last month?" +- "Same kind of timeout I had in X." +- "这个错好像之前见过" +- "这个 bug 是不是上次那个?" +- "怎么又是这个 error?" + +Reach: `trellis mem search "<error message fragment>" --global`. Anchor on a short, distinctive token from the actual error string. + +## Self-pattern spotting + +The user is asking whether they keep repeating the same kind of mistake or decision. + +- "Do I always make this mistake?" +- "How often have I run into X?" +- "Is this a recurring thing for me?" +- "我每次都踩这个坑吗?" +- "我老犯这个错?" +- "这类问题之前出现过几次?" + +Reach: `trellis mem search "<topic>" --global --limit 50` and scan the dates / projects in the listing. Optionally `extract` two or three for comparison. + +## Finish-work retrospective (on demand) + +The user explicitly wants to look back at this task — not as a forced step, only when they ask. + +- "Summarize what we did in this task." +- "What were the key decisions / surprises?" +- "Write up the lessons from this round." +- "总结一下这次的经验" +- "记一下这次踩的坑" +- "复盘下这个任务" + +Reach: identify the current task's session id (from `.trellis/.runtime/sessions/*.json` or `mem list --task <task-dir>`), then `extract <id> --phase brainstorm` and `--phase implement`. Present a summary — surface concrete file:line citations where possible. Whether to also write the summary somewhere (PRD, spec, notes file) is the user's call; offer, don't auto-write. + +## Anti-patterns: do NOT reach for `mem` here + +- "What does this function do?" → read the file. +- "Why is this test failing?" → read the test output and the file. +- "What's the right pattern for X in our codebase?" → grep / read spec files. +- "What's the latest npm version of Y?" → call `npm view`. +- "Fix this bug." → debug. Reach for `mem` only if you suspect prior context exists; otherwise it is noise. + +The bar stays: would a senior teammate ask "didn't we already talk about this?" before answering? If yes, reach for `mem`. If no, don't. diff --git a/.agents/skills/trellis-spec-bootstrap/SKILL.md b/.agents/skills/trellis-spec-bootstrap/SKILL.md new file mode 100644 index 0000000..e1650df --- /dev/null +++ b/.agents/skills/trellis-spec-bootstrap/SKILL.md @@ -0,0 +1,41 @@ +--- +name: trellis-spec-bootstrap +description: "Bootstrap project-specific Trellis coding specs with a platform-neutral single-agent workflow. Use when creating or refreshing .trellis/spec guidelines, analyzing a codebase with GitNexus, ABCoder, or source inspection, decomposing package/layer spec work, and writing real codebase-backed spec docs without placeholder text." +--- + +# Trellis Spec Bootstrap + +Use this skill to create or refresh `.trellis/spec/` guidelines from the real codebase. One capable agent owns the full loop: analyze the repository, choose the spec boundaries, write the docs, and verify the result. The workflow does not depend on a specific host, CLI, or agent brand. + +## Workflow + +1. Confirm Trellis is initialized and inspect the current `.trellis/spec/` tree. +2. Analyze the repository architecture with the best available tools: GitNexus, ABCoder, language tooling, and direct source reads. +3. Decompose the spec work by package and layer only when that reflects the actual codebase. +4. Fill or reshape the spec files with concrete patterns, file paths, examples, and anti-patterns from the project. +5. Verify that the final specs are internally consistent and contain no template placeholders. + +## Reference Routing + +| Need | Read | +|------|------| +| Repository architecture analysis | [references/repository-analysis.md](references/repository-analysis.md) | +| Spec work decomposition and task planning | [references/spec-task-planning.md](references/spec-task-planning.md) | +| Writing high-signal Trellis spec files | [references/spec-writing.md](references/spec-writing.md) | +| GitNexus and ABCoder MCP setup | [references/mcp-setup.md](references/mcp-setup.md) | + +## Operating Rules + +- Treat templates as starting points, not contracts. Delete, rename, split, or add spec files when the repository calls for it. +- Prefer source-backed rules over generic advice. Every important recommendation should point at a real file or repeated local pattern. +- Keep execution single-owner by default. Optional helper agents are an implementation detail, not a requirement or user-visible dependency. +- Do not write platform-specific instructions unless the target project already standardizes on that platform. +- Do not leave placeholder text, empty headings, or copied boilerplate in `.trellis/spec/`. + +## Done Criteria + +- `.trellis/spec/` describes the project as it exists now. +- Each relevant package or layer has practical coding guidance with real examples. +- Non-applicable template sections are removed. +- `index.md` files match the final spec file set. +- Any required setup or analysis assumptions are documented in the relevant spec or task notes. diff --git a/.agents/skills/trellis-spec-bootstrap/references/mcp-setup.md b/.agents/skills/trellis-spec-bootstrap/references/mcp-setup.md new file mode 100644 index 0000000..629fcbd --- /dev/null +++ b/.agents/skills/trellis-spec-bootstrap/references/mcp-setup.md @@ -0,0 +1,90 @@ +# MCP Setup + +GitNexus and ABCoder are recommended when bootstrapping Trellis specs because they expose architecture and AST context to the agent. They are tool choices, not platform requirements. Configure them through whatever MCP mechanism your agent host provides. + +## GitNexus + +GitNexus builds a code knowledge graph from the repository. Use it for module boundaries, execution flows, dependency relationships, blast radius, and graph queries. + +### Install and Index + +```bash +# Run from the repository root. +npx gitnexus analyze + +# Check index status. +npx gitnexus status + +# Re-index after code changes when the analysis is stale. +npx gitnexus analyze +``` + +The index is written to `.gitnexus/`. Keep embeddings only if the project already uses them; otherwise a normal index is enough for spec bootstrapping. + +### MCP Server Command + +Use this server command in the host's MCP configuration: + +```bash +npx -y gitnexus mcp +``` + +### Useful Tools + +| Tool | Purpose | +|------|---------| +| `gitnexus_query` | Find execution flows and functional areas by concept | +| `gitnexus_context` | Inspect callers, callees, references, and process participation for a symbol | +| `gitnexus_impact` | Understand blast radius before changing a symbol | +| `gitnexus_detect_changes` | Check changed symbols and affected flows before finishing | +| `gitnexus_cypher` | Run direct graph queries | +| `gitnexus_list_repos` | List indexed repositories | + +## ABCoder + +ABCoder parses code into UniAST and gives precise package, file, and node-level structure. Use it for signatures, type shapes, implementations, dependencies, and reverse references. + +### Install + +```bash +go install github.com/cloudwego/abcoder@latest +abcoder --help +``` + +### Parse Repositories + +```bash +abcoder parse /absolute/path/to/package \ + --lang typescript \ + --name package-name \ + --output ~/abcoder-asts +``` + +For monorepos, parse each package with a stable `--name` so task notes can reference the same repository names. + +### MCP Server Command + +Use this server command in the host's MCP configuration: + +```bash +abcoder mcp ~/abcoder-asts +``` + +### Useful Tools + +| Tool | Layer | Purpose | +|------|-------|---------| +| `list_repos` | 1 | List parsed repositories | +| `get_repo_structure` | 2 | Inspect packages and files | +| `get_package_structure` | 3 | Inspect nodes within a package | +| `get_file_structure` | 3 | Inspect functions, classes, types, and signatures in a file | +| `get_ast_node` | 4 | Retrieve code, dependencies, references, and implementations | + +## Verification + +After configuration, verify from the agent host that both MCP servers are visible. Then run one simple query against each server before starting the spec writing pass. + +```bash +ls .gitnexus/meta.json +ls ~/abcoder-asts/*.json +``` diff --git a/.agents/skills/trellis-spec-bootstrap/references/repository-analysis.md b/.agents/skills/trellis-spec-bootstrap/references/repository-analysis.md new file mode 100644 index 0000000..1309d29 --- /dev/null +++ b/.agents/skills/trellis-spec-bootstrap/references/repository-analysis.md @@ -0,0 +1,59 @@ +# Repository Analysis + +The goal is to discover the project's real architecture before writing rules. Do not start from generic spec templates and fill blanks. Start from the code, then let the spec structure follow. + +## Analysis Order + +1. Read the existing `.trellis/spec/` tree and note which files are templates, outdated, or already project-specific. +2. Inspect package manifests, build scripts, workspace config, and top-level documentation to identify packages and runtime layers. +3. Use GitNexus for execution flows, module clusters, dependency hubs, and impact-sensitive areas. +4. Use ABCoder or language-native tooling for exact signatures, types, class boundaries, and implementation examples. +5. Read representative source and test files directly before turning any finding into a spec rule. + +## What To Capture + +| Area | Questions | +|------|-----------| +| Package boundaries | What does each package own? What imports cross boundaries? | +| Runtime layers | Which code is CLI, backend, frontend, worker, shared library, test-only, or tooling? | +| Core abstractions | Which types, services, stores, commands, routes, or adapters define the system shape? | +| Data flow | Where does user input enter, how is it validated, and where does state persist? | +| Error handling | How are failures represented, logged, surfaced, and tested? | +| Configuration | Where do defaults, environment config, generated files, and templates live? | +| Tests | Which test styles are trusted examples for new work? | + +## GitNexus Usage + +Start broad, then inspect specific symbols: + +```text +gitnexus_query({query: "CLI command execution flow"}) +gitnexus_query({query: "template generation and migration"}) +gitnexus_context({name: "SymbolName"}) +gitnexus_cypher({query: "MATCH (n)-[r]->(m) RETURN n.name, type(r), m.name LIMIT 30"}) +``` + +Use GitNexus results to find important files and flows. Do not quote graph output as the final authority until you have checked the relevant source files. + +## ABCoder Usage + +Use ABCoder when the spec needs exact code shapes: + +```text +list_repos() +get_repo_structure({repo_name: "package-name"}) +get_file_structure({repo_name: "package-name", file_path: "src/example.ts"}) +get_ast_node({repo_name: "package-name", node_ids: [{mod_path: "...", pkg_path: "...", name: "SymbolName"}]}) +``` + +ABCoder is most valuable for documenting constructor patterns, function signatures, type contracts, and reference chains. + +## Analysis Notes + +Keep short notes while analyzing. The notes should include: + +- Package or layer name. +- Files that define the local pattern. +- Rules the spec should teach. +- Anti-patterns found in old code, comments, tests, or migration paths. +- Spec files that should be created, deleted, renamed, or merged. diff --git a/.agents/skills/trellis-spec-bootstrap/references/spec-task-planning.md b/.agents/skills/trellis-spec-bootstrap/references/spec-task-planning.md new file mode 100644 index 0000000..dca2687 --- /dev/null +++ b/.agents/skills/trellis-spec-bootstrap/references/spec-task-planning.md @@ -0,0 +1,61 @@ +# Spec Task Planning + +Use a single agent as the default execution model. The agent may create Trellis tasks for traceability, but the skill should not require a specific platform, CLI, or parallel worker model. + +## Decomposition + +Create spec work units around real ownership boundaries: + +- One package when a package has its own conventions. +- One layer when the same package has distinct frontend, backend, CLI, worker, or shared-library rules. +- One cross-cutting guide when a pattern spans packages and is not owned by one layer. + +Avoid artificial decomposition. A small library usually needs one focused spec pass, not several tasks. + +## Task Shape + +When a Trellis task is useful, write a concise PRD with these sections: + +```markdown +# Fill <package-or-layer> Trellis Specs + +## Goal +Write project-specific `.trellis/spec/` guidance for <scope>. + +## Scope +- Spec directory: +- Source directories to inspect: +- Tests to inspect: +- Out of scope: + +## Architecture Context +Summarize the concrete findings from repository analysis. + +## Files To Create Or Update +- `.trellis/spec/.../index.md` +- `.trellis/spec/.../<topic>.md` + +## Rules +- Adapt the spec file set to the real codebase. +- Use real source examples with file paths. +- Remove template-only sections that do not apply. +- Do not modify product source code unless the task explicitly asks for it. + +## Acceptance Criteria +- [ ] Specs contain concrete examples and anti-patterns from the repository. +- [ ] No placeholder text remains. +- [ ] Index files match the final spec files. +- [ ] Claims are backed by source files, tests, or project docs. +``` + +## Optional Helper Agents + +If the host supports subagents, helpers can inspect independent packages or run verification. They are optional. The main agent still owns integration and final quality. + +Helper tasks must have clear ownership: + +- Read-only research tasks may inspect any source needed for the assigned scope. +- Write tasks should own disjoint spec directories. +- Verification tasks should check placeholder removal, broken links, and consistency. + +Do not encode helper-agent names, vendor-specific commands, or platform-specific routing in the skill. Put only the required work and acceptance criteria in the task. diff --git a/.agents/skills/trellis-spec-bootstrap/references/spec-writing.md b/.agents/skills/trellis-spec-bootstrap/references/spec-writing.md new file mode 100644 index 0000000..6bc7dec --- /dev/null +++ b/.agents/skills/trellis-spec-bootstrap/references/spec-writing.md @@ -0,0 +1,70 @@ +# Spec Writing + +Trellis specs are coding guidance for future agents. They should explain how to work in this repository, not how a generic project might be organized. + +## Write From Evidence + +Each important rule should be backed by one of these: + +- A source file that demonstrates the preferred pattern. +- A test file that shows expected behavior. +- A project document that defines the convention. +- A repeated pattern across multiple files. + +Use short snippets only when they make the rule clearer. Prefer linking to the file path and naming the symbol or behavior. + +## File Structure + +Keep the spec tree aligned with the project: + +- Keep `index.md` as the navigation file for the spec directory. +- Split topics when developers would look for them independently. +- Merge topics when separate files would repeat the same rule. +- Delete template files that do not apply. +- Add new files for important local patterns the template missed. + +## Content Standards + +Good spec sections include: + +- When the rule applies. +- The local pattern to follow. +- The source or test files that prove the pattern. +- Common mistakes or anti-patterns. +- Verification commands or checks when they are specific and reliable. + +Avoid: + +- Placeholder prose. +- Generic framework advice. +- Tool instructions that only work in one agent host. +- Long copied code blocks. +- Rules based on a single accidental implementation detail. + +## Example Shape + +```markdown +## Command Handlers + +Command handlers should keep argument parsing, validation, and side effects separate. The local pattern is: + +- Parse CLI flags at the command boundary. +- Convert raw inputs into typed task options before invoking core logic. +- Keep filesystem writes in the command or service layer, not in template helpers. + +Reference files: +- `packages/cli/src/commands/example.ts` +- `packages/cli/test/commands/example.test.ts` + +Avoid passing raw `process.argv` or unvalidated config objects into shared helpers. +``` + +## Final Pass + +Before finishing: + +```bash +grep -R "To be filled\\|TODO: fill\\|placeholder" .trellis/spec +``` + +Also check links, index files, and whether any spec still describes a template rather than this repository. diff --git a/.agents/skills/trellis-start/SKILL.md b/.agents/skills/trellis-start/SKILL.md new file mode 100644 index 0000000..e557bff --- /dev/null +++ b/.agents/skills/trellis-start/SKILL.md @@ -0,0 +1,64 @@ +--- +name: trellis-start +description: "Initializes an AI development session by reading workflow guides, developer identity, git status, active tasks, and project guidelines from .trellis/. Classifies incoming tasks and routes to brainstorm, direct edit, or task workflow. Use when beginning a new coding session, resuming work, starting a new task, or re-establishing project context." +--- + +# Start Session + +Initialize a Trellis-managed development session. This platform has no session-start hook, so manually load the equivalent compact context by following these steps. + +--- + +## Step 1: Current state +Identity, git status, current task, active tasks, journal location. + +```bash +python ./.trellis/scripts/get_context.py +``` + +If this output includes a line beginning `Trellis update available:`, copy the full line verbatim when summarizing session context. Do not shorten operational command hints. + +## Step 2: Workflow overview +Compact Phase Index, request triage rules, planning artifact contract, and the step-detail command. + +```bash +python ./.trellis/scripts/get_context.py --mode phase +``` + +Full guide in `.trellis/workflow.md` (read on demand). + +## Step 3: Guideline indexes +Discover packages + spec layers, then read each relevant index file. + +```bash +python ./.trellis/scripts/get_context.py --mode packages +cat .trellis/spec/guides/index.md +cat .trellis/spec/<package>/<layer>/index.md # for each relevant layer +``` + +Index files list the specific guideline docs to read when you actually start coding. + +## Step 4: Decide next action +From Step 1 you know the current task and status. Check the task directory: + +- **Active task status `planning` + no `prd.md`** → Phase 1.1. Load the `trellis-brainstorm` skill. +- **Active task status `planning` + `prd.md` exists** → stay in Phase 1. Lightweight tasks can be PRD-only; complex tasks need `design.md` + `implement.md`. Load the relevant Phase 1 step detail before `task.py start`. +- **Active task status `in_progress`** → Phase 2 step 2.1. Load the step detail: + ```bash + python ./.trellis/scripts/get_context.py --mode phase --step 2.1 --platform codex + ``` +- **No active task** → classify first. For simple conversation / small task, ask only whether this turn should create a Trellis task. For complex work, ask whether you may create a Trellis task and enter planning. If the user says no, skip Trellis for this session. + +--- + +## Skill routing (quick reference) + +| User intent | Skill | +|---|---| +| New feature / unclear requirements | `trellis-brainstorm` | +| About to write code | `trellis-before-dev` | +| Done coding / quality check | `trellis-check` | +| Stuck / fixed same bug multiple times | `trellis-break-loop` | +| Learned something worth capturing | `trellis-update-spec` | + +Full rules + anti-rationalization table in `.trellis/workflow.md`. diff --git a/.agents/skills/trellis-update-spec/SKILL.md b/.agents/skills/trellis-update-spec/SKILL.md new file mode 100644 index 0000000..81bad08 --- /dev/null +++ b/.agents/skills/trellis-update-spec/SKILL.md @@ -0,0 +1,356 @@ +--- +name: trellis-update-spec +description: "Captures executable contracts and coding conventions into .trellis/spec/ documents. Use when learning something valuable from debugging, implementing, or discussion that should be preserved for future sessions." +--- + +# Update Code-Spec - Capture Executable Contracts + +When you learn something valuable (from debugging, implementing, or discussion), use this to update the relevant code-spec documents. + +**Timing**: After completing a task, fixing a bug, or discovering a new pattern + +--- + +## Code-Spec First Rule (CRITICAL) + +In this project, "spec" for implementation work means **code-spec**: +- Executable contracts (not principle-only text) +- Concrete signatures, payload fields, env keys, and boundary behavior +- Testable validation/error behavior + +If the change touches infra or cross-layer contracts, code-spec depth is mandatory. + +### Mandatory Triggers + +Apply code-spec depth when the change includes any of: +- New/changed command or API signature +- Cross-layer request/response contract change +- Database schema/migration change +- Infra integration (storage, queue, cache, secrets, env wiring) + +### Mandatory Output (7 Sections) + +For triggered tasks, include all sections below: +1. Scope / Trigger +2. Signatures (command/API/DB) +3. Contracts (request/response/env) +4. Validation & Error Matrix +5. Good/Base/Bad Cases +6. Tests Required (with assertion points) +7. Wrong vs Correct (at least one pair) + +--- + +## When to Update Code-Specs + +| Trigger | Example | Target Spec | +|---------|---------|-------------| +| **Implemented a feature** | Added a new integration or module | Relevant spec file | +| **Made a design decision** | Chose extensibility pattern over simplicity | Relevant spec + "Design Decisions" section | +| **Fixed a bug** | Found a subtle issue with error handling | Relevant spec (e.g., error-handling docs) | +| **Discovered a pattern** | Found a better way to structure code | Relevant spec file | +| **Hit a gotcha** | Learned that X must be done before Y | Relevant spec + "Common Mistakes" section | +| **Established a convention** | Team agreed on naming pattern | Quality guidelines | +| **New thinking trigger** | "Don't forget to check X before doing Y" | `guides/*.md` (as a checklist item) | + +**Key Insight**: Code-spec updates are NOT just for problems. Every feature implementation contains design decisions and contracts that future AI/developers need to execute safely. + +--- + +## Spec Structure Overview + +``` +.trellis/spec/ +├── <layer>/ # Per-layer coding standards (e.g., backend/, frontend/, api/) +│ ├── index.md # Overview and links +│ └── *.md # Topic-specific guidelines +└── guides/ # Thinking checklists (NOT coding specs!) + ├── index.md # Guide index + └── *.md # Topic-specific guides +``` + +### CRITICAL: Code-Spec vs Guide - Know the Difference + +| Type | Location | Purpose | Content Style | +|------|----------|---------|---------------| +| **Code-Spec** | `<layer>/*.md` | Tell AI "how to implement safely" | Signatures, contracts, matrices, cases, test points | +| **Guide** | `guides/*.md` | Help AI "what to think about" | Checklists, questions, pointers to specs | + +**Decision Rule**: Ask yourself: + +- "This is **how to write** the code" → Put in a spec layer directory +- "This is **what to consider** before writing" → Put in `guides/` + +**Example**: + +| Learning | Wrong Location | Correct Location | +|----------|----------------|------------------| +| "Use API X not API Y for this task" | ❌ `guides/` (too specific for a thinking guide) | ✅ Relevant spec file (concrete convention) | +| "Remember to check X when doing Y" | ❌ Spec file (too abstract for a spec) | ✅ `guides/` (thinking checklist) | + +**Guides should be short checklists that point to specs**, not duplicate the detailed rules. + +--- + +## Update Process + +### Step 1: Identify What You Learned + +Answer these questions: + +1. **What did you learn?** (Be specific) +2. **Why is it important?** (What problem does it prevent?) +3. **Where does it belong?** (Which spec file?) + +### Step 2: Classify the Update Type + +| Type | Description | Action | +|------|-------------|--------| +| **Design Decision** | Why we chose approach X over Y | Add to "Design Decisions" section | +| **Project Convention** | How we do X in this project | Add to relevant section with examples | +| **New Pattern** | A reusable approach discovered | Add to "Patterns" section | +| **Forbidden Pattern** | Something that causes problems | Add to "Anti-patterns" or "Don't" section | +| **Common Mistake** | Easy-to-make error | Add to "Common Mistakes" section | +| **Convention** | Agreed-upon standard | Add to relevant section | +| **Gotcha** | Non-obvious behavior | Add warning callout | + +### Step 3: Read the Target Code-Spec + +Before editing, read the current code-spec to: +- Understand existing structure +- Avoid duplicating content +- Find the right section for your update + +```bash +cat .trellis/spec/<category>/<file>.md +``` + +### Step 4: Make the Update + +Follow these principles: + +1. **Be Specific**: Include concrete examples, not just abstract rules +2. **Explain Why**: State the problem this prevents +3. **Show Contracts**: Add signatures, payload fields, and error behavior +4. **Show Code**: Add code snippets for key patterns +5. **Keep it Short**: One concept per section + +### Step 5: Update the Index (if needed) + +If you added a new section or the code-spec status changed, update the category's `index.md`. + +--- + +## Update Templates + +### Mandatory Template for Infra/Cross-Layer Work + +```markdown +## Scenario: <name> + +### 1. Scope / Trigger +- Trigger: <why this requires code-spec depth> + +### 2. Signatures +- Backend command/API/DB signature(s) + +### 3. Contracts +- Request fields (name, type, constraints) +- Response fields (name, type, constraints) +- Environment keys (required/optional) + +### 4. Validation & Error Matrix +- <condition> -> <error> + +### 5. Good/Base/Bad Cases +- Good: ... +- Base: ... +- Bad: ... + +### 6. Tests Required +- Unit/Integration/E2E with assertion points + +### 7. Wrong vs Correct +#### Wrong +... +#### Correct +... +``` + +### Adding a Design Decision + +```markdown +### Design Decision: [Decision Name] + +**Context**: What problem were we solving? + +**Options Considered**: +1. Option A - brief description +2. Option B - brief description + +**Decision**: We chose Option X because... + +**Example**: +\`\`\`typescript +// How it's implemented +code example +\`\`\` + +**Extensibility**: How to extend this in the future... +``` + +### Adding a Project Convention + +```markdown +### Convention: [Convention Name] + +**What**: Brief description of the convention. + +**Why**: Why we do it this way in this project. + +**Example**: +\`\`\`typescript +// How to follow this convention +code example +\`\`\` + +**Related**: Links to related conventions or specs. +``` + +### Adding a New Pattern + +```markdown +### Pattern Name + +**Problem**: What problem does this solve? + +**Solution**: Brief description of the approach. + +**Example**: +\`\`\` +// Good +code example + +// Bad +code example +\`\`\` + +**Why**: Explanation of why this works better. +``` + +### Adding a Forbidden Pattern + +```markdown +### Don't: Pattern Name + +**Problem**: +\`\`\` +// Don't do this +bad code example +\`\`\` + +**Why it's bad**: Explanation of the issue. + +**Instead**: +\`\`\` +// Do this instead +good code example +\`\`\` +``` + +### Adding a Common Mistake + +```markdown +### Common Mistake: Description + +**Symptom**: What goes wrong + +**Cause**: Why this happens + +**Fix**: How to correct it + +**Prevention**: How to avoid it in the future +``` + +### Adding a Gotcha + +```markdown +> **Warning**: Brief description of the non-obvious behavior. +> +> Details about when this happens and how to handle it. +``` + +--- + +## Interactive Mode + +If you're unsure what to update, answer these prompts: + +1. **What did you just finish?** + - [ ] Fixed a bug + - [ ] Implemented a feature + - [ ] Refactored code + - [ ] Had a discussion about approach + +2. **What did you learn or decide?** + - Design decision (why X over Y) + - Project convention (how we do X) + - Non-obvious behavior (gotcha) + - Better approach (pattern) + +3. **Would future AI/developers need to know this?** + - To understand how the code works → Yes, update spec + - To maintain or extend the feature → Yes, update spec + - To avoid repeating mistakes → Yes, update spec + - Purely one-off implementation detail → Maybe skip + +4. **Which area does it relate to?** + - [ ] Backend code + - [ ] Frontend code + - [ ] Cross-layer data flow + - [ ] Code organization/reuse + - [ ] Quality/testing + +--- + +## Quality Checklist + +Before finishing your code-spec update: + +- [ ] Is the content specific and actionable? +- [ ] Did you include a code example? +- [ ] Did you explain WHY, not just WHAT? +- [ ] Did you include executable signatures/contracts? +- [ ] Did you include validation and error matrix? +- [ ] Did you include Good/Base/Bad cases? +- [ ] Did you include required tests with assertion points? +- [ ] Is it in the right code-spec file? +- [ ] Does it duplicate existing content? +- [ ] Would a new team member understand it? + +--- + +## Relationship to Other Commands + +``` +Development Flow: + Learn something → `update-spec` (Trellis command) → Knowledge captured + ↑ ↓ + `break-loop` (Trellis command) ←──────────────────── Future sessions benefit + (deep bug analysis) +``` + +- ``break-loop` (Trellis command)` - Analyzes bugs deeply, often reveals spec updates needed +- ``update-spec` (Trellis command)` - Actually makes the updates +- ``finish-work` (Trellis command)` - Reminds you to check if specs need updates + +--- + +## Core Philosophy + +> **Code-specs are living documents. Every debugging session, every "aha moment" is an opportunity to make the implementation contract clearer.** + +The goal is **institutional memory**: +- What one person learns, everyone benefits from +- What AI learns in one session, persists to future sessions +- Mistakes become documented guardrails diff --git a/.claude/agents/trellis-check.md b/.claude/agents/trellis-check.md new file mode 100644 index 0000000..7883deb --- /dev/null +++ b/.claude/agents/trellis-check.md @@ -0,0 +1,115 @@ +--- +name: trellis-check +description: | + Code quality check expert. Reviews code changes against specs and self-fixes issues. +tools: Read, Write, Edit, Bash, Glob, Grep, mcp__exa__web_search_exa, mcp__exa__get_code_context_exa +--- +# Check Agent + +You are the Check Agent in the Trellis workflow. + +## Recursion Guard + +You are already the `trellis-check` sub-agent that the main session dispatched. Do the review and fixes directly. + +- Do NOT spawn another `trellis-check` or `trellis-implement` sub-agent. +- If SessionStart context, workflow-state breadcrumbs, or workflow.md say to dispatch `trellis-implement` / `trellis-check`, treat that as a main-session instruction that is already satisfied by your current role. +- Only the main session may dispatch Trellis implement/check agents. If more implementation work is needed, report that recommendation instead of spawning. + +## Trellis Context Loading Protocol + +Look for the `<!-- trellis-hook-injected -->` marker in your input above. + +- **If the marker is present**: task artifacts, spec, and research files have already been auto-loaded for you above. Proceed with the check work directly. +- **If the marker is absent**: hook injection didn't fire (Windows + Claude Code, `--continue` resume, fork distribution, hooks disabled, etc.). Find the active task path from your dispatch prompt's first line `Active task: <path>`, then Read `<task-path>/check.jsonl`, each listed file, `<task-path>/prd.md`, `<task-path>/design.md` if present, and `<task-path>/implement.md` if present before doing the work. + +## Context + +Before checking, read: +- `.trellis/spec/` - Development guidelines +- Task `prd.md` - Requirements document +- Task `design.md` - Technical design (if exists) +- Task `implement.md` - Execution plan (if exists) +- Pre-commit checklist for quality standards + +## Core Responsibilities + +1. **Get code changes** - Use git diff to get uncommitted code +2. **Review task artifacts** - Check changes against prd.md, design.md if present, and implement.md if present +3. **Check against specs** - Verify code follows guidelines +4. **Self-fix** - Fix issues yourself, not just report them +5. **Run verification** - typecheck and lint + +## Important + +**Fix issues yourself**, don't just report them. + +You have write and edit tools, you can modify code directly. + +--- + +## Workflow + +### Step 1: Get Changes + +```bash +git diff --name-only # List changed files +git diff # View specific changes +``` + +### Step 2: Check Against Specs and Task Artifacts + +Read the task's prd.md, design.md if present, and implement.md if present, then read relevant specs in `.trellis/spec/` to check code: + +- Does it satisfy the task requirements +- Does it follow the technical design and implementation plan when present +- Does it follow directory structure conventions +- Does it follow naming conventions +- Does it follow code patterns +- Are there missing types +- Are there potential bugs + +### Step 3: Self-Fix + +After finding issues: + +1. Fix the issue directly (use edit tool) +2. Record what was fixed +3. Continue checking other issues + +### Step 4: Run Verification + +Run project's lint and typecheck commands to verify changes. + +If failed, fix issues and re-run. + +--- + +## Report Format + +```markdown +## Self-Check Complete + +### Files Checked + +- src/components/Feature.tsx +- src/hooks/useFeature.ts + +### Issues Found and Fixed + +1. `<file>:<line>` - <what was fixed> +2. `<file>:<line>` - <what was fixed> + +### Issues Not Fixed + +(If there are issues that cannot be self-fixed, list them here with reasons) + +### Verification Results + +- TypeCheck: Passed +- Lint: Passed + +### Summary + +Checked X files, found Y issues, all fixed. +``` diff --git a/.claude/agents/trellis-implement.md b/.claude/agents/trellis-implement.md new file mode 100644 index 0000000..37e1b96 --- /dev/null +++ b/.claude/agents/trellis-implement.md @@ -0,0 +1,110 @@ +--- +name: trellis-implement +description: | + Code implementation expert. Understands specs and requirements, then implements features. No git commit allowed. +tools: Read, Write, Edit, Bash, Glob, Grep, mcp__exa__web_search_exa, mcp__exa__get_code_context_exa +--- +# Implement Agent + +You are the Implement Agent in the Trellis workflow. + +## Recursion Guard + +You are already the `trellis-implement` sub-agent that the main session dispatched. Do the implementation work directly. + +- Do NOT spawn another `trellis-implement` or `trellis-check` sub-agent. +- If SessionStart context, workflow-state breadcrumbs, or workflow.md say to dispatch `trellis-implement` / `trellis-check`, treat that as a main-session instruction that is already satisfied by your current role. +- Only the main session may dispatch Trellis implement/check agents. If more parallel work is needed, report that recommendation instead of spawning. + +## Trellis Context Loading Protocol + +Look for the `<!-- trellis-hook-injected -->` marker in your input above. + +- **If the marker is present**: prd / spec / research files have already been auto-loaded for you above. Proceed with the implementation work directly. +- **If the marker is absent**: hook injection didn't fire (Windows + Claude Code, `--continue` resume, fork distribution, hooks disabled, etc.). Find the active task path from your dispatch prompt's first line `Active task: <path>`, then Read `<task-path>/implement.jsonl`, each listed file, `<task-path>/prd.md`, `<task-path>/design.md` if present, and `<task-path>/implement.md` if present before doing the work. + +## Context + +Before implementing, read: +- `.trellis/workflow.md` - Project workflow +- `.trellis/spec/` - Development guidelines +- Task `prd.md` - Requirements document +- Task `design.md` - Technical design (if exists) +- Task `implement.md` - Execution plan (if exists) + +## Core Responsibilities + +1. **Understand specs** - Read relevant spec files in `.trellis/spec/` +2. **Understand task artifacts** - Read prd.md, design.md if present, and implement.md if present +3. **Implement features** - Write code following specs and task artifacts +4. **Self-check** - Ensure code quality +5. **Report results** - Report completion status + +## Forbidden Operations + +**Do NOT execute these git commands:** + +- `git commit` +- `git push` +- `git merge` + +--- + +## Workflow + +### 1. Understand Specs + +Read relevant specs based on task type: + +- Spec layers: `.trellis/spec/<package>/<layer>/` +- Shared guides: `.trellis/spec/guides/` + +### 2. Understand Requirements + +Read the task's prd.md, design.md if present, and implement.md if present: + +- What are the core requirements +- Key points of technical design +- Implementation order, validation commands, and rollback points + +### 3. Implement Features + +- Write code following specs and task artifacts +- Follow existing code patterns +- Only do what's required, no over-engineering + +### 4. Verify + +Run project's lint and typecheck commands to verify changes. + +--- + +## Report Format + +```markdown +## Implementation Complete + +### Files Modified + +- `src/components/Feature.tsx` - New component +- `src/hooks/useFeature.ts` - New hook + +### Implementation Summary + +1. Created Feature component... +2. Added useFeature hook... + +### Verification Results + +- Lint: Passed +- TypeCheck: Passed +``` + +--- + +## Code Standards + +- Follow existing code patterns +- Don't add unnecessary abstractions +- Only do what's required, no over-engineering +- Keep code readable diff --git a/.claude/agents/trellis-research.md b/.claude/agents/trellis-research.md new file mode 100644 index 0000000..4d984de --- /dev/null +++ b/.claude/agents/trellis-research.md @@ -0,0 +1,137 @@ +--- +name: trellis-research +description: | + Code and tech search expert. Finds files, patterns, and tech solutions, and PERSISTS every finding to the current task's research/ directory. No code modifications outside that directory. +tools: Read, Write, Glob, Grep, Bash, mcp__exa__web_search_exa, mcp__exa__get_code_context_exa, Skill, mcp__chrome-devtools__* +--- +# Research Agent + +You are the Research Agent in the Trellis workflow. + +## Core Principle + +**You do one thing: find, explain, and PERSIST information.** + +Conversations get compacted; files don't. Every research output MUST end up as a file under `{TASK_DIR}/research/`. Returning findings only through the chat reply is a failure — the caller cannot read them next session. + +--- + +## Core Responsibilities + +1. **Internal Search** — locate files/components, understand code logic, discover patterns (Glob, Grep, Read) +2. **External Search** — library docs, API references, best practices (web search) +3. **Persist** — write each research topic to `{TASK_DIR}/research/<topic>.md` +4. **Report** — return file paths + one-line summaries to the main agent (not full content) + +--- + +## Workflow + +### Step 1: Resolve Current Task + +Run `python ./.trellis/scripts/task.py current --source` → active task path. If no active task is set, ask the user where to write output; do NOT guess. + +Ensure `{TASK_DIR}/research/` exists: + +```bash +mkdir -p <TASK_DIR>/research +``` + +### Step 2: Understand Search Request + +Classify: internal / external / mixed. Determine scope (global / specific directory) and expected shape (file list / pattern notes / tech comparison). + +### Step 3: Execute Search + +Run independent searches in parallel (Glob + Grep + web) for efficiency. + +### Step 4: Persist Each Topic + +For each distinct research topic, Write a markdown file at `{TASK_DIR}/research/<topic-slug>.md`. Use the File Format below. + +### Step 5: Report to Main Agent + +Reply with ONLY: + +- List of files written (paths relative to repo root) +- One-line summary per file +- Any critical caveats that the main agent needs to know right now + +Do NOT paste full research content into the reply. The files are the contract. + +--- + +## Scope Limits (Strict) + +### Write ALLOWED + +- `{TASK_DIR}/research/*.md` — your own output +- Creating `{TASK_DIR}/research/` if it doesn't exist (via `mkdir -p`) + +### Write FORBIDDEN + +- Code files (`src/`, `lib/`, …) +- Spec files (`.trellis/spec/`) — main agent should use `update-spec` skill instead +- `.trellis/scripts/`, `.trellis/workflow.md`, platform config (`.claude/`, `.cursor/`, etc.) +- Other task directories +- Any git operation (commit / push / branch / merge) + +If the user asks you to edit code, decline and suggest spawning `implement` instead. + +--- + +## File Format + +Each `{TASK_DIR}/research/<topic>.md` should follow: + +```markdown +# Research: <topic> + +- **Query**: <original query> +- **Scope**: <internal / external / mixed> +- **Date**: <YYYY-MM-DD> + +## Findings + +### Files Found + +| File Path | Description | +|---|---| +| `src/services/xxx.ts` | Main implementation | +| `src/types/xxx.ts` | Type definitions | + +### Code Patterns + +<describe patterns, cite file:line> + +### External References + +- [Library X docs](url) — <why relevant, version constraints> + +### Related Specs + +- `.trellis/spec/xxx.md` — <description> + +## Caveats / Not Found + +<anything incomplete or uncertain> +``` + +--- + +## Guidelines + +### DO + +- Provide specific file paths and line numbers +- Quote actual code snippets +- Persist every topic to its own file +- Return file paths in your reply, not the full content +- Mark "not found" explicitly when searches come up empty + +### DON'T + +- Don't write code or modify files outside `{TASK_DIR}/research/` +- Don't guess uncertain info +- Don't paste full research text into the reply (files are the deliverable) +- Don't propose improvements or critique implementation (that's not your role) diff --git a/.claude/commands/trellis/continue.md b/.claude/commands/trellis/continue.md new file mode 100644 index 0000000..b83d926 --- /dev/null +++ b/.claude/commands/trellis/continue.md @@ -0,0 +1,56 @@ +# Continue Current Task + +Resume work on the current task — pick up at the right phase/step in `.trellis/workflow.md`. + +--- + +## Step 1: Load Current Context + +```bash +python ./.trellis/scripts/get_context.py +``` + +Confirms: current task, git state, recent commits. + +## Step 2: Load the Phase Index + +```bash +python ./.trellis/scripts/get_context.py --mode phase +``` + +Shows the Phase Index (Plan / Execute / Finish) with routing + skill mapping. + +## Step 3: Decide Where You Are + +`get_context.py` shows the active task's `status` field. Route by `status` + artifact presence. This command replaces the user needing to remember the Trellis flow; it does not itself approve implementation. + +- `status=planning` + no `prd.md` → **1.1** (load `trellis-brainstorm`) +- `status=planning` + `prd.md` only → decide whether the task is lightweight or complex. Lightweight can move to **1.4** review; complex returns to **1.1** to add `design.md` + `implement.md`. +- `status=planning` + complex artifacts complete + sub-agent jsonl not curated (only the seed `_example` row) → **1.3** +- `status=planning` + required artifacts complete + required jsonl curated or inline mode → **1.4** (ask for start review; only run `task.py start` after user confirms) +- `status=in_progress` + implementation not started → **2.1** +- `status=in_progress` + implementation done, not yet checked → **2.2** +- `status=in_progress` + check passed → **3.1** +- `status=completed` (rare; usually archived immediately) → archive flow + +Phase rules (full detail in `.trellis/workflow.md`): + +1. Run steps **in order** within a phase — `[required]` steps must not be skipped +2. `[once]` steps are already done if the required output exists. `prd.md` alone can be enough only for lightweight tasks; complex tasks also need `design.md` and `implement.md`. +3. You may go back to an earlier phase if discoveries require it + +## Step 4: Load the Specific Step + +Once you know which step to resume at: + +```bash +python ./.trellis/scripts/get_context.py --mode phase --step <X.X> --platform claude +``` + +Follow the loaded instructions. After each `[required]` step completes, move to the next. + +--- + +## Reference + +Full workflow and detailed phase steps live in `.trellis/workflow.md`. This command is only an entry point — the canonical guidance is there. diff --git a/.claude/commands/trellis/finish-work.md b/.claude/commands/trellis/finish-work.md new file mode 100644 index 0000000..f095dcb --- /dev/null +++ b/.claude/commands/trellis/finish-work.md @@ -0,0 +1,66 @@ +# Finish Work + +Wrap up the current session: archive the active task (and any other completed-but-unarchived tasks the user wants to clean up) and record the session journal. Code commits are NOT done here — those happen in workflow Phase 3.4 before you invoke this command. + +## Step 1: Survey current state + +```bash +python ./.trellis/scripts/get_context.py --mode record +``` + +This prints: + +- **My active tasks** — review whether any besides the current one are actually done (code merged, AC met) and should be archived this round. +- **Git status** — quick visual on what's dirty. +- **Recent commits** — you'll need their hashes in Step 4 for `--commit`. + +If `--mode record` surfaces other completed tasks not tied to the current session, surface them to the user with a one-shot confirmation: "These N tasks look done — archive them too in this round? [y/N]". Default is no; the current active task is always archived in Step 3 regardless. + +## Step 2: Sanity check — classify dirty paths + +Run: + +```bash +git status --porcelain +``` + +Filter out paths under `.trellis/workspace/` and `.trellis/tasks/` — those are managed by `add_session.py` and `task.py archive` auto-commits and will appear dirty as part of this skill's own work. + +For each remaining dirty path, decide whether it belongs to **the current task** or to **other parallel work** (e.g., another terminal window editing the same repo). Heuristics: + +- Paths referenced in the current task's `prd.md` / `implement.jsonl` / `check.jsonl` → current task +- Paths in code areas matching the task's stated scope, or that you remember editing this session → current task +- Paths in unrelated areas you have no recollection of touching this session → other parallel work + +Then route: + +- **Any remaining path looks like current-task work** — bail out with: + > "Working tree has uncommitted code changes from this task: `<list>`. Return to workflow Phase 3.4 to commit them before running `/trellis:finish-work`." + + Do NOT run `git commit` here. Do NOT prompt the user to commit. The user goes back to Phase 3.4 and the AI drives the batched commit there. +- **All remaining paths look unrelated** (other parallel-window work) — report them once and continue to Step 3: + > "FYI, dirty files outside this task's scope — leaving them for the other window: `<list>`." +- **Genuinely unsure** — ask the user once: "Are `<list>` this task's work I forgot to commit, or another window's? (commit / ignore)" — then route per their answer. + +## Step 3: Archive task(s) + +```bash +python ./.trellis/scripts/task.py archive <task-name> +``` + +At minimum: the current active task (if any). Plus any extra tasks the user confirmed in Step 1. Each archive produces a `chore(task): archive ...` commit via the script's auto-commit. + +If there is no active task and the user did not confirm any cleanup archives, skip this step. + +## Step 4: Record session journal + +```bash +python ./.trellis/scripts/add_session.py \ + --title "Session Title" \ + --commit "hash1,hash2" \ + --summary "Brief summary" +``` + +Use the work-commit hashes produced in Phase 3.4 (visible in Step 1's `Recent commits` list, or via `git log --oneline`) for `--commit`. Do not include the archive commit hashes from Step 3. This produces a `chore: record journal` commit. + +Final git log order: `<work commits from 3.4>` → `chore(task): archive ...` (one or more) → `chore: record journal`. diff --git a/.claude/hooks/inject-subagent-context.py b/.claude/hooks/inject-subagent-context.py new file mode 100644 index 0000000..9547ca9 --- /dev/null +++ b/.claude/hooks/inject-subagent-context.py @@ -0,0 +1,771 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Multi-Platform Sub-Agent Context Injection Hook + +Injects task-specific context when sub-agents (implement, check, research) are spawned. + +Core Design Philosophy: +- Hook is responsible for injecting all context, subagent works autonomously with complete info +- Each agent has a dedicated jsonl file defining its context +- No resume needed, no segmentation, behavior controlled by code not prompt + +Trigger: PreToolUse (before Task tool call) + +Context Source: Trellis active task resolver points to task directory +- implement.jsonl - Implement agent dedicated context +- check.jsonl - Check agent dedicated context +- prd.md - Requirements document +- design.md - Technical design for complex tasks +- implement.md - Execution plan for complex tasks +- codex-review-output.txt - Code Review results +""" +from __future__ import annotations + +# IMPORTANT: Suppress all warnings FIRST +import warnings +warnings.filterwarnings("ignore") + +import json +import os +import sys +from pathlib import Path +from typing import Any + +# IMPORTANT: Force stdout to use UTF-8 on Windows +# This fixes UnicodeEncodeError when outputting non-ASCII characters +if sys.platform.startswith("win"): + import io as _io + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] + elif hasattr(sys.stdout, "detach"): + sys.stdout = _io.TextIOWrapper(sys.stdout.detach(), encoding="utf-8", errors="replace") # type: ignore[union-attr] + + +# ============================================================================= +# Path Constants (change here to rename directories) +# ============================================================================= + +DIR_WORKFLOW = ".trellis" +DIR_SPEC = "spec" +FILE_TASK_JSON = "task.json" + +# ============================================================================= +# Subagent Constants (change here to rename subagent types) +# ============================================================================= + +AGENT_IMPLEMENT = "trellis-implement" +AGENT_CHECK = "trellis-check" +AGENT_RESEARCH = "trellis-research" + +# Agents that require a task directory +AGENTS_REQUIRE_TASK = (AGENT_IMPLEMENT, AGENT_CHECK) +# All supported agents +AGENTS_ALL = (AGENT_IMPLEMENT, AGENT_CHECK, AGENT_RESEARCH) + + +def find_repo_root(start_path: str) -> str | None: + """ + Find git repo root from start_path upwards + + Returns: + Repo root path, or None if not found + """ + current = Path(start_path).resolve() + while current != current.parent: + if (current / ".git").exists(): + return str(current) + current = current.parent + return None + + +def _detect_platform(input_data: dict) -> str | None: + if isinstance(input_data.get("cursor_version"), str): + return "cursor" + env_map = { + "CLAUDE_PROJECT_DIR": "claude", + "CURSOR_PROJECT_DIR": "cursor", + "CODEBUDDY_PROJECT_DIR": "codebuddy", + "FACTORY_PROJECT_DIR": "droid", + "GEMINI_PROJECT_DIR": "gemini", + "QODER_PROJECT_DIR": "qoder", + "KIRO_PROJECT_DIR": "kiro", + "COPILOT_PROJECT_DIR": "copilot", + } + for env_name, platform in env_map.items(): + if os.environ.get(env_name): + return platform + script_parts = set(Path(sys.argv[0]).parts) + if ".claude" in script_parts: + return "claude" + if ".cursor" in script_parts: + return "cursor" + if ".gemini" in script_parts: + return "gemini" + if ".qoder" in script_parts: + return "qoder" + if ".codebuddy" in script_parts: + return "codebuddy" + if ".factory" in script_parts: + return "droid" + if ".kiro" in script_parts: + return "kiro" + return None + + +def get_current_task(repo_root: str, input_data: dict) -> str | None: + """Resolve current task directory through the unified active task resolver.""" + scripts_dir = Path(repo_root) / DIR_WORKFLOW / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + try: + from common.active_task import resolve_active_task # type: ignore[import-not-found] + except Exception: + return None + + active = resolve_active_task( + Path(repo_root), + input_data, + platform=_detect_platform(input_data), + ) + return active.task_path + + +def read_file_content(base_path: str, file_path: str) -> str | None: + """Read file content, return None if file doesn't exist""" + full_path = os.path.join(base_path, file_path) + if os.path.exists(full_path) and os.path.isfile(full_path): + try: + with open(full_path, "r", encoding="utf-8") as f: + return f.read() + except Exception: + return None + return None + + +def read_directory_contents( + base_path: str, dir_path: str, max_files: int = 20 +) -> list[tuple[str, str]]: + """ + Read all .md files in a directory + + Args: + base_path: Base path (usually repo_root) + dir_path: Directory relative path + max_files: Max files to read (prevent huge directories) + + Returns: + [(file_path, content), ...] + """ + full_path = os.path.join(base_path, dir_path) + if not os.path.exists(full_path) or not os.path.isdir(full_path): + return [] + + results = [] + try: + # Only read .md files, sorted by filename + md_files = sorted( + [ + f + for f in os.listdir(full_path) + if f.endswith(".md") and os.path.isfile(os.path.join(full_path, f)) + ] + ) + + for filename in md_files[:max_files]: + file_full_path = os.path.join(full_path, filename) + relative_path = os.path.join(dir_path, filename) + try: + with open(file_full_path, "r", encoding="utf-8") as f: + content = f.read() + results.append((relative_path, content)) + except Exception: + continue + except Exception: + pass + + return results + + +def read_jsonl_entries(base_path: str, jsonl_path: str) -> list[tuple[str, str]]: + """ + Read all file/directory contents referenced in jsonl file + + Schema: + {"file": "path/to/file.md", "reason": "..."} + {"file": "path/to/dir/", "type": "directory", "reason": "..."} + {"_example": "..."} # seed row — skipped (no `file` field) + + Rows without a ``file`` field (e.g. the self-describing seed line written + by ``task.py create`` before the agent has curated entries) are skipped + silently. If the resulting entry list is empty, a stderr warning is + emitted so the operator can debug missing context. + + Returns: + [(path, content), ...] + """ + full_path = os.path.join(base_path, jsonl_path) + if not os.path.exists(full_path): + print( + f"[inject-subagent-context] WARN: {jsonl_path} not found — " + f"sub-agent will receive only task artifacts", + file=sys.stderr, + ) + return [] + + results = [] + saw_real_entry = False + try: + with open(full_path, "r", encoding="utf-8") as f: + for line in f: + line = line.strip() + if not line: + continue + try: + item = json.loads(line) + file_path = item.get("file") or item.get("path") + entry_type = item.get("type", "file") + + if not file_path: + # Seed / comment row — skip silently + continue + + saw_real_entry = True + if entry_type == "directory": + # Read all .md files in directory + dir_contents = read_directory_contents(base_path, file_path) + results.extend(dir_contents) + else: + # Read single file + content = read_file_content(base_path, file_path) + if content: + results.append((file_path, content)) + except json.JSONDecodeError: + continue + except Exception: + pass + + if not saw_real_entry: + print( + f"[inject-subagent-context] WARN: {jsonl_path} has no curated " + f"entries (only seed / empty) — sub-agent will receive only " + f"task artifacts. See workflow.md planning artifact guidance.", + file=sys.stderr, + ) + + return results + + + + +def get_agent_context(repo_root: str, task_dir: str, agent_type: str) -> str: + """ + Get context from {agent_type}.jsonl for the specified agent. + Only reads implement.jsonl or check.jsonl (the two JSONL files the task system creates). + """ + context_parts = [] + + agent_jsonl = f"{task_dir}/{agent_type}.jsonl" + for file_path, content in read_jsonl_entries(repo_root, agent_jsonl): + context_parts.append(f"=== {file_path} ===\n{content}") + + return "\n\n".join(context_parts) + + +def get_implement_context(repo_root: str, task_dir: str) -> str: + """ + Complete context for Implement Agent + + Read order: + 1. All files in implement.jsonl (spec/research manifests) + 2. prd.md (requirements) + 3. design.md if present (technical design) + 4. implement.md if present (execution plan) + """ + context_parts = [] + + # 1. Read implement.jsonl + base_context = get_agent_context(repo_root, task_dir, "implement") + if base_context: + context_parts.append(base_context) + + # 2. Requirements document + prd_content = read_file_content(repo_root, f"{task_dir}/prd.md") + if prd_content: + context_parts.append(f"=== {task_dir}/prd.md (Requirements) ===\n{prd_content}") + + # 3. Technical design for complex tasks + design_content = read_file_content(repo_root, f"{task_dir}/design.md") + if design_content: + context_parts.append( + f"=== {task_dir}/design.md (Technical Design) ===\n{design_content}" + ) + + # 4. Execution plan for complex tasks + implement_plan_content = read_file_content(repo_root, f"{task_dir}/implement.md") + if implement_plan_content: + context_parts.append( + f"=== {task_dir}/implement.md (Execution Plan) ===\n{implement_plan_content}" + ) + + return "\n\n".join(context_parts) + + +def get_check_context(repo_root: str, task_dir: str) -> str: + """ + Context for Check Agent: check.jsonl + task artifacts. + """ + context_parts = [] + + for file_path, content in read_jsonl_entries(repo_root, f"{task_dir}/check.jsonl"): + context_parts.append(f"=== {file_path} ===\n{content}") + + prd_content = read_file_content(repo_root, f"{task_dir}/prd.md") + if prd_content: + context_parts.append(f"=== {task_dir}/prd.md (Requirements) ===\n{prd_content}") + + design_content = read_file_content(repo_root, f"{task_dir}/design.md") + if design_content: + context_parts.append( + f"=== {task_dir}/design.md (Technical Design) ===\n{design_content}" + ) + + implement_plan_content = read_file_content(repo_root, f"{task_dir}/implement.md") + if implement_plan_content: + context_parts.append( + f"=== {task_dir}/implement.md (Execution Plan) ===\n{implement_plan_content}" + ) + + return "\n\n".join(context_parts) + + +def get_finish_context(repo_root: str, task_dir: str) -> str: + """ + Context for Finish phase: reuses check.jsonl + prd.md + (Finish is a final check, same context source.) + """ + return get_check_context(repo_root, task_dir) + + + +def build_implement_prompt(original_prompt: str, context: str) -> str: + """Build complete prompt for Implement""" + return f"""<!-- trellis-hook-injected --> +# Implement Agent Task + +You are the Implement Agent in the Multi-Agent Pipeline. + +## Your Context + +All the information you need has been prepared for you: + +{context} + +--- + +## Your Task + +{original_prompt} + +--- + +## Workflow + +1. **Understand specs** - All dev specs are injected above, understand them + 2. **Understand task artifacts** - Read requirements, technical design if present, and execution plan if present + 3. **Implement feature** - Implement following specs and task artifacts +4. **Self-check** - Ensure code quality against check specs + +## Important Constraints + +- Do NOT execute git commit, only code modifications +- Follow all dev specs injected above +- Report list of modified/created files when done""" + + +def build_check_prompt(original_prompt: str, context: str) -> str: + """Build complete prompt for Check""" + return f"""<!-- trellis-hook-injected --> +# Check Agent Task + +You are the Check Agent in the Multi-Agent Pipeline (code and cross-layer checker). + +## Your Context + +All check specs and dev specs you need: + +{context} + +--- + +## Your Task + +{original_prompt} + +--- + +## Workflow + +1. **Get changes** - Run `git diff --name-only` and `git diff` to get code changes +2. **Check against specs** - Check item by item against specs above +3. **Self-fix** - Fix issues directly, don't just report +4. **Run verification** - Run project's lint and typecheck commands + +## Important Constraints + +- Fix issues yourself, don't just report +- Must execute complete checklist in check specs +- Pay special attention to impact radius analysis (L1-L5)""" + + +def build_finish_prompt(original_prompt: str, context: str) -> str: + """Build complete prompt for Finish (final check before PR)""" + return f"""<!-- trellis-hook-injected --> +# Finish Agent Task + +You are performing the final check before creating a PR. + +## Your Context + +Finish checklist and requirements: + +{context} + +--- + +## Your Task + +{original_prompt} + +--- + +## Workflow + +1. **Review changes** - Run `git diff --name-only` to see all changed files + 2. **Verify task artifacts** - Check requirements in prd.md and, when present, design.md / implement.md +3. **Spec sync** - Analyze whether changes introduce new patterns, contracts, or conventions + - If new pattern/convention found: read target spec file → update it → update index.md if needed + - If infra/cross-layer change: follow the 7-section mandatory template from update-spec.md + - If pure code fix with no new patterns: skip this step +4. **Run final checks** - Execute lint and typecheck +5. **Confirm ready** - Ensure code is ready for PR + +## Important Constraints + +- You MAY update spec files when gaps are detected (use update-spec.md as guide) +- MUST read the target spec file BEFORE editing (avoid duplicating existing content) +- Do NOT update specs for trivial changes (typos, formatting, obvious fixes) +- If critical CODE issues found, report them clearly (fix specs, not code) +- Verify all acceptance criteria in prd.md are met +- Verify design.md and implement.md constraints when those files are present""" + + + +def get_research_context(repo_root: str, task_dir: str | None) -> str: + """ + Context for Research Agent — project structure overview for spec directories. + + `task_dir` kept for signature parity with get_implement_context / get_check_context + so the dispatcher can call them uniformly. + """ + _ = task_dir + context_parts = [] + + # 1. Project structure overview (dynamically discover spec directories) + spec_path = f"{DIR_WORKFLOW}/{DIR_SPEC}" + spec_root = Path(repo_root) / DIR_WORKFLOW / DIR_SPEC + + # Build spec tree dynamically + tree_lines = [f"{spec_path}/"] + if spec_root.is_dir(): + pkg_dirs = sorted(d for d in spec_root.iterdir() if d.is_dir()) + for i, pkg_dir in enumerate(pkg_dirs): + is_last = i == len(pkg_dirs) - 1 + prefix = "└── " if is_last else "├── " + layers = sorted(d.name for d in pkg_dir.iterdir() if d.is_dir()) + layer_info = f" ({', '.join(layers)})" if layers else "" + tree_lines.append(f"{prefix}{pkg_dir.name}/{layer_info}") + + spec_tree = "\n".join(tree_lines) + + project_structure = f"""## Project Spec Directory Structure + +``` +{spec_tree} +``` + +To get structured package info, run: `python ./{DIR_WORKFLOW}/scripts/get_context.py --mode packages` + +## Search Tips + +- Spec files: `{spec_path}/**/*.md` +- Code search: Use Glob and Grep tools +- Tech solutions: Use mcp__exa__web_search_exa or mcp__exa__get_code_context_exa""" + + context_parts.append(project_structure) + + return "\n\n".join(context_parts) + + +def build_research_prompt(original_prompt: str, context: str) -> str: + """Build complete prompt for Research""" + return f"""# Research Agent Task + +You are the Research Agent in the Multi-Agent Pipeline (search researcher). + +## Core Principle + +**You do one thing: find and explain information.** + +You are a documenter, not a reviewer. + +## Project Info + +{context} + +--- + +## Your Task + +{original_prompt} + +--- + +## Workflow + +1. **Understand query** - Determine search type (internal/external) and scope +2. **Plan search** - List search steps for complex queries +3. **Execute search** - Execute multiple independent searches in parallel +4. **Organize results** - Output structured report + +## Search Tools + +| Tool | Purpose | +|------|---------| +| Glob | Search by filename pattern | +| Grep | Search by content | +| Read | Read file content | +| mcp__exa__web_search_exa | External web search | +| mcp__exa__get_code_context_exa | External code/doc search | + +## Strict Boundaries + +**Only allowed**: Describe what exists, where it is, how it works + +**Forbidden** (unless explicitly asked): +- Suggest improvements +- Criticize implementation +- Recommend refactoring +- Modify any files + +## Report Format + +Provide structured search results including: +- List of files found (with paths) +- Code pattern analysis (if applicable) +- Related spec documents +- External references (if any)""" + + +def _string_value(value: Any) -> str: + if isinstance(value, str): + stripped = value.strip() + return stripped + return "" + + +def _extract_subagent_name(value: Any) -> str: + """Extract a sub-agent name from common platform encodings. + + Cursor's native Task args encode custom sub-agents as a protobuf oneof, + which can appear in hook JSON as either ``{"custom": {"name": "..."}}`` + or ``{"type": {"case": "custom", "value": {"name": "..."}}}``. + """ + direct = _string_value(value) + if direct: + return direct + + if not isinstance(value, dict): + return "" + + for key in ("name", "subagent_type_name", "subagentTypeName"): + direct = _string_value(value.get(key)) + if direct: + return direct + + custom = value.get("custom") + if isinstance(custom, dict): + custom_name = _string_value(custom.get("name")) + if custom_name: + return custom_name + + oneof = value.get("type") + if isinstance(oneof, dict): + case_name = _string_value(oneof.get("case")) + if case_name == "custom": + nested_value = oneof.get("value") + if isinstance(nested_value, dict): + custom_name = _string_value(nested_value.get("name")) + if custom_name: + return custom_name + if case_name: + return case_name + + case_name = _string_value(value.get("case")) + if case_name == "custom": + nested_value = value.get("value") + if isinstance(nested_value, dict): + custom_name = _string_value(nested_value.get("name")) + if custom_name: + return custom_name + if case_name: + return case_name + + for agent_name in AGENTS_ALL: + if agent_name in value: + return agent_name + + return "" + + +def _extract_subagent_type(tool_input: dict) -> str: + for key in ( + "subagent_type", + "subagentType", + "subagent_type_name", + "subagentTypeName", + "agent_type", + "agentType", + "name", + ): + agent_name = _extract_subagent_name(tool_input.get(key)) + if agent_name: + return agent_name + return "" + + +def _parse_hook_input(input_data: dict) -> tuple[str, str, dict]: + """Parse hook input across different platform formats. + + Returns (subagent_type, original_prompt, tool_input). + Handles: + - Claude Code / Qoder / CodeBuddy / Droid: tool_name=Task|Agent, tool_input.subagent_type + - Cursor: tool_name=Task|Subagent, tool_input.subagent_type + - Copilot CLI: toolName=task (camelCase key, lowercase value) + - Gemini CLI: tool_name IS the agent name (BeforeTool matcher already filtered) + - Kiro: agentSpawn hook, agent_name field at top level + """ + tool_input = input_data.get("tool_input", {}) + + # Standard format: Task/Agent tool with subagent_type + tool_name = input_data.get("tool_name", "") or input_data.get("toolName", "") + if tool_name.lower() in ("task", "agent", "subagent"): + return ( + _extract_subagent_type(tool_input), + tool_input.get("prompt", ""), + tool_input, + ) + + # Kiro: agentSpawn hook passes agent_name at top level + agent_name = input_data.get("agent_name", "") + if agent_name: + return agent_name, tool_input.get("prompt", input_data.get("prompt", "")), tool_input + + # Gemini CLI: BeforeTool where tool_name IS the agent name + # (matcher already ensured it's one of our agents) + if tool_name in AGENTS_ALL: + return tool_name, tool_input.get("prompt", ""), tool_input + + # Copilot CLI: toolName field (camelCase), value might be the agent name + tool_name_camel = input_data.get("toolName", "") + if tool_name_camel in AGENTS_ALL: + return tool_name_camel, input_data.get("toolArgs", ""), tool_input + + return "", "", tool_input + + +def main(): + if os.environ.get("TRELLIS_HOOKS") == "0" or os.environ.get("TRELLIS_DISABLE_HOOKS") == "1": + sys.exit(0) + + try: + input_data = json.load(sys.stdin) + except json.JSONDecodeError: + sys.exit(0) + + subagent_type, original_prompt, tool_input = _parse_hook_input(input_data) + cwd = input_data.get("cwd", os.getcwd()) + + # Only handle subagent types we care about + if subagent_type not in AGENTS_ALL: + sys.exit(0) + + # Find repo root + repo_root = find_repo_root(cwd) + if not repo_root: + sys.exit(0) + + # Get current task directory (research doesn't require it) + task_dir = get_current_task(repo_root, input_data) + + # implement/check need task directory + if subagent_type in AGENTS_REQUIRE_TASK: + if not task_dir: + sys.exit(0) + # Check if task directory exists + task_dir_full = os.path.join(repo_root, task_dir) + if not os.path.exists(task_dir_full): + sys.exit(0) + + # Check for [finish] marker in prompt (check agent with finish context) + is_finish_phase = "[finish]" in original_prompt.lower() + + # Get context and build prompt based on subagent type + if subagent_type == AGENT_IMPLEMENT: + assert task_dir is not None # validated above + context = get_implement_context(repo_root, task_dir) + new_prompt = build_implement_prompt(original_prompt, context) + elif subagent_type == AGENT_CHECK: + assert task_dir is not None # validated above + if is_finish_phase: + # Finish phase: use finish context (lighter, focused on final verification) + context = get_finish_context(repo_root, task_dir) + new_prompt = build_finish_prompt(original_prompt, context) + else: + # Regular check phase: use check context (full specs for self-fix loop) + context = get_check_context(repo_root, task_dir) + new_prompt = build_check_prompt(original_prompt, context) + elif subagent_type == AGENT_RESEARCH: + # Research can work without task directory + context = get_research_context(repo_root, task_dir) + new_prompt = build_research_prompt(original_prompt, context) + else: + sys.exit(0) + + if not context: + sys.exit(0) + + # Return updated input — use a multi-format output that covers all platforms. + # Most platforms ignore unrecognized fields, so we include multiple formats. + # The platform picks whichever fields it understands. + updated = {**tool_input, "prompt": new_prompt} + output = { + # Claude Code / Qoder / CodeBuddy / Droid format + "hookSpecificOutput": { + "hookEventName": "PreToolUse", + "permissionDecision": "allow", + "updatedInput": updated, + }, + # Cursor format + "permission": "allow", + "updated_input": updated, + # Gemini format + "updatedInput": updated, + } + + print(json.dumps(output, ensure_ascii=False)) + sys.exit(0) + + +if __name__ == "__main__": + main() diff --git a/.claude/hooks/inject-workflow-state.py b/.claude/hooks/inject-workflow-state.py new file mode 100644 index 0000000..fda556b --- /dev/null +++ b/.claude/hooks/inject-workflow-state.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Trellis per-turn breadcrumb hook (UserPromptSubmit / BeforeAgent equivalent). + +Runs on every user prompt. Resolves the active task through Trellis' +session-aware active task resolver and emits a short <workflow-state> +block reminding the main AI what task is active and its expected flow. + +The emitted ``hookEventName`` field is platform-aware: most hosts expect +``UserPromptSubmit`` (Claude Code naming, also accepted by Cursor / Qoder / +CodeBuddy / Droid / Codex / Copilot wiring), but Gemini CLI 0.40.x renamed +its per-turn event to ``BeforeAgent`` and its schema validator rejects the +legacy name. ``_detect_platform`` picks the right value at runtime. +Breadcrumb text is pulled exclusively from workflow.md +[workflow-state:STATUS] tag blocks — workflow.md is the single source of +truth. There are no fallback dicts in this script: when workflow.md is +missing or a tag is absent, the breadcrumb degrades to a generic +"Refer to workflow.md for current step." line so users see (and fix) +the broken state instead of the hook silently masking it. + +Shared across all hook-capable platforms (Claude, Cursor, Codex, Qoder, +CodeBuddy, Droid, Gemini, Copilot). Kiro is not wired (no per-turn +hook entry point). Written to each platform's hooks directory via +writeSharedHooks() at init time. + +Silent exit 0 cases (no output): + - No .trellis/ directory found (not a Trellis project) + - task.json malformed or missing status +""" +from __future__ import annotations + +import json +import os +import re +import sys +from pathlib import Path + +# Force UTF-8 on stdin/stdout/stderr on Windows. Default codepage there is +# cp936 / cp1252 / etc. — non-ASCII content (Chinese task names, prd snippets) +# both in stdin (hook payload from host CLI) and stdout (our emitted blocks) +# raises UnicodeDecodeError / UnicodeEncodeError. Equivalent to `python -X utf8` +# but applied per-stream so we don't depend on host CLI's command wiring. +if sys.platform.startswith("win"): + import io as _io + for _stream_name in ("stdin", "stdout", "stderr"): + _stream = getattr(sys, _stream_name, None) + if _stream is None: + continue + if hasattr(_stream, "reconfigure"): + try: + _stream.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] + except Exception: + pass + elif hasattr(_stream, "detach"): + try: + setattr(sys, _stream_name, _io.TextIOWrapper(_stream.detach(), encoding="utf-8", errors="replace")) + except Exception: + pass +from typing import Optional + + +# Bootstrap notice for Codex while the session has no active task. Codex does not +# get the full SessionStart overview; this short reminder points the main session +# at the start skill once and leaves the per-turn state block compact. +CODEX_NO_TASK_BOOTSTRAP_NOTICE = """<trellis-bootstrap> +If you have not already loaded Trellis context this session, read the `trellis-start` skill once. +</trellis-bootstrap>""" + + +# --------------------------------------------------------------------------- +# CWD-robust Trellis root discovery (fixes hook-path-robustness for this hook) +# --------------------------------------------------------------------------- + +def find_trellis_root(start: Path) -> Optional[Path]: + """Walk up from start to find directory containing .trellis/. + + Handles CWD drift: subdirectory launches, monorepo packages, etc. + Returns None if no .trellis/ found (silent no-op). + """ + cur = start.resolve() + while cur != cur.parent: + if (cur / ".trellis").is_dir(): + return cur + cur = cur.parent + return None + + +# --------------------------------------------------------------------------- +# Active task discovery +# --------------------------------------------------------------------------- + +def _detect_platform(input_data: dict) -> str | None: + if isinstance(input_data.get("cursor_version"), str): + return "cursor" + env_map = { + "CLAUDE_PROJECT_DIR": "claude", + "CURSOR_PROJECT_DIR": "cursor", + "CODEBUDDY_PROJECT_DIR": "codebuddy", + "FACTORY_PROJECT_DIR": "droid", + "GEMINI_PROJECT_DIR": "gemini", + "QODER_PROJECT_DIR": "qoder", + "KIRO_PROJECT_DIR": "kiro", + "COPILOT_PROJECT_DIR": "copilot", + } + for env_name, platform in env_map.items(): + if os.environ.get(env_name): + return platform + script_parts = set(Path(sys.argv[0]).parts) + if ".claude" in script_parts: + return "claude" + if ".cursor" in script_parts: + return "cursor" + if ".codex" in script_parts: + return "codex" + if ".gemini" in script_parts: + return "gemini" + if ".qoder" in script_parts: + return "qoder" + if ".codebuddy" in script_parts: + return "codebuddy" + if ".factory" in script_parts: + return "droid" + if ".kiro" in script_parts: + return "kiro" + return None + + +def _resolve_active_task(root: Path, input_data: dict): + scripts_dir = root / ".trellis" / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + from common.active_task import resolve_active_task # type: ignore[import-not-found] + + return resolve_active_task(root, input_data, platform=_detect_platform(input_data)) + + +def get_active_task(root: Path, input_data: dict) -> Optional[tuple[str, str, str]]: + """Return (task_id, status, source) from the current active task.""" + active = _resolve_active_task(root, input_data) + if not active.task_path: + return None + + task_dir = Path(active.task_path) + if not task_dir.is_absolute(): + task_dir = root / task_dir + if active.stale: + return task_dir.name, f"stale_{active.source_type}", active.source + + task_json = task_dir / "task.json" + if not task_json.is_file(): + return None + try: + data = json.loads(task_json.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + return None + + task_id = data.get("id") or task_dir.name + status = data.get("status", "") + if not isinstance(status, str) or not status: + return None + return task_id, status, active.source + + +# --------------------------------------------------------------------------- +# Breadcrumb loading: parse workflow.md, fall back to hardcoded defaults +# --------------------------------------------------------------------------- + +# Supports STATUS values with letters, digits, underscores, hyphens +# (so "in-review" / "blocked-by-team" work alongside "in_progress"). +_TAG_RE = re.compile( + r"\[workflow-state:([A-Za-z0-9_-]+)\]\s*\n(.*?)\n\s*\[/workflow-state:\1\]", + re.DOTALL, +) + +def load_breadcrumbs(root: Path) -> dict[str, str]: + """Parse workflow.md for [workflow-state:STATUS] blocks. + + Returns {status: body_text}. workflow.md is the single source of + truth — there are no fallback dicts in this script. Missing tags + (or a missing/unreadable workflow.md) fall back to a generic line + in build_breadcrumb so users see the broken state and fix + workflow.md, rather than the hook silently masking the issue. + """ + workflow = root / ".trellis" / "workflow.md" + if not workflow.is_file(): + return {} + try: + content = workflow.read_text(encoding="utf-8") + except OSError: + return {} + + result: dict[str, str] = {} + for match in _TAG_RE.finditer(content): + status = match.group(1) + body = match.group(2).strip() + if body: + result[status] = body + return result + + +def _read_trellis_config(root: Path) -> dict: + """Load .trellis/config.yaml via the bundled trellis_config helper. + + The helper lives in .trellis/scripts/common; the hook lives outside the + scripts tree, so we extend sys.path before importing. + """ + scripts_dir = root / ".trellis" / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + try: + from common.trellis_config import read_trellis_config # type: ignore[import-not-found] + except Exception: + return {} + try: + return read_trellis_config(root) + except Exception: + return {} + + +def _codex_mode_banner(config: dict) -> str: + """Emit a `<codex-mode>` banner for the additionalContext payload. + + Reads `codex.dispatch_mode` from .trellis/config.yaml; defaults to + `inline` when missing or invalid because Codex sub-agents run with + `fork_turns="none"` isolation and can't inherit the parent session's + task context. The banner makes the active mode explicit to Codex AI + per turn, complementing the workflow-state body which is per-status. + Mode tells AI which dispatch protocol to follow; workflow-state tells + AI what step it's at. + """ + mode = "inline" + if isinstance(config, dict): + codex_cfg = config.get("codex") + if isinstance(codex_cfg, dict): + cfg_mode = codex_cfg.get("dispatch_mode") + if cfg_mode in ("inline", "sub-agent"): + mode = cfg_mode + if mode == "sub-agent": + meaning = ( + "sub-agent: implement/check work defaults to Trellis sub-agents; " + "the main session still coordinates, clarifies, updates specs, commits, and finishes." + ) + else: + meaning = ( + "inline: the main session implements/checks directly; " + "do not dispatch implement/check sub-agents." + ) + return f"<codex-mode>{meaning}</codex-mode>" + + +def resolve_breadcrumb_key( + status: str, platform: str | None, config: dict +) -> str: + """Pick the breadcrumb tag key based on Codex dispatch_mode. + + Codex defaults to ``inline`` because sub-agents run with ``fork_turns="none"`` + isolation and can't inherit the parent session's task context. Users can + opt into ``codex.dispatch_mode: sub-agent`` in ``.trellis/config.yaml`` + to use the parallel ``<status>-inline`` tag → ``<status>`` flip. Invalid + or missing values fall back to inline. + + Non-codex platforms return the plain status unchanged. + """ + if platform == "codex": + mode = "inline" + if isinstance(config, dict): + codex_cfg = config.get("codex") + if isinstance(codex_cfg, dict): + cfg_mode = codex_cfg.get("dispatch_mode") + if cfg_mode in ("inline", "sub-agent"): + mode = cfg_mode + return f"{status}-inline" if mode == "inline" else status + return status + + +def build_breadcrumb( + task_id: Optional[str], + status: str, + templates: dict[str, str], + source: str | None = None, + breadcrumb_key: str | None = None, +) -> str: + """Build the <workflow-state>...</workflow-state> block. + + - Known status (tag present in workflow.md) → detailed template body + - Unknown status (no tag, or workflow.md missing) → generic + "Refer to workflow.md for current step." line + - `no_task` pseudo-status (task_id is None) → header omits task info + """ + lookup_key = breadcrumb_key or status + body = templates.get(lookup_key) + if body is None and lookup_key != status: + body = templates.get(status) + if body is None: + body = "Refer to workflow.md for current step." + header = f"Status: {status}" if task_id is None else f"Task: {task_id} ({status})" + return f"<workflow-state>\n{header}\n{body}\n</workflow-state>" + + +# --------------------------------------------------------------------------- +# Entry +# --------------------------------------------------------------------------- + +def main() -> int: + if os.environ.get("TRELLIS_HOOKS") == "0" or os.environ.get("TRELLIS_DISABLE_HOOKS") == "1": + return 0 + + try: + data = json.load(sys.stdin) + except (json.JSONDecodeError, ValueError): + data = {} + + cwd_str = data.get("cwd") or os.getcwd() + cwd = Path(cwd_str) + + root = find_trellis_root(cwd) + if root is None: + return 0 # not a Trellis project + + templates = load_breadcrumbs(root) + platform = _detect_platform(data) + config = _read_trellis_config(root) + task = get_active_task(root, data) + if task is None: + # No active task — still emit a breadcrumb nudging AI toward + # trellis-brainstorm + task.py create when user describes real work. + no_task_key = resolve_breadcrumb_key("no_task", platform, config) + breadcrumb = build_breadcrumb( + None, "no_task", templates, breadcrumb_key=no_task_key + ) + else: + task_id, status, source = task + status_key = resolve_breadcrumb_key(status, platform, config) + source_for_breadcrumb = None if platform == "codex" else source + breadcrumb = build_breadcrumb( + task_id, status, templates, source_for_breadcrumb, breadcrumb_key=status_key + ) + if platform == "codex": + parts: list[str] = [] + if task is None: + parts.append(CODEX_NO_TASK_BOOTSTRAP_NOTICE) + parts.append(_codex_mode_banner(config)) + parts.append(breadcrumb) + breadcrumb = "\n\n".join(parts) + + # Gemini CLI 0.40.x rejects "UserPromptSubmit" — its per-turn event is + # named "BeforeAgent". Other platforms (Claude/Cursor/Qoder/CodeBuddy/ + # Droid/Codex/Copilot) accept the original Claude-style name. + hook_event_name = ( + "BeforeAgent" if platform == "gemini" else "UserPromptSubmit" + ) + + output = { + "hookSpecificOutput": { + "hookEventName": hook_event_name, + "additionalContext": breadcrumb, + } + } + print(json.dumps(output)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/hooks/session-start.py b/.claude/hooks/session-start.py new file mode 100644 index 0000000..238076e --- /dev/null +++ b/.claude/hooks/session-start.py @@ -0,0 +1,831 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Session Start Hook - Inject structured context +""" +from __future__ import annotations + +# IMPORTANT: Suppress all warnings FIRST +import warnings +warnings.filterwarnings("ignore") + +import json +import os +import re +import shlex +import subprocess +import sys +from io import StringIO +from pathlib import Path + + +def _normalize_windows_shell_path(path_str: str) -> str: + """Normalize Unix-style shell paths to real Windows paths. + + On Windows, shells like Git Bash / MSYS2 / Cygwin may report paths like + `/d/Users/...` or `/cygdrive/d/Users/...`. `Path.resolve()` will misinterpret + these as `D:/d/Users...` on drive D: (or similar), breaking repo root + detection. + + This function is intentionally conservative: it only rewrites patterns that + unambiguously represent a drive letter mount. + """ + if not isinstance(path_str, str) or not path_str: + return path_str + + # Only relevant on Windows; keep other platforms untouched. + if not sys.platform.startswith("win"): + return path_str + + p = path_str.strip() + + # Already a Windows drive path (C:\... or C:/...) + if re.match(r"^[A-Za-z]:[\/]", p): + return p + + # MSYS/Git-Bash style: /c/Users/... or /d/Work/... + m = re.match(r"^/([A-Za-z])/(.*)", p) + if m: + drive, rest = m.group(1).upper(), m.group(2) + rest = rest.replace('/', '\\') + return f"{drive}:\\{rest}" + + # Cygwin style: /cygdrive/c/Users/... + m = re.match(r"^/cygdrive/([A-Za-z])/(.*)", p) + if m: + drive, rest = m.group(1).upper(), m.group(2) + rest = rest.replace('/', '\\') + return f"{drive}:\\{rest}" + + # WSL mounted drive (sometimes leaked into env): /mnt/c/Users/... + m = re.match(r"^/mnt/([A-Za-z])/(.*)", p) + if m: + drive, rest = m.group(1).upper(), m.group(2) + rest = rest.replace('/', '\\') + return f"{drive}:\\{rest}" + + return path_str + + +FIRST_REPLY_NOTICE = """<first-reply-notice> +First visible reply: say once in Chinese that Trellis SessionStart context is loaded, then answer directly. +This notice is one-shot: do not repeat it after the first assistant reply in the same session. +</first-reply-notice>""" + +# Force UTF-8 on stdin/stdout/stderr on Windows. Default codepage there is +# cp936 / cp1252 / etc. — non-ASCII content (Chinese task names, prd snippets) +# both in stdin (hook payload from host CLI) and stdout (our emitted blocks) +# raises UnicodeDecodeError / UnicodeEncodeError. Equivalent to `python -X utf8` +# but applied per-stream so we don't depend on host CLI's command wiring. +if sys.platform.startswith("win"): + import io as _io + for _stream_name in ("stdin", "stdout", "stderr"): + _stream = getattr(sys, _stream_name, None) + if _stream is None: + continue + if hasattr(_stream, "reconfigure"): + try: + _stream.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] + except Exception: + pass + elif hasattr(_stream, "detach"): + try: + setattr(sys, _stream_name, _io.TextIOWrapper(_stream.detach(), encoding="utf-8", errors="replace")) + except Exception: + pass + + + +def _has_curated_jsonl_entry(jsonl_path: Path) -> bool: + """Return True iff jsonl has at least one row with a ``file`` field. + + A freshly seeded jsonl only contains a ``{"_example": ...}`` row (no + ``file`` key) — that is NOT "ready". Readiness requires at least one + curated entry. Matches the contract used by hook-inject and pull-based + sub-agent context loaders. + """ + try: + for line in jsonl_path.read_text(encoding="utf-8").splitlines(): + line = line.strip() + if not line: + continue + try: + row = json.loads(line) + except json.JSONDecodeError: + continue + if isinstance(row, dict) and row.get("file"): + return True + except (OSError, UnicodeDecodeError): + return False + return False + + +def should_skip_injection() -> bool: + """Check if any platform's non-interactive flag is set, or if Trellis + hooks are explicitly disabled via TRELLIS_HOOKS=0 / TRELLIS_DISABLE_HOOKS=1. + """ + if os.environ.get("TRELLIS_HOOKS") == "0": + return True + if os.environ.get("TRELLIS_DISABLE_HOOKS") == "1": + return True + non_interactive_vars = [ + "CLAUDE_NON_INTERACTIVE", + "QODER_NON_INTERACTIVE", + "CODEBUDDY_NON_INTERACTIVE", + "FACTORY_NON_INTERACTIVE", + "CURSOR_NON_INTERACTIVE", + "GEMINI_NON_INTERACTIVE", + "KIRO_NON_INTERACTIVE", + "COPILOT_NON_INTERACTIVE", + ] + return any(os.environ.get(var) == "1" for var in non_interactive_vars) + + +def read_file(path: Path, fallback: str = "") -> str: + try: + return path.read_text(encoding="utf-8") + except (FileNotFoundError, PermissionError): + return fallback + + +def _repo_relative(repo_root: Path, path: Path) -> str: + try: + return path.relative_to(repo_root).as_posix() + except ValueError: + return str(path) + + +def _run_git(repo_root: Path, args: list[str]) -> str: + try: + result = subprocess.run( + ["git", *args], + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=3, + cwd=str(repo_root), + ) + except (subprocess.TimeoutExpired, FileNotFoundError, PermissionError): + return "" + if result.returncode != 0: + return "" + return result.stdout.strip() + + +def _format_git_state(repo_root: Path) -> str: + branch = _run_git(repo_root, ["branch", "--show-current"]) or "(detached)" + dirty_lines = [ + line for line in _run_git(repo_root, ["status", "--porcelain"]).splitlines() + if line.strip() + ] + dirty_text = "clean" if not dirty_lines else f"dirty {len(dirty_lines)} paths" + return f"Git: branch {branch}; {dirty_text}." + + +def _detect_platform(input_data: dict) -> str | None: + if isinstance(input_data.get("cursor_version"), str): + return "cursor" + env_map = { + "CLAUDE_PROJECT_DIR": "claude", + "CURSOR_PROJECT_DIR": "cursor", + "CODEBUDDY_PROJECT_DIR": "codebuddy", + "FACTORY_PROJECT_DIR": "droid", + "GEMINI_PROJECT_DIR": "gemini", + "QODER_PROJECT_DIR": "qoder", + "KIRO_PROJECT_DIR": "kiro", + "COPILOT_PROJECT_DIR": "copilot", + } + for env_name, platform in env_map.items(): + if os.environ.get(env_name): + return platform + script_parts = set(Path(sys.argv[0]).parts) + if ".claude" in script_parts: + return "claude" + if ".cursor" in script_parts: + return "cursor" + if ".codex" in script_parts: + return "codex" + if ".gemini" in script_parts: + return "gemini" + if ".qoder" in script_parts: + return "qoder" + if ".codebuddy" in script_parts: + return "codebuddy" + if ".factory" in script_parts: + return "droid" + if ".kiro" in script_parts: + return "kiro" + return None + + +def _resolve_context_key(trellis_dir: Path, input_data: dict) -> str | None: + scripts_dir = trellis_dir / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + from common.active_task import resolve_context_key # type: ignore[import-not-found] + + return resolve_context_key(input_data, platform=_detect_platform(input_data)) + + +def _persist_context_key_for_bash(context_key: str | None) -> None: + """Expose Trellis session identity to later Claude Code Bash commands. + + Claude Code SessionStart hooks can append exports to CLAUDE_ENV_FILE; those + variables are then available to Bash tools in the same conversation. Without + this bridge, `task.py start` has hook stdin during SessionStart but no + session identity when the AI later runs it as a normal shell command. + """ + if not context_key: + return + env_file = os.environ.get("CLAUDE_ENV_FILE") + if not env_file: + return + try: + with open(env_file, "a", encoding="utf-8") as handle: + handle.write(f"export TRELLIS_CONTEXT_ID={shlex.quote(context_key)}\n") + except OSError: + pass + + +def _resolve_active_task(trellis_dir: Path, input_data: dict): + scripts_dir = trellis_dir / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + from common.active_task import resolve_active_task # type: ignore[import-not-found] + + return resolve_active_task( + trellis_dir.parent, + input_data, + platform=_detect_platform(input_data), + ) + + +def run_script(script_path: Path, context_key: str | None = None) -> str: + try: + if script_path.suffix == ".py": + # Add PYTHONIOENCODING to force UTF-8 in subprocess + env = os.environ.copy() + env["PYTHONIOENCODING"] = "utf-8" + if context_key: + env["TRELLIS_CONTEXT_ID"] = context_key + cmd = [sys.executable, "-W", "ignore", str(script_path)] + else: + env = os.environ.copy() + if context_key: + env["TRELLIS_CONTEXT_ID"] = context_key + cmd = [str(script_path)] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=5, + cwd=script_path.parent.parent.parent, + env=env, + ) + return result.stdout if result.returncode == 0 else "No context available" + except (subprocess.TimeoutExpired, FileNotFoundError, PermissionError): + return "No context available" + + +def _normalize_task_ref(task_ref: str) -> str: + normalized = task_ref.strip() + if not normalized: + return "" + + path_obj = Path(normalized) + if path_obj.is_absolute(): + return str(path_obj) + + normalized = normalized.replace("\\", "/") + while normalized.startswith("./"): + normalized = normalized[2:] + + if normalized.startswith("tasks/"): + return f".trellis/{normalized}" + + return normalized + + +def _resolve_task_dir(trellis_dir: Path, task_ref: str) -> Path: + normalized = _normalize_task_ref(task_ref) + path_obj = Path(normalized) + if path_obj.is_absolute(): + return path_obj + if normalized.startswith(".trellis/"): + return trellis_dir.parent / path_obj + return trellis_dir / "tasks" / path_obj + + +def _get_task_status(trellis_dir: Path, input_data: dict) -> str: + """Return compact active-task status, artifact presence, and next action.""" + active = _resolve_active_task(trellis_dir, input_data) + + if not active.task_path: + return ( + "Status: NO ACTIVE TASK\n" + "Next-Action: Classify the current turn before creating any Trellis task. " + "Simple conversation / small task asks only whether this turn should create a Trellis task. " + "Complex task asks whether task creation and planning are allowed." + ) + + task_ref = active.task_path + task_dir = _resolve_task_dir(trellis_dir, task_ref) + if active.stale or not task_dir.is_dir(): + return ( + f"Status: STALE POINTER\nTask: {task_ref}\n" + f"Next-Action: Run `python ./.trellis/scripts/task.py finish` to clear the stale pointer, " + "then ask the user what to work on next." + ) + + task_json_path = task_dir / "task.json" + task_data = {} + if task_json_path.is_file(): + try: + task_data = json.loads(task_json_path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, PermissionError): + pass + + task_title = task_data.get("title", task_ref) + task_status = task_data.get("status", "unknown") + artifact_names = ("prd.md", "design.md", "implement.md", "implement.jsonl", "check.jsonl") + present = [name for name in artifact_names if (task_dir / name).is_file()] + if (task_dir / "research").is_dir(): + present.append("research/") + present_line = ", ".join(present) if present else "(none)" + + if task_status == "completed": + return ( + f"Status: COMPLETED\nTask: {task_title}\n" + f"Present: {present_line}\n" + "Next-Action: Run `/trellis:finish-work`. If the working tree is dirty, return to Phase 3.4 first." + ) + + has_prd = (task_dir / "prd.md").is_file() + has_design = (task_dir / "design.md").is_file() + has_implement_plan = (task_dir / "implement.md").is_file() + implement_jsonl = task_dir / "implement.jsonl" + check_jsonl = task_dir / "check.jsonl" + jsonl_ready = ( + (not implement_jsonl.is_file() or _has_curated_jsonl_entry(implement_jsonl)) + and (not check_jsonl.is_file() or _has_curated_jsonl_entry(check_jsonl)) + ) + + if task_status == "planning" and not has_prd: + return ( + f"Status: PLANNING\nTask: {task_title}\n" + f"Present: {present_line}\n" + "Next-Action: Load `trellis-brainstorm` and write `prd.md`. Stay in planning." + ) + + if task_status == "planning": + missing_complex = [ + name for name, exists in ( + ("design.md", has_design), + ("implement.md", has_implement_plan), + ) + if not exists + ] + next_bits: list[str] = [] + if missing_complex: + next_bits.append( + "Lightweight task can request start review with PRD-only; " + f"complex task must add {', '.join(missing_complex)} before start" + ) + else: + next_bits.append("Planning artifacts are present; ask for review before `task.py start`") + if not jsonl_ready: + next_bits.append("curate `implement.jsonl` and `check.jsonl` before sub-agent mode start") + return ( + f"Status: PLANNING\nTask: {task_title}\n" + f"Present: {present_line}\n" + f"Next-Action: {'; '.join(next_bits)}. Do not enter implementation until the user confirms start." + ) + + return ( + f"Status: {str(task_status).upper()}\nTask: {task_title}\n" + f"Present: {present_line}\n" + "Next-Action: Follow the matching per-turn workflow-state. " + "Implementation/check context order is jsonl entries -> `prd.md` -> `design.md if present` -> `implement.md if present`." + ) + + +def _load_trellis_config(trellis_dir: Path, input_data: dict) -> tuple: + """Load Trellis config for session-start decisions. + + Returns: + (is_mono, packages_dict, spec_scope, task_pkg, default_pkg) + """ + scripts_dir = trellis_dir / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + + try: + from common.config import get_default_package, get_packages, get_spec_scope, is_monorepo # type: ignore[import-not-found] + from common.paths import get_current_task # type: ignore[import-not-found] + + repo_root = trellis_dir.parent + is_mono = is_monorepo(repo_root) + packages = get_packages(repo_root) or {} + scope = get_spec_scope(repo_root) + + # Get active task's package + task_pkg = None + current = get_current_task( + repo_root, + input_data, + platform=_detect_platform(input_data), + ) + if current: + task_json = repo_root / current / "task.json" + if task_json.is_file(): + try: + data = json.loads(task_json.read_text(encoding="utf-8")) + if isinstance(data, dict): + tp = data.get("package") + if isinstance(tp, str) and tp: + task_pkg = tp + except (json.JSONDecodeError, OSError): + pass + + default_pkg = get_default_package(repo_root) + return is_mono, packages, scope, task_pkg, default_pkg + except Exception: + return False, {}, None, None, None + + +def _check_legacy_spec(trellis_dir: Path, is_mono: bool, packages: dict) -> str | None: + """Check for legacy spec directory structure in monorepo. + + Returns warning message if legacy structure detected, None otherwise. + """ + if not is_mono or not packages: + return None + + spec_dir = trellis_dir / "spec" + if not spec_dir.is_dir(): + return None + + # Check for legacy flat spec dirs (spec/backend/, spec/frontend/ with index.md) + has_legacy = False + for legacy_name in ("backend", "frontend"): + legacy_dir = spec_dir / legacy_name + if legacy_dir.is_dir() and (legacy_dir / "index.md").is_file(): + has_legacy = True + break + + if not has_legacy: + return None + + # Check which packages are missing spec/<pkg>/ directory + missing = [ + name for name in sorted(packages.keys()) + if not (spec_dir / name).is_dir() + ] + + if not missing: + return None # All packages have spec dirs + + if len(missing) == len(packages): + return ( + f"[!] Legacy spec structure detected: found `spec/backend/` or `spec/frontend/` " + f"but no package-scoped `spec/<package>/` directories.\n" + f"Monorepo packages: {', '.join(sorted(packages.keys()))}\n" + f"Please reorganize: `spec/backend/` -> `spec/<package>/backend/`" + ) + return ( + f"[!] Partial spec migration detected: packages {', '.join(missing)} " + f"still missing `spec/<pkg>/` directory.\n" + f"Please complete migration for all packages." + ) + + +def _resolve_spec_scope( + is_mono: bool, + packages: dict, + scope, + task_pkg: str | None, + default_pkg: str | None, +) -> set | None: + """Resolve which packages should have their specs injected. + + Returns: + Set of package names to include, or None for full scan. + """ + if not is_mono or not packages: + return None # Single-repo: full scan + + if scope is None: + return None # No scope configured: full scan + + if isinstance(scope, str) and scope == "active_task": + if task_pkg and task_pkg in packages: + return {task_pkg} + if default_pkg and default_pkg in packages: + return {default_pkg} + return None # Fallback to full scan + + if isinstance(scope, list): + valid = set() + for entry in scope: + if entry in packages: + valid.add(entry) + else: + print( + f"Warning: spec_scope contains unknown package: {entry}, ignoring", + file=sys.stderr, + ) + + if valid: + # Warn if active task is out of scope + if task_pkg and task_pkg not in valid: + print( + f"Warning: active task package '{task_pkg}' is out of configured spec_scope", + file=sys.stderr, + ) + return valid + + # All entries invalid: fallback chain + print( + "Warning: all spec_scope entries invalid, falling back to task/default/full", + file=sys.stderr, + ) + if task_pkg and task_pkg in packages: + return {task_pkg} + if default_pkg and default_pkg in packages: + return {default_pkg} + return None # Full scan + + return None # Unknown scope type: full scan + + +def _collect_spec_index_paths(trellis_dir: Path, allowed_pkgs: set | None) -> list[str]: + paths: list[str] = [] + guides_index = trellis_dir / "spec" / "guides" / "index.md" + if guides_index.is_file(): + paths.append(".trellis/spec/guides/index.md") + + spec_dir = trellis_dir / "spec" + if not spec_dir.is_dir(): + return paths + + for sub in sorted(spec_dir.iterdir()): + if not sub.is_dir() or sub.name.startswith(".") or sub.name == "guides": + continue + + index_file = sub / "index.md" + if index_file.is_file(): + paths.append(f".trellis/spec/{sub.name}/index.md") + continue + + if allowed_pkgs is not None and sub.name not in allowed_pkgs: + continue + for nested in sorted(sub.iterdir()): + if not nested.is_dir(): + continue + nested_index = nested / "index.md" + if nested_index.is_file(): + paths.append(f".trellis/spec/{sub.name}/{nested.name}/index.md") + + return paths + + +def _build_compact_current_state( + trellis_dir: Path, + input_data: dict, + spec_index_paths: list[str], +) -> str: + repo_root = trellis_dir.parent + lines: list[str] = [] + + try: + from common.paths import get_active_journal_file, get_developer, get_tasks_dir, count_lines # type: ignore[import-not-found] + from common.tasks import iter_active_tasks # type: ignore[import-not-found] + except Exception: + get_active_journal_file = None # type: ignore[assignment] + get_developer = None # type: ignore[assignment] + get_tasks_dir = None # type: ignore[assignment] + count_lines = None # type: ignore[assignment] + iter_active_tasks = None # type: ignore[assignment] + + developer = get_developer(repo_root) if get_developer else None + lines.append(f"Developer: {developer or '(not initialized)'}") + lines.append(_format_git_state(repo_root)) + + active = _resolve_active_task(trellis_dir, input_data) + if active.task_path: + task_dir = _resolve_task_dir(trellis_dir, active.task_path) + status = "unknown" + task_json = task_dir / "task.json" + if task_json.is_file(): + try: + data = json.loads(task_json.read_text(encoding="utf-8")) + if isinstance(data, dict): + status = str(data.get("status") or "unknown") + except (json.JSONDecodeError, OSError): + pass + lines.append(f"Current task: {_repo_relative(repo_root, task_dir)}; status={status}.") + else: + lines.append("Current task: none.") + + if get_tasks_dir and iter_active_tasks: + try: + task_count = sum(1 for _ in iter_active_tasks(get_tasks_dir(repo_root))) + lines.append( + f"Active tasks: {task_count} total. Use `python ./.trellis/scripts/task.py list --mine` only if needed." + ) + except Exception: + pass + + if get_active_journal_file and count_lines: + journal = get_active_journal_file(repo_root) + if journal: + lines.append( + f"Journal: {_repo_relative(repo_root, journal)}, {count_lines(journal)} / 2000 lines." + ) + + if spec_index_paths: + lines.append(f"Spec indexes: {len(spec_index_paths)} available.") + + return "\n".join(lines) + + +def _extract_range(content: str, start_header: str, end_header: str) -> str: + """Extract lines starting at `## start_header` up to (but excluding) `## end_header`. + + Both parameters are full header lines WITHOUT the `## ` prefix (e.g. "Phase Index"). + Returns empty string if start header is not found. + End header missing → extracts to end of file. + """ + lines = content.splitlines() + start: int | None = None + end: int = len(lines) + start_match = f"## {start_header}" + end_match = f"## {end_header}" + for i, line in enumerate(lines): + stripped = line.strip() + if start is None and stripped == start_match: + start = i + continue + if start is not None and stripped == end_match: + end = i + break + if start is None: + return "" + return "\n".join(lines[start:end]).rstrip() + + +_BREADCRUMB_TAG_RE = re.compile( + r"\[workflow-state:([A-Za-z0-9_-]+)\]\s*\n.*?\n\s*\[/workflow-state:\1\]", + re.DOTALL, +) + + +def _strip_breadcrumb_tag_blocks(content: str) -> str: + """Remove `[workflow-state:STATUS]...[/workflow-state:STATUS]` blocks. + + The tag blocks live inside `## Phase Index` (since v0.5.0-rc.0, when + they were colocated with their phase summaries) and are consumed by the + UserPromptSubmit hook (`inject-workflow-state.py`). The session-start + payload already covers the full step bodies, so re-inlining the + breadcrumbs here would just duplicate context. + """ + stripped = _BREADCRUMB_TAG_RE.sub("", content) + stripped = re.sub(r"<!--.*?-->", "", stripped, flags=re.DOTALL) + stripped = re.sub(r"^\[(?!/?workflow-state:)/?[^\]\n]+\]\s*\n?", "", stripped, flags=re.MULTILINE) + return re.sub(r"\n{3,}", "\n\n", stripped).strip() + + +def _build_workflow_overview(workflow_path: Path) -> str: + """Inject only the compact Phase Index summary for SessionStart.""" + content = read_file(workflow_path) + if not content: + return "No workflow.md found" + + out_lines = [ + "# Development Workflow - Session Summary", + "Full guide: .trellis/workflow.md. Step detail: `python ./.trellis/scripts/get_context.py --mode phase --step <X.Y>`.", + "", + ] + + phases = _extract_range(content, "Phase Index", "Phase 1: Plan") + if phases: + out_lines.append(_strip_breadcrumb_tag_blocks(phases).rstrip()) + + return "\n".join(out_lines).rstrip() + + +def main(): + if should_skip_injection(): + sys.exit(0) + + try: + hook_input = json.loads(sys.stdin.read()) + if not isinstance(hook_input, dict): + hook_input = {} + except (json.JSONDecodeError, ValueError): + hook_input = {} + + # Try platform-specific env vars, hook cwd, fallback to cwd + project_dir_env_vars = [ + "CLAUDE_PROJECT_DIR", + "QODER_PROJECT_DIR", + "CODEBUDDY_PROJECT_DIR", + "FACTORY_PROJECT_DIR", + "CURSOR_PROJECT_DIR", + "GEMINI_PROJECT_DIR", + "KIRO_PROJECT_DIR", + "COPILOT_PROJECT_DIR", + ] + project_dir = None + for var in project_dir_env_vars: + val = os.environ.get(var) + if val: + project_dir = Path(_normalize_windows_shell_path(val)).resolve() + break + if project_dir is None: + project_dir = Path(_normalize_windows_shell_path(hook_input.get("cwd", "."))).resolve() + + trellis_dir = project_dir / ".trellis" + context_key = _resolve_context_key(trellis_dir, hook_input) + _persist_context_key_for_bash(context_key) + + # Load config for scope filtering and legacy detection + is_mono, packages, scope_config, task_pkg, default_pkg = _load_trellis_config( + trellis_dir, + hook_input, + ) + allowed_pkgs = _resolve_spec_scope(is_mono, packages, scope_config, task_pkg, default_pkg) + + output = StringIO() + + spec_index_paths = _collect_spec_index_paths(trellis_dir, allowed_pkgs) + + output.write("""<session-context> +Trellis compact SessionStart context. Use it to orient the session; load details on demand. +</session-context> + +""") + output.write(FIRST_REPLY_NOTICE) + output.write("\n\n") + + # Legacy migration warning + legacy_warning = _check_legacy_spec(trellis_dir, is_mono, packages) + if legacy_warning: + output.write(f"<migration-warning>\n{legacy_warning}\n</migration-warning>\n\n") + + output.write("<current-state>\n") + output.write(_build_compact_current_state(trellis_dir, hook_input, spec_index_paths)) + output.write("\n</current-state>\n\n") + + output.write("<trellis-workflow>\n") + output.write(_build_workflow_overview(trellis_dir / "workflow.md")) + output.write("\n</trellis-workflow>\n\n") + + output.write("<guidelines>\n") + output.write( + "Task context order for implementation/check: jsonl entries -> `prd.md` -> " + "`design.md if present` -> `implement.md if present`. Missing optional artifacts " + "are skipped for lightweight tasks.\n\n" + ) + + if spec_index_paths: + output.write("## Available indexes (read on demand)\n") + for p in spec_index_paths: + output.write(f"- {p}\n") + output.write("\n") + + output.write( + "Discover more via: " + "`python ./.trellis/scripts/get_context.py --mode packages`\n" + ) + output.write("</guidelines>\n\n") + + # Check task status and inject structured tag + task_status = _get_task_status(trellis_dir, hook_input) + output.write(f"<task-status>\n{task_status}\n</task-status>\n\n") + + output.write("""<ready> +Context loaded. Follow <task-status>. Load workflow/spec/task details only when needed. +</ready>""") + + context_text = output.getvalue() + result = { + # Claude Code / Qoder / CodeBuddy / Droid / Gemini / Copilot format + "hookSpecificOutput": { + "hookEventName": "SessionStart", + "additionalContext": context_text, + }, + # Cursor sessionStart format (top-level snake_case per Cursor docs) + "additional_context": context_text, + } + + # Output JSON - stdout is already configured for UTF-8 + print(json.dumps(result, ensure_ascii=False), flush=True) + + +if __name__ == "__main__": + main() diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 0000000..5c46b7f --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,73 @@ +{ + "env": { + "CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR": "1" + }, + "hooks": { + "SessionStart": [ + { + "matcher": "startup", + "hooks": [ + { + "type": "command", + "command": "python .claude/hooks/session-start.py", + "timeout": 30 + } + ] + }, + { + "matcher": "clear", + "hooks": [ + { + "type": "command", + "command": "python .claude/hooks/session-start.py", + "timeout": 30 + } + ] + }, + { + "matcher": "compact", + "hooks": [ + { + "type": "command", + "command": "python .claude/hooks/session-start.py", + "timeout": 30 + } + ] + } + ], + "PreToolUse": [ + { + "matcher": "Task", + "hooks": [ + { + "type": "command", + "command": "python .claude/hooks/inject-subagent-context.py", + "timeout": 30 + } + ] + }, + { + "matcher": "Agent", + "hooks": [ + { + "type": "command", + "command": "python .claude/hooks/inject-subagent-context.py", + "timeout": 30 + } + ] + } + ], + "UserPromptSubmit": [ + { + "hooks": [ + { + "type": "command", + "command": "python .claude/hooks/inject-workflow-state.py", + "timeout": 15 + } + ] + } + ] + }, + "enabledPlugins": {} +} diff --git a/.claude/skills/trellis-before-dev/SKILL.md b/.claude/skills/trellis-before-dev/SKILL.md new file mode 100644 index 0000000..096f8bf --- /dev/null +++ b/.claude/skills/trellis-before-dev/SKILL.md @@ -0,0 +1,40 @@ +--- +name: trellis-before-dev +description: "Discovers and injects project-specific coding guidelines from .trellis/spec/ before implementation begins. Reads spec indexes, pre-development checklists, and shared thinking guides for the target package. Use when starting a new coding task, before writing any code, switching to a different package, or needing to refresh project conventions and standards." +--- + +Read the relevant development guidelines before starting your task. + +Execute these steps: + +1. **Read current task artifacts**: + - `prd.md` for requirements and acceptance criteria + - `design.md` if present for technical design + - `implement.md` if present for execution order and validation plan + +2. **Discover packages and their spec layers**: + ```bash + python ./.trellis/scripts/get_context.py --mode packages + ``` + +3. **Identify which specs apply** to your task based on: + - Which package you're modifying (e.g., `cli/`, `docs-site/`) + - What type of work (backend, frontend, unit-test, docs, etc.) + - Any spec/research paths referenced by the task artifacts + +4. **Read the spec index** for each relevant module: + ```bash + cat .trellis/spec/<package>/<layer>/index.md + ``` + Follow the **"Pre-Development Checklist"** section in the index. + +5. **Read the specific guideline files** listed in the Pre-Development Checklist that are relevant to your task. The index is NOT the goal — it points you to the actual guideline files (e.g., `error-handling.md`, `conventions.md`, `mock-strategies.md`). Read those files to understand the coding standards and patterns. + +6. **Always read shared guides**: + ```bash + cat .trellis/spec/guides/index.md + ``` + +7. Understand the coding standards and patterns you need to follow, then proceed with your development plan. + +This step is **mandatory** before writing any code. diff --git a/.claude/skills/trellis-brainstorm/SKILL.md b/.claude/skills/trellis-brainstorm/SKILL.md new file mode 100644 index 0000000..bd7aeb4 --- /dev/null +++ b/.claude/skills/trellis-brainstorm/SKILL.md @@ -0,0 +1,112 @@ +--- +name: trellis-brainstorm +description: "Guides collaborative requirements discovery before implementation. Creates task directory, seeds PRD, asks high-value questions one at a time, researches technical choices, and converges on MVP scope. Use when requirements are unclear, there are multiple valid approaches, or the user describes a new feature or complex task." +--- + +# Trellis Brainstorm + +## Non-Negotiable Interview Contract + +Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer. + +Ask the questions one at a time. + +## Non-Negotiable Evidence Rule + +If a question can be answered by exploring the codebase, explore the codebase instead. + +This is mandatory. Before asking the user a question, first check whether the answer is already available in code, tests, configs, docs, existing specs, or task history. + +Do not ask the user to confirm facts that the repository can answer. Ask only for product intent, preference, scope, risk tolerance, or decisions that remain ambiguous after inspection. + +--- + +Use this skill during Phase 1 planning to turn the user's request into clear requirements and planning artifacts. + +## Preconditions + +Use this skill only after task-creation consent has been given and the user is ready to enter Trellis planning. + +If no task exists yet, create one: + +```bash +TASK_DIR=$(python ./.trellis/scripts/task.py create "<short task title>" --slug <slug>) +``` + +Use a concise title from the user's request. Use a slug without a date prefix. `task.py create` adds the `MM-DD-` directory prefix automatically. + +`task.py create` creates the default `prd.md`. Update that file with the current understanding before asking follow-up questions. + +## Planning Flow + +1. Capture the user's request and initial known facts in `prd.md`. +2. Inspect available evidence before asking questions: + - code, tests, fixtures, and configs + - README files, docs, existing specs, and domain notes + - related Trellis tasks, research files, and session history when present +3. Separate what you found into: + - confirmed facts + - product intent still needed from the user + - scope or risk decisions still needed from the user + - likely out-of-scope items +4. Ask the single highest-value remaining question. +5. Include your recommended answer with the question. +6. After each user answer, update `prd.md` before continuing. +7. For complex tasks, create or update `design.md` and `implement.md` before implementation starts. + +Do not invent a project-specific product/spec hierarchy. If the repository already has product, domain, or spec docs, use them. If it does not, proceed with the evidence that exists. + +## Question Rules + +Ask only one question per message. + +Each question must include: + +- the decision needed +- why the answer matters +- your recommended answer +- the trade-off if the user chooses differently + +Do not ask process questions such as whether to search, inspect files, or continue brainstorming. Do the evidence work directly. Ask the user only when the remaining issue is a product decision, preference, scope boundary, or risk tolerance choice. + +## Artifact Rules + +`prd.md` records requirements and acceptance: + +- goal and user value +- confirmed facts +- requirements +- acceptance criteria +- out of scope +- open questions that still block planning + +`design.md` records technical design for complex tasks: + +- architecture and boundaries +- data flow and contracts +- compatibility and migration notes +- important trade-offs +- operational or rollback considerations + +`implement.md` records execution planning for complex tasks: + +- ordered implementation checklist +- validation commands +- risky files or rollback points +- follow-up checks before `task.py start` + +Lightweight tasks may have only `prd.md`. Complex tasks must have `prd.md`, `design.md`, and `implement.md` before `task.py start`. + +`implement.md` is not a replacement for `implement.jsonl`. Use JSONL files only for manifest-style spec and research references when the task needs them. + +## Quality Bar + +Before declaring planning ready: + +- `prd.md` contains testable acceptance criteria. +- Repository-answerable questions have already been answered through inspection. +- Remaining open questions are genuinely about user intent or scope. +- Complex tasks have `design.md` and `implement.md`. +- The user has reviewed the final planning artifacts or explicitly approved proceeding. + +Do not start implementation until the user approves or asks for implementation. diff --git a/.claude/skills/trellis-break-loop/SKILL.md b/.claude/skills/trellis-break-loop/SKILL.md new file mode 100644 index 0000000..ef2b50c --- /dev/null +++ b/.claude/skills/trellis-break-loop/SKILL.md @@ -0,0 +1,130 @@ +--- +name: trellis-break-loop +description: "Deep bug analysis to break the fix-forget-repeat cycle. Analyzes root cause category, why fixes failed, prevention mechanisms, and captures knowledge into specs. Use after fixing a bug to prevent the same class of bugs." +--- + +# Break the Loop - Deep Bug Analysis + +When debug is complete, use this for deep analysis to break the "fix bug -> forget -> repeat" cycle. + +--- + +## Analysis Framework + +Analyze the bug you just fixed from these 5 dimensions: + +### 1. Root Cause Category + +Which category does this bug belong to? + +| Category | Characteristics | Example | +|----------|-----------------|---------| +| **A. Missing Spec** | No documentation on how to do it | New feature without checklist | +| **B. Cross-Layer Contract** | Interface between layers unclear | API returns different format than expected | +| **C. Change Propagation Failure** | Changed one place, missed others | Changed function signature, missed call sites | +| **D. Test Coverage Gap** | Unit test passes, integration fails | Works alone, breaks when combined | +| **E. Implicit Assumption** | Code relies on undocumented assumption | Timestamp seconds vs milliseconds | + +### 2. Why Fixes Failed (if applicable) + +If you tried multiple fixes before succeeding, analyze each failure: + +- **Surface Fix**: Fixed symptom, not root cause +- **Incomplete Scope**: Found root cause, didn't cover all cases +- **Tool Limitation**: grep missed it, type check wasn't strict +- **Mental Model**: Kept looking in same layer, didn't think cross-layer + +### 3. Prevention Mechanisms + +What mechanisms would prevent this from happening again? + +| Type | Description | Example | +|------|-------------|---------| +| **Documentation** | Write it down so people know | Update thinking guide | +| **Architecture** | Make the error impossible structurally | Type-safe wrappers | +| **Compile-time** | Strict type checking, no escape hatches | Signature change causes compile error | +| **Runtime** | Monitoring, alerts, scans | Detect orphan entities | +| **Test Coverage** | E2E tests, integration tests | Verify full flow | +| **Code Review** | Checklist, PR template | "Did you check X?" | + +### 4. Systematic Expansion + +What broader problems does this bug reveal? + +- **Similar Issues**: Where else might this problem exist? +- **Design Flaw**: Is there a fundamental architecture issue? +- **Process Flaw**: Is there a development process improvement? +- **Knowledge Gap**: Is the team missing some understanding? + +### 5. Knowledge Capture + +Solidify insights into the system: + +- [ ] Update `.trellis/spec/guides/` thinking guides +- [ ] Update relevant `.trellis/spec/` docs +- [ ] Create issue record (if applicable) +- [ ] Create feature ticket for root fix +- [ ] Update check guidelines if needed + +--- + +## Output Format + +Please output analysis in this format: + +```markdown +## Bug Analysis: [Short Description] + +### 1. Root Cause Category +- **Category**: [A/B/C/D/E] - [Category Name] +- **Specific Cause**: [Detailed description] + +### 2. Why Fixes Failed (if applicable) +1. [First attempt]: [Why it failed] +2. [Second attempt]: [Why it failed] +... + +### 3. Prevention Mechanisms +| Priority | Mechanism | Specific Action | Status | +|----------|-----------|-----------------|--------| +| P0 | ... | ... | TODO/DONE | + +### 4. Systematic Expansion +- **Similar Issues**: [List places with similar problems] +- **Design Improvement**: [Architecture-level suggestions] +- **Process Improvement**: [Development process suggestions] + +### 5. Knowledge Capture +- [ ] [Documents to update / tickets to create] +``` + +--- + +## Core Philosophy + +> **The value of debugging is not in fixing the bug, but in making this class of bugs never happen again.** + +Three levels of insight: +1. **Tactical**: How to fix THIS bug +2. **Strategic**: How to prevent THIS CLASS of bugs +3. **Philosophical**: How to expand thinking patterns + +30 minutes of analysis saves 30 hours of future debugging. + +--- + +## After Analysis: Immediate Actions + +**IMPORTANT**: After completing the analysis above, you MUST immediately: + +1. **Update spec/guides** - Don't just list TODOs, actually update the relevant files: + - If it's a cross-platform issue → update `cross-platform-thinking-guide.md` + - If it's a cross-layer issue → update `cross-layer-thinking-guide.md` + - If it's a code reuse issue → update `code-reuse-thinking-guide.md` + - If it's domain-specific → update `backend/*.md` or `frontend/*.md` + +2. **Sync templates** - After updating `.trellis/spec/`, sync to `src/templates/markdown/spec/` + +3. **Commit the spec updates** - This is the primary output, not just the analysis text + +> **The analysis is worthless if it stays in chat. The value is in the updated specs.** diff --git a/.claude/skills/trellis-check/SKILL.md b/.claude/skills/trellis-check/SKILL.md new file mode 100644 index 0000000..856ee0d --- /dev/null +++ b/.claude/skills/trellis-check/SKILL.md @@ -0,0 +1,98 @@ +--- +name: trellis-check +description: "Comprehensive quality verification: spec compliance, lint, type-check, tests, cross-layer data flow, code reuse, and consistency checks. Use when code is written and needs quality verification, before committing changes, or to catch context drift during long sessions." +--- + +# Code Quality Check + +Comprehensive quality verification for recently written code. Combines spec compliance, cross-layer safety, and pre-commit checks. + +--- + +## Step 1: Identify What Changed + +```bash +git diff --name-only HEAD +git status +``` + +## Step 2: Read Task Artifacts and Applicable Specs + +Read the current task artifacts in order: + +- `prd.md` +- `design.md` if present +- `implement.md` if present + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +For each changed package/layer, read the spec index and follow its **Quality Check** section: + +```bash +cat .trellis/spec/<package>/<layer>/index.md +``` + +Read the specific guideline files referenced — the index is a pointer, not the goal. + +## Step 3: Run Project Checks + +Run the project's lint, type-check, and test commands. Fix any failures before proceeding. + +## Step 4: Review Against Checklist + +### Code Quality + +- [ ] Linter passes? +- [ ] Type checker passes (if applicable)? +- [ ] Tests pass? +- [ ] No debug logging left in? +- [ ] No suppressed warnings or type-safety bypasses? + +### Test Coverage + +- [ ] New function → unit test added? +- [ ] Bug fix → regression test added? +- [ ] Changed behavior → existing tests updated? + +### Spec Sync + +- [ ] Does `.trellis/spec/` need updates? (new patterns, conventions, lessons learned) + +> "If I fixed a bug or discovered something non-obvious, should I document it so future me won't hit the same issue?" → If YES, update the relevant spec doc. + +## Step 5: Cross-Layer Dimensions (if applicable) + +Skip this step if your change is confined to a single layer. + +### A. Data Flow (changes touch 3+ layers) + +- [ ] Read flow traces correctly: Storage → Service → API → UI +- [ ] Write flow traces correctly: UI → API → Service → Storage +- [ ] Types/schemas correctly passed between layers? +- [ ] Errors properly propagated to caller? + +### B. Code Reuse (modifying constants, creating utilities) + +- [ ] Searched for existing similar code before creating new? + ```bash + grep -r "pattern" src/ + ``` +- [ ] If 2+ places define same value → extracted to shared constant? +- [ ] After batch modification, all occurrences updated? + +### C. Import/Dependency (creating new files) + +- [ ] Correct import paths (relative vs absolute)? +- [ ] No circular dependencies? + +### D. Same-Layer Consistency + +- [ ] Other places using the same concept are consistent? + +--- + +## Step 6: Report and Fix + +Report violations found and fix them directly. Re-run project checks after fixes. diff --git a/.claude/skills/trellis-meta/SKILL.md b/.claude/skills/trellis-meta/SKILL.md new file mode 100644 index 0000000..590bfac --- /dev/null +++ b/.claude/skills/trellis-meta/SKILL.md @@ -0,0 +1,73 @@ +--- +name: trellis-meta +description: "Understand and customize the local Trellis architecture inside a user project. Use when modifying .trellis plus platform hooks, settings, agents, skills, commands, prompts, or workflows generated by trellis init." +--- + +# Trellis Meta + +This skill is for local Trellis users who have already run `trellis init` in a project. After reading it, an AI should understand the Trellis architecture, operating model, and customization entry points inside that user project, then modify the generated `.trellis/` and platform directory files according to the user's request. + +The default operating scope is local files in the user project: + +- `.trellis/`: workflow, config, tasks, spec, workspace, scripts, and runtime state. +- Platform directories: `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, `.kiro/`, `.gemini/`, `.qoder/`, `.codebuddy/`, `.github/`, `.factory/`, `.pi/`, `.kilocode/`, `.agent/`, `.windsurf/`, and similar directories. +- Shared skill layer: `.agents/skills/`. + +Do not assume the user has the Trellis source repository. Do not default to modifying the global npm install directory or `node_modules`. + +## How To Use + +1. Read `references/local-architecture/overview.md` first to establish the local Trellis system model. +2. If the request involves a specific AI tool, read `references/platform-files/platform-map.md` and the relevant platform file notes. +3. If the user wants to change behavior, read `references/customize-local/overview.md`, then open the specific customization topic. +4. Before editing, read the actual files in the user project and treat local content as authoritative. + +## References + +### Local Architecture + +- `references/local-architecture/overview.md`: The three-layer local Trellis architecture and customization principles. +- `references/local-architecture/generated-files.md`: Files generated by `trellis init` and their customization boundaries. +- `references/local-architecture/workflow.md`: Phases, routing, and workflow-state blocks in `.trellis/workflow.md`. +- `references/local-architecture/task-system.md`: Task directories, active tasks, JSONL context, and task runtime. +- `references/local-architecture/spec-system.md`: How `.trellis/spec/` is organized and injected. +- `references/local-architecture/workspace-memory.md`: `.trellis/workspace/`, journals, and cross-session memory. +- `references/local-architecture/context-injection.md`: Hooks, sub-agent preludes, and context injection paths. + +### Platform Files + +- `references/platform-files/overview.md`: How shared `.trellis/` files relate to platform directories. +- `references/platform-files/platform-map.md`: Platform directories and paths for skills, agents, hooks, and extensions. +- `references/platform-files/hooks-and-settings.md`: How settings/config files, hooks, plugins, and extensions connect to Trellis. +- `references/platform-files/agents.md`: Local file responsibilities for `trellis-research`, `trellis-implement`, and `trellis-check`. +- `references/platform-files/skills-and-commands.md`: Differences between skills, commands, prompts, and workflows, plus how to change them. + +### Local Customization + +- `references/customize-local/overview.md`: Choose the right local customization entry point for the user's request. +- `references/customize-local/change-workflow.md`: Change phases, routing, next actions, and workflow-state. +- `references/customize-local/change-task-lifecycle.md`: Change task creation, status, archive behavior, and hooks. +- `references/customize-local/change-context-loading.md`: Change how tasks, specs, journals, and hook context are loaded. +- `references/customize-local/change-hooks.md`: Change platform hooks, settings, and shell session bridges. +- `references/customize-local/change-agents.md`: Change research, implement, and check agent behavior. +- `references/customize-local/change-skills-or-commands.md`: Add or modify local skills, commands, prompts, and workflows. +- `references/customize-local/change-spec-structure.md`: Adjust the project spec structure under `.trellis/spec/`. +- `references/customize-local/add-project-local-conventions.md`: Put team rules into project-local specs or local skills. + +## Current Rules + +- `.trellis/workflow.md` is the local workflow source of truth. +- `.trellis/config.yaml` is the project-level Trellis configuration and task hook configuration entry point. +- `.trellis/spec/` stores the user's project-specific coding conventions and design constraints. +- `.trellis/tasks/` stores task PRDs, technical notes, research files, and JSONL context. +- `.trellis/workspace/` stores developer journals and cross-session memory. +- Platform settings/config files decide which hooks, agents, skills, commands, prompts, and workflows actually run. +- `.trellis/.template-hashes.json` and `.trellis/.runtime/` are management/runtime state files. Confirm necessity before editing them. + +## Do Not + +- Do not treat Trellis upstream source code as the default target for local customization. +- Do not modify the global npm install directory or `node_modules/@mindfoldhq/trellis` to implement project needs. +- Do not overwrite user-modified local files with default templates. +- Do not put team-private project rules into the public `trellis-meta`; put project rules in `.trellis/spec/` or a project-local skill. +- Do not describe removed historical mechanisms as current Trellis behavior. diff --git a/.claude/skills/trellis-meta/references/customize-local/add-project-local-conventions.md b/.claude/skills/trellis-meta/references/customize-local/add-project-local-conventions.md new file mode 100644 index 0000000..d32ca2d --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/add-project-local-conventions.md @@ -0,0 +1,83 @@ +# Add Project-Local Conventions + +Often the user does not need to change Trellis mechanics; they need local AI to understand their team's conventions. In that case, prefer `.trellis/spec/` or a project-local skill instead of editing `trellis-meta`. + +## Where To Put Things + +| Content type | Location | +| --- | --- | +| Rules code must follow | `.trellis/spec/<layer>/` | +| Cross-layer thinking methods | `.trellis/spec/guides/` | +| AI capability for a project-specific flow | Platform-local skill | +| One-off task material | `.trellis/tasks/<task>/` | +| Session summary | `.trellis/workspace/<developer>/journal-N.md` | + +## Create A Project-Local Skill + +If the user wants AI to know "how this project customizes Trellis," create a local skill: + +```text +.claude/skills/trellis-local/ +└── SKILL.md +``` + +Example: + +```md +--- +name: trellis-local +description: "Project-local Trellis customizations for this repository. Use when changing this project's Trellis workflow, hooks, local agents, or team-specific conventions." +--- + +# Trellis Local + +## Local Scope + +This skill documents this repository's Trellis customizations only. + +## Custom Workflow Rules + +- ... + +## Local Hook Changes + +- ... + +## Local Agent Changes + +- ... +``` + +For multi-platform projects, place equivalent versions in other platform skill directories, or use `.agents/skills/` for platforms that support the shared layer. + +## Write To `.trellis/spec/` + +If the content is a coding convention, write it to spec. Examples: + +```text +.trellis/spec/backend/error-handling.md +.trellis/spec/frontend/components.md +.trellis/spec/guides/cross-platform-thinking-guide.md +``` + +After writing it, update the corresponding `index.md` so AI can find the new rule from the entry point. + +## Make The Current Task Use New Conventions + +After writing a spec, add it to the current task context: + +```bash +python ./.trellis/scripts/task.py add-context <task> implement ".trellis/spec/backend/error-handling.md" "Error handling conventions" +python ./.trellis/scripts/task.py add-context <task> check ".trellis/spec/backend/error-handling.md" "Review error handling" +``` + +## Do Not Store Project-Private Rules In `trellis-meta` + +`trellis-meta` is a public skill for understanding Trellis architecture and local customization entry points. Put project-private content in: + +- `.trellis/spec/` +- a project-local skill +- the current task +- workspace journal + +This prevents future updates to Trellis's built-in `trellis-meta` from overwriting the team's own conventions. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-agents.md b/.claude/skills/trellis-meta/references/customize-local/change-agents.md new file mode 100644 index 0000000..9b63531 --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-agents.md @@ -0,0 +1,54 @@ +# Change Local Agents + +When the user wants to change `trellis-research`, `trellis-implement`, or `trellis-check` behavior, edit platform agent files in the user project. + +## Read These Files First + +1. Target platform agent directory +2. `.trellis/workflow.md` Phase 2 / research routing +3. Current task `prd.md` +4. Current task `implement.jsonl` / `check.jsonl` +5. Relevant hook or agent prelude + +## Common Paths + +| Platform | Path | +| --- | --- | +| Claude Code | `.claude/agents/trellis-*.md` | +| Cursor | `.cursor/agents/trellis-*.md` | +| OpenCode | `.opencode/agents/trellis-*.md` | +| Codex | `.codex/agents/trellis-*.toml` | +| Kiro | `.kiro/agents/trellis-*.json` | +| Gemini CLI | `.gemini/agents/trellis-*.md` | +| Qoder | `.qoder/agents/trellis-*.md` | +| CodeBuddy | `.codebuddy/agents/trellis-*.md` | +| Factory Droid | `.factory/droids/trellis-*.md` | +| Pi Agent | `.pi/agents/trellis-*.md` | + +Use the actual paths in the user project as authoritative. + +## Common Needs + +| Need | Which agent to edit | +| --- | --- | +| Research must write files, not only reply in chat | `trellis-research` | +| Certain local specs must be read before implementation | `trellis-implement` + `implement.jsonl` configuration rules | +| Specific commands must run during checking | `trellis-check` | +| Agent must not modify certain directories | The corresponding agent's write boundary instructions | +| Agent output format must be fixed | The corresponding agent's final/reporting instructions | + +## Modification Principles + +1. **Preserve role boundaries**: research investigates and persists; implement writes implementation; check reviews and fixes. +2. **Do not hard-code project specs into agents**: long-term specs belong in `.trellis/spec/`; agents are responsible for reading them. +3. **Make read order explicit**: active task -> PRD -> info -> JSONL -> spec/research. +4. **Make write boundaries explicit**: which directories may be written and which may not. +5. **Synchronize across platforms**: when the user configured multiple platforms, decide whether to change only the current platform or all platform agents. + +## Agent Pull Platforms + +If an agent file contains a prelude for "read task/context after startup," do not remove those steps when editing. Otherwise the agent will work only from chat context and bypass Trellis's core mechanism. + +## Hook Push Platforms + +If context is injected by a hook, the agent file should still retain responsibility boundaries. Do not remove PRD/spec requirements from the agent just because a hook injects context. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-context-loading.md b/.claude/skills/trellis-meta/references/customize-local/change-context-loading.md new file mode 100644 index 0000000..83bcd63 --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-context-loading.md @@ -0,0 +1,84 @@ +# Change Local Context Loading + +Context loading determines when AI reads workflow, task, spec, research, workspace, and git status. Read this page when the user says "AI does not know the current task," "the agent did not read specs," or "there is too much/too little context." + +## Read These Files First + +1. `.trellis/workflow.md` +2. `.trellis/scripts/get_context.py` +3. `.trellis/scripts/common/session_context.py` +4. `.trellis/scripts/common/task_context.py` +5. `.trellis/scripts/common/active_task.py` +6. Current platform hooks or agent files +7. The current task's `implement.jsonl` / `check.jsonl` + +## Context Sources + +| Source | Purpose | +| --- | --- | +| `.trellis/workflow.md` | Workflow and next-action hints. | +| `.trellis/tasks/<task>/prd.md` | Current task requirements. | +| `.trellis/tasks/<task>/design.md` | Complex task technical design. | +| `.trellis/tasks/<task>/implement.md` | Complex task execution plan. | +| `.trellis/tasks/<task>/implement.jsonl` | Spec/research to read before implementation. | +| `.trellis/tasks/<task>/check.jsonl` | Spec/research to read during checking. | +| `.trellis/spec/` | Project specs. | +| `.trellis/workspace/` | Session records. | +| git status | Current working tree changes. | + +## Common Needs And Edit Points + +| Need | Edit point | +| --- | --- | +| Inject more/less information in new sessions | `session_context.py` or the platform `session-start` hook. | +| Change hints on each user input | `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The `inject-workflow-state` hook is parser-only and reads the block verbatim. | +| Agent did not read specs | Task JSONL, agent prelude, `inject-subagent-context` hook. | +| Active task is lost | `active_task.py` and platform session identity propagation. | +| Change JSONL validation rules | `task_context.py`. | + +## JSONL Rules + +`implement.jsonl` / `check.jsonl` are the key context loading interface: + +```jsonl +{"file": ".trellis/spec/backend/index.md", "reason": "Backend conventions"} +{"file": ".trellis/tasks/04-28-x/research/api.md", "reason": "API research"} +``` + +Include only spec/research files. Do not put code files that will be modified into these manifests; agents read code files themselves during implementation. + +## Change Session Context + +If the user wants every new session to see more project state, edit: + +- `.trellis/scripts/common/session_context.py` +- the corresponding platform `session-start` hook + +Context cannot grow without bound. Prefer injecting indexes and paths so the AI can read detailed files on demand. + +## Change Sub-Agent Context + +First determine which mode the platform uses: + +- hook push: edit the `inject-subagent-context` hook. +- agent pull: edit the read steps in the corresponding `trellis-implement` / `trellis-check` agent file. + +In both modes, make sure the agent ultimately reads: + +1. active task +2. the corresponding JSONL +3. spec/research referenced by the JSONL +4. `prd.md` +5. `design.md` if present +6. `implement.md` if present + +## Troubleshooting Order + +```bash +python ./.trellis/scripts/task.py current --source +python ./.trellis/scripts/task.py list-context <task> +python ./.trellis/scripts/task.py validate <task> +python ./.trellis/scripts/get_context.py --mode packages +``` + +Confirm the task and JSONL are correct before editing hooks/agents. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-hooks.md b/.claude/skills/trellis-meta/references/customize-local/change-hooks.md new file mode 100644 index 0000000..093a171 --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-hooks.md @@ -0,0 +1,57 @@ +# Change Local Hooks + +Hooks are the automation layer that connects a platform to Trellis. When the user wants to change "when context is injected," "how shell commands inherit a session," or "which files are read before an agent starts," hooks are usually the edit point. + +## Read These Files First + +1. Target platform settings/config, such as `.claude/settings.json`, `.codex/hooks.json`, `.cursor/hooks.json` +2. Target platform hooks directory +3. `.trellis/scripts/common/active_task.py` +4. `.trellis/scripts/common/session_context.py` +5. `.trellis/workflow.md` + +## Common Hook Types + +| Hook | Purpose | +| --- | --- | +| session-start | Injects a Trellis overview when a session starts, clears, or compacts. | +| workflow-state | Injects a state hint on each user input. | +| sub-agent context | Injects PRD/spec/research before an agent starts. | +| shell session bridge | Lets `task.py` commands in shell see the same session identity. | + +## Modification Steps + +1. Find the hook registration in settings/config. +2. Confirm the registered script path exists. +3. Read the hook script and identify inputs, outputs, and called `.trellis/scripts/`. +4. Modify hook behavior. +5. If the hook depends on workflow content, synchronize `.trellis/workflow.md`. + +## Example: Change New-Session Injection Content + +First find the session-start hook: + +```text +.claude/settings.json +.claude/hooks/session-start.py +``` + +If the hook ultimately calls `.trellis/scripts/get_context.py` or `session_context.py`, editing the local script is usually more robust than hard-coding content in the hook. + +## Example: Agent Did Not Read JSONL + +First confirm: + +```bash +python ./.trellis/scripts/task.py current --source +python ./.trellis/scripts/task.py validate <task> +``` + +If the task and JSONL are correct, determine whether the platform uses hook push or agent pull. For hook push, edit `inject-subagent-context`; for agent pull, edit the agent file. + +## Notes + +- Settings handle registration, hook scripts handle behavior; inspect both together. +- Different platforms support different hook events. Do not directly copy another platform's settings. +- Hooks should read project-local `.trellis/`; they should not depend on Trellis upstream source paths. +- Hook failures should produce visible errors so AI does not silently lose context. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-skills-or-commands.md b/.claude/skills/trellis-meta/references/customize-local/change-skills-or-commands.md new file mode 100644 index 0000000..84590a1 --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-skills-or-commands.md @@ -0,0 +1,78 @@ +# Change Local Skills, Commands, Prompts, And Workflows + +When the user wants to change AI entry points, auto-trigger rules, or explicit command behavior, edit skills, commands, prompts, or workflows in local platform directories. + +## Read These Files First + +1. `.trellis/workflow.md` +2. Target platform skill/command/prompt/workflow directory +3. Related agent or hook files +4. Whether project rules already exist in `.trellis/spec/` + +## Which Entry Type To Choose + +| Goal | Recommendation | +| --- | --- | +| AI should automatically know a capability | Add or modify a skill. | +| User wants to trigger manually with a command | Add or modify a command/prompt/workflow. | +| Team project conventions | Prefer `.trellis/spec/` or a project-local skill. | +| Change Trellis flow semantics | Synchronize `.trellis/workflow.md`. | + +## Modify A Skill + +A skill is usually: + +```text +<skill-name>/ +├── SKILL.md +└── references/ +``` + +`SKILL.md` should be short and responsible for triggering/routing. Put long content in `references/` so AI can read it on demand. + +The frontmatter description should specify when to use the skill. Example: + +```yaml +description: "Use when customizing this project's deployment workflow and release checklist." +``` + +Do not write vague descriptions such as "helpful project skill"; they can trigger incorrectly. + +## Modify A Command/Prompt/Workflow + +Explicit entry points should state: + +- How the user triggers it. +- Which `.trellis/` files to read. +- Which scripts to run. +- How to report after completion. + +If a command only repeats workflow rules, prefer making it reference/read `.trellis/workflow.md` instead of maintaining a second copy of the flow. + +## Common Paths + +| Platform | Entry directories | +| --- | --- | +| Claude Code | `.claude/skills/`, `.claude/commands/` | +| Cursor | `.cursor/skills/`, `.cursor/commands/` | +| OpenCode | `.opencode/skills/`, `.opencode/commands/` | +| Codex | `.agents/skills/`, `.codex/skills/` | +| GitHub Copilot | `.github/skills/`, `.github/prompts/` | +| Kilo / Antigravity / Windsurf | workflows + skills | + +## Add A Project-Local Skill + +If the user wants to document team-private customizations, create a project-local skill, for example: + +```text +.claude/skills/project-trellis-local/ +└── SKILL.md +``` + +For multi-platform projects, add equivalent versions in each platform skill directory, or use `.agents/skills/` on platforms that support the shared layer. + +## Notes + +- Do not mix every platform's syntax into one file. +- Do not change only one platform entry point while claiming all platforms are supported. +- Do not hide long-term engineering conventions inside a command; write them to `.trellis/spec/`. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-spec-structure.md b/.claude/skills/trellis-meta/references/customize-local/change-spec-structure.md new file mode 100644 index 0000000..14e0cd8 --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-spec-structure.md @@ -0,0 +1,83 @@ +# Change Local Spec Structure + +When the user wants to change the engineering conventions AI follows, add new spec layers, or adjust monorepo package mapping, edit `.trellis/spec/` and `.trellis/config.yaml`. + +## Read These Files First + +1. `.trellis/config.yaml` +2. `.trellis/spec/` +3. `.trellis/workflow.md` planning artifact guidance and Phase 3.3 +4. Current task `implement.jsonl` / `check.jsonl` + +## Common Needs + +| Need | Edit location | +| --- | --- | +| Add backend/frontend/docs/test spec layer | `.trellis/spec/<layer>/` or `.trellis/spec/<package>/<layer>/` | +| Add shared thinking guides | `.trellis/spec/guides/` | +| Adjust monorepo packages | `packages` in `.trellis/config.yaml` | +| Change default package | `default_package` in `.trellis/config.yaml` | +| Control spec scanning scope | `spec_scope` in `.trellis/config.yaml` | +| Make a task read a new spec | Task `implement.jsonl` / `check.jsonl` | + +## Add A Spec Layer + +Single-repository example: + +```text +.trellis/spec/security/ +├── index.md +└── auth.md +``` + +Monorepo example: + +```text +.trellis/spec/webapp/security/ +├── index.md +└── auth.md +``` + +`index.md` should include: + +- What code this layer applies to. +- Pre-Development Checklist. +- Quality Check. +- Links to specific guideline files. + +## Update Context + +Adding a spec does not mean every task automatically reads it. The current task must reference it in JSONL: + +```bash +python ./.trellis/scripts/task.py add-context <task> implement ".trellis/spec/webapp/security/index.md" "Security conventions" +python ./.trellis/scripts/task.py add-context <task> check ".trellis/spec/webapp/security/index.md" "Security review rules" +``` + +## Change Monorepo Packages + +Example `.trellis/config.yaml`: + +```yaml +packages: + webapp: + path: apps/web + api: + path: apps/api +default_package: webapp +``` + +After editing, run: + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +Use this output to confirm AI can see the correct packages and spec layers. + +## Notes + +- Specs are user project conventions and can be changed according to project needs. +- Do not put temporary task information into specs; put temporary information in the task. +- Do not put long-term conventions only in agents or commands; preserve them in specs. +- After changing spec structure, check whether existing task JSONL files still point to files that exist. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-task-lifecycle.md b/.claude/skills/trellis-meta/references/customize-local/change-task-lifecycle.md new file mode 100644 index 0000000..208e0da --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-task-lifecycle.md @@ -0,0 +1,90 @@ +# Change Local Task Lifecycle + +Task lifecycle includes creation, start, context configuration, finish, archive, parent/child tasks, and lifecycle hooks. The default customization targets are `.trellis/tasks/`, `.trellis/config.yaml`, and `.trellis/scripts/`. + +## Read These Files First + +1. `.trellis/workflow.md` +2. `.trellis/config.yaml` +3. `.trellis/scripts/task.py` +4. `.trellis/scripts/common/task_store.py` +5. `.trellis/scripts/common/task_utils.py` +6. The current task's `.trellis/tasks/<task>/task.json` + +## Common Needs And Edit Points + +| Need | Edit point | +| --- | --- | +| Automatically sync an external system after task creation | `hooks.after_create` in `.trellis/config.yaml`. | +| Automatically update status after task start | `hooks.after_start` in `.trellis/config.yaml`. | +| Run a script after task finish | `hooks.after_finish` in `.trellis/config.yaml`. | +| Clean external resources after archive | `hooks.after_archive` in `.trellis/config.yaml`. | +| Change default task fields | `.trellis/scripts/common/task_store.py`. | +| Change task parsing/search | `.trellis/scripts/common/task_utils.py`. | +| Change active task behavior | `.trellis/scripts/common/active_task.py`. | + +## lifecycle hooks + +`.trellis/config.yaml` supports: + +```yaml +hooks: + after_create: + - "python .trellis/scripts/hooks/my_sync.py create" + after_start: + - "python .trellis/scripts/hooks/my_sync.py start" + after_finish: + - "python .trellis/scripts/hooks/my_sync.py finish" + after_archive: + - "python .trellis/scripts/hooks/my_sync.py archive" +``` + +Hook commands receive the `TASK_JSON_PATH` environment variable, pointing to the current task's `task.json`. Hook failures should usually warn, but not block the main task operation. + +## Change Task Fields + +If the user wants to add project-local fields, prefer putting them under `meta` in `task.json` to avoid breaking existing scripts' assumptions about standard fields. + +Example: + +```json +"meta": { + "linearIssue": "ENG-123", + "risk": "high" +} +``` + +If standard fields really need to change, inspect every local script that reads `task.json`. + +## Change Active Task + +Active task is session-level state stored in `.trellis/.runtime/sessions/`. Do not fall back to a global `.current-task` model. If the user wants to change active task behavior, edit: + +- `.trellis/scripts/common/active_task.py` +- platform hooks or shell session bridges +- active task descriptions in `.trellis/workflow.md` + +### `task.py create` Sets the Active Pointer + +`cmd_create` in `.trellis/scripts/common/task_store.py` calls `set_active_task` best-effort right after writing the new task directory. The behavior: + +- When the calling shell carries session identity (`TRELLIS_CONTEXT_ID` env var, or any platform-specific session env that `resolve_context_key` recognizes — see `active_task.py:_ENV_SESSION_KEYS`), the per-session pointer at `.trellis/.runtime/sessions/<context_key>.json` is rewritten to point at the new task. The task's `status=planning` and `[workflow-state:planning]` fires on the very next `UserPromptSubmit`. +- When session identity is unavailable (raw CLI invocation outside an AI session, or a platform that doesn't propagate identity to shell), the task directory is still created and `status=planning` is still written, but the active pointer is left untouched. The user can attach the task later with `task.py start <dir>` once they're back in an AI session. + +This makes `[workflow-state:planning]` the live breadcrumb during the brainstorm and JSONL curation work that follows `task.py create`. The pre-R7 behavior left the breadcrumb stuck on `no_task` until `task.py start`, so the planning block was effectively dead text. + +If you fork `task.py` to add a new creation path (e.g. an external import that bypasses `cmd_create`), audit whether your path also calls `set_active_task`. Without that call, your created tasks will not surface as active. The full status writer table is in `.trellis/spec/cli/backend/workflow-state-contract.md`. + +## Modification Steps + +1. Confirm the current task with `python ./.trellis/scripts/task.py current --source`. +2. Read the current task's `task.json` and confirm status and fields. +3. For configuration needs, edit `.trellis/config.yaml` first. +4. For script behavior needs, then edit `.trellis/scripts/`. +5. If the AI flow changed, synchronize `.trellis/workflow.md`. + +## Do Not + +- Do not directly edit `.trellis/.runtime/sessions/` to "fix" business state. +- Do not hard-code project-private fields into scripts; prefer `meta`. +- Do not default to asking the user to fork Trellis CLI. diff --git a/.claude/skills/trellis-meta/references/customize-local/change-workflow.md b/.claude/skills/trellis-meta/references/customize-local/change-workflow.md new file mode 100644 index 0000000..aa2e663 --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/change-workflow.md @@ -0,0 +1,65 @@ +# Change Local Workflow + +When the user wants to change Trellis phases, next-action hints, whether to create tasks, whether to use sub-agents, or when to check/wrap up, edit `.trellis/workflow.md` first. + +## Read These Files First + +1. `.trellis/workflow.md` +2. Entry files for the current platform, such as skills/commands/prompts/workflows +3. The current task's `task.json` and `prd.md` + +## Common Needs And Edit Points + +| Need | Edit point | +| --- | --- | +| Change phase names or phase order | `Phase Index` and the corresponding Phase sections. | +| Change whether to create a task when there is no task | `[workflow-state:no_task]` state block. | +| Change the next step during planning | Phase 1 and `[workflow-state:planning]`. | +| Change whether an agent is required during in_progress | Phase 2 and `[workflow-state:in_progress]`. | +| Change wrap-up after completion | Phase 3 and `[workflow-state:completed]`. | +| Change which skill a user intent triggers | `Skill Routing` table. | + +## Modification Steps + +1. Find the relevant section in `.trellis/workflow.md`. +2. When changing rules, keep explicit trigger conditions and next actions. +3. If adding or renaming a skill/agent, synchronize the corresponding files in platform directories. +4. Workflow-state changes only need an edit to the `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The hook is parser-only — it reads whatever you put in the block. Keep the opening and closing tags' STATUS strings identical (`[workflow-state:foo]…[/workflow-state:foo]`); mismatched STATUS pairs are silently dropped. +5. Make the AI reread `.trellis/workflow.md`; do not keep using rules from the old conversation. + +## Example: Relax Task Creation Requirements + +To change when task creation can be skipped, usually edit `[workflow-state:no_task]`: + +```md +[workflow-state:no_task] +Task is not required when the answer is a one-reply explanation, no files are changed, and no research is needed. +[/workflow-state:no_task] +``` + +If the formal Phase 1 flow also needs to change, synchronize the Phase 1 section. + +## Example: One Platform Does Not Use Sub-Agents + +If the user wants only one platform to avoid sub-agents, first confirm whether that platform has a separate group in the workflow. Then change Phase 2 routing for that platform group instead of deleting all `trellis-implement` / `trellis-check` instructions across platforms. + +## `/trellis:continue` Route Table + +`/trellis:continue` resumes a task by deciding which phase step to load next. The decision combines `task.json.status` with the presence of artifacts inside the task directory. The mapping is fixed in the command itself; forks that add custom statuses must extend both the workflow.md tag block and this table. + +| `status` | Artifact state | Resume at | +| --- | --- | --- | +| `planning` | `prd.md` missing | Phase 1.1 (load `trellis-brainstorm`) | +| `planning` | lightweight task with `prd.md` complete | ask for start review, then run `task.py start` | +| `planning` | complex task missing `design.md` or `implement.md` | complete missing planning artifacts | +| `planning` | complex task has `prd.md`, `design.md`, and `implement.md` | ask for start review, then run `task.py start` | +| `in_progress` | no implementation in conversation history | Phase 2.1 (`trellis-implement`) | +| `in_progress` | implementation done, no `trellis-check` run | Phase 2.2 (`trellis-check`) | +| `in_progress` | check passed | Phase 3.1 (verify quality + spec update) | +| `completed` | task is still in active tree | Phase 3.5 (run `/trellis:finish-work` to archive) | + +When you add a custom status (e.g. `in-review`), add a `[workflow-state:in-review]` block in `.trellis/workflow.md` for the per-turn breadcrumb AND extend this route table — usually by editing the `/trellis:continue` command file (`.{platform}/commands/trellis/continue.md` or equivalent) to add a row that decides where to resume from. Without the route entry, `/trellis:continue` will fall through to a default branch and the user will not land on the step you intended. + +## Notes + +`.trellis/workflow.md` is the local project workflow, not an immutable template. The user can adapt it to team habits. After editing it, platform entry files may still contain old descriptions, so inspect them too. diff --git a/.claude/skills/trellis-meta/references/customize-local/overview.md b/.claude/skills/trellis-meta/references/customize-local/overview.md new file mode 100644 index 0000000..ac16a4c --- /dev/null +++ b/.claude/skills/trellis-meta/references/customize-local/overview.md @@ -0,0 +1,55 @@ +# Local Customization Overview + +This directory is for local AI working in a user project where Trellis was installed through npm and `trellis init` has already been run. The AI should modify generated `.trellis/` and platform directories inside the project, not Trellis CLI upstream source code. + +## First Determine What The User Actually Wants To Change + +| User wording | Read first | +| --- | --- | +| "Change the Trellis flow / phases / next prompt" | `change-workflow.md` | +| "Change task creation, status, archive, or hooks" | `change-task-lifecycle.md` | +| "AI did not read context / change injected content" | `change-context-loading.md` | +| "A platform hook is not behaving as expected" | `change-hooks.md` | +| "Change implement/check/research agent behavior" | `change-agents.md` | +| "Add a skill/command/workflow/prompt" | `change-skills-or-commands.md` | +| "Adjust the project spec structure" | `change-spec-structure.md` | +| "Add team conventions and local notes" | `add-project-local-conventions.md` | + +## General Operation Order + +1. **Confirm platform and directories**: inspect which directories exist, such as `.claude/`, `.codex/`, `.cursor/`. +2. **Confirm the current active task**: run `python ./.trellis/scripts/task.py current --source`. +3. **Read the local source of truth**: prefer `.trellis/workflow.md`, `.trellis/config.yaml`, and relevant platform files. +4. **Modify narrowly**: edit only files related to the user's request. +5. **Synchronize semantics**: if a shared flow changes, check whether platform entry points also need changes; if a platform entry changes, check whether `.trellis/workflow.md` still agrees. + +## Local File Priority + +| Layer | Files | +| --- | --- | +| Workflow | `.trellis/workflow.md` | +| Project configuration | `.trellis/config.yaml` | +| Task material | `.trellis/tasks/<task>/` | +| Project specs | `.trellis/spec/` | +| Runtime scripts | `.trellis/scripts/` | +| Platform integration | `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, and similar directories | +| Shared skill | `.agents/skills/` | + +## Things Not To Do By Default + +- Do not edit the global npm install directory. +- Do not edit `node_modules/@mindfoldhq/trellis`. +- Do not assume the user has the Trellis GitHub repository. +- Do not overwrite local files already modified by the user with default templates. +- Do not put team project rules into public `trellis-meta`; project rules belong in `.trellis/spec/` or a local skill. + +## When To Inspect Upstream Source + +Switch to an upstream source-code perspective only when the user explicitly expresses one of these goals: + +- "I want to open a PR to Trellis" +- "I want to change npm package publish contents" +- "I want to fork Trellis" +- "I want to modify the generation logic for `trellis init/update`" + +Otherwise, default to modifying local Trellis files inside the user project. diff --git a/.claude/skills/trellis-meta/references/local-architecture/context-injection.md b/.claude/skills/trellis-meta/references/local-architecture/context-injection.md new file mode 100644 index 0000000..4a7517b --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/context-injection.md @@ -0,0 +1,68 @@ +# Local Context Injection System + +Trellis context injection aims to make AI read the right files at the right time instead of relying on model memory. In a user project, injection is implemented by `.trellis/` scripts together with platform hooks, agents, and skills. + +## Injected Context Types + +| Type | Source | Purpose | +| --- | --- | --- | +| session context | `.trellis/scripts/get_context.py` | Current developer, git status, active task, active tasks, journal, packages. | +| workflow context | `.trellis/workflow.md` | Current Trellis flow and next action. | +| spec context | `.trellis/spec/` + task JSONL | Specs that must be followed during implementation/checking. | +| task context | `.trellis/tasks/<task>/prd.md`, `design.md`, `implement.md`, `research/` | Current task requirements, design, execution plan, and research. | +| platform context | Platform hooks/settings/agents | Lets different AI tools read the files above through their own mechanisms. | + +## session-start + +Platforms with session-start support inject a Trellis overview when a session starts, clears, compacts, or receives a similar event. Injected content usually includes: + +- workflow summary. +- current task status. +- active tasks. +- spec index paths. +- developer identity and git status. + +If the user feels the AI does not know the current task in a new session, first check whether the platform's session-start hook or equivalent mechanism is installed and running. + +## workflow-state + +workflow-state is a lightweight hint injected around each user turn. Based on current task status, it selects a block from `.trellis/workflow.md`, such as `no_task`, `planning`, `in_progress`, or `completed`. + +If the user wants to change "what the AI should do next in a given state," edit the corresponding state block in `.trellis/workflow.md` first. + +## sub-agent context + +Implement and check agents need task context. Trellis has two loading modes: + +1. **hook push**: a platform hook injects jsonl-referenced files plus `prd.md`, `design.md` if present, and `implement.md` if present before the agent starts. +2. **agent pull**: the agent definition instructs the agent to read the active task, jsonl context, and task artifacts after startup. + +In both modes, JSONL files in the task directory are the manifest for spec/research context. Task artifacts are read separately in this order: `prd.md` -> `design.md if present` -> `implement.md if present`. + +## JSONL Reading Rules + +`implement.jsonl` and `check.jsonl` contain one JSON object per line: + +```jsonl +{"file": ".trellis/spec/backend/index.md", "reason": "Backend rules"} +``` + +Readers should skip seed rows without a `file` field. When configuring JSONL, the AI should include only spec/research files, not pre-register code files that will be modified. + +## Active Task And Context Key + +Active task state lives in `.trellis/.runtime/sessions/` and is isolated per session. Hooks try to resolve the context key from platform events, environment variables, transcript paths, or `TRELLIS_CONTEXT_ID`. + +If shell commands cannot see the same context key, `task.py current --source` may report no active task. In that case, check whether the platform passes session identity into the shell instead of hand-writing a global current-task file. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Change session-start injected content | The platform's `session-start` hook or plugin file. | +| Change per-turn workflow-state rules | `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The platform workflow-state hook parses these blocks verbatim and embeds no fallback text. | +| Change how sub-agents read context | Platform agent definitions, the `inject-subagent-context` hook, or agent preludes. | +| Change JSONL validation/display | `.trellis/scripts/common/task_context.py`. | +| Change active task resolution | `.trellis/scripts/common/active_task.py`. | + +When modifying context injection, verify two things: new sessions can see the correct task, and sub-agents can see the correct task artifacts/spec/research. diff --git a/.claude/skills/trellis-meta/references/local-architecture/generated-files.md b/.claude/skills/trellis-meta/references/local-architecture/generated-files.md new file mode 100644 index 0000000..66f832d --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/generated-files.md @@ -0,0 +1,80 @@ +# Local Files Generated After Init + +`trellis init` writes the Trellis runtime into the user project. Later, `trellis update` tries to update Trellis-managed template files, but it uses `.trellis/.template-hashes.json` to determine which files have already been modified by the user. + +This page only describes files that are visible and editable inside the user project. + +## `.trellis/` + +```text +.trellis/ +├── workflow.md +├── config.yaml +├── .developer +├── .version +├── .template-hashes.json +├── .runtime/ +├── scripts/ +├── spec/ +├── tasks/ +└── workspace/ +``` + +| Path | Usually editable? | Notes | +| --- | --- | --- | +| `.trellis/workflow.md` | Yes | Local workflow documentation and AI routing rules. | +| `.trellis/config.yaml` | Yes | Project configuration, hooks, packages, journal line limits, and related settings. | +| `.trellis/spec/` | Yes | Project specs, intended to be updated regularly by users and AI. | +| `.trellis/tasks/` | Yes | Task material and research artifacts, maintained by the task workflow. | +| `.trellis/workspace/` | Yes | Session records, usually written by `add_session.py`. | +| `.trellis/scripts/` | Carefully | Local runtime. It can be customized, but only after understanding the call chain. | +| `.trellis/.runtime/` | No | Runtime state, usually written automatically by hooks/scripts. | +| `.trellis/.developer` | Carefully | Current developer identity. | +| `.trellis/.version` | No | Trellis version record used by update/migration logic. | +| `.trellis/.template-hashes.json` | No | Template hash record. Do not hand-write business rules here. | + +## Platform Directories + +Different platforms generate different directories. Common categories: + +| Category | Example paths | Purpose | +| --- | --- | --- | +| hooks | `.claude/hooks/`, `.codex/hooks/`, `.cursor/hooks/` | Inject session context, workflow-state, and sub-agent context. | +| settings | `.claude/settings.json`, `.codex/hooks.json`, `.qoder/settings.json` | Tell the platform when to run hooks or plugins. | +| agents | `.claude/agents/`, `.codex/agents/`, `.kiro/agents/` | Define agents such as `trellis-research`, `trellis-implement`, and `trellis-check`. | +| skills | `.claude/skills/`, `.agents/skills/`, `.qoder/skills/` | Skills that auto-trigger or can be read by AI. | +| commands/prompts/workflows | `.cursor/commands/`, `.github/prompts/`, `.windsurf/workflows/` | Explicit user-invoked command or workflow entry points. | + +When modifying a platform directory, also confirm whether `.trellis/workflow.md` still describes the same flow. + +## Meaning Of Template Hashes + +`.trellis/.template-hashes.json` records the content hash from the last time Trellis wrote a template file. `trellis update` uses it to distinguish three cases: + +| Case | Update behavior | +| --- | --- | +| File was not modified by the user | It can be updated automatically. | +| File was modified by the user | Prompt the user to overwrite, keep, or generate `.new`. | +| File is no longer a current template | It may be deleted, renamed, or preserved according to migration rules. | + +When an AI customizes local Trellis files, it does not need to maintain hashes manually. It is normal for Trellis update to recognize the result as "modified by the user." + +## Local Customization Boundaries + +Editable by default: + +- `.trellis/workflow.md` +- `.trellis/config.yaml` +- `.trellis/spec/**` +- `.trellis/scripts/**` +- Platform hooks, settings, agents, skills, commands, prompts, and workflows + +Do not edit by default: + +- Global npm install directory +- `node_modules/@mindfoldhq/trellis` +- Trellis GitHub repository source code +- Concrete state files under `.trellis/.runtime/**` +- Hash contents inside `.trellis/.template-hashes.json` + +Switch to the Trellis CLI source-code perspective only when the user explicitly wants to contribute upstream. diff --git a/.claude/skills/trellis-meta/references/local-architecture/overview.md b/.claude/skills/trellis-meta/references/local-architecture/overview.md new file mode 100644 index 0000000..99c7f73 --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/overview.md @@ -0,0 +1,51 @@ +# Local Trellis Architecture Overview + +`trellis-meta` is for user projects that have already run `trellis init`. The user's machine usually has only the npm-installed `trellis` command plus the Trellis files generated inside the project; it may not have the Trellis CLI source code. + +Therefore, when an AI uses this skill, the default customization target is local files inside the user project: + +- `.trellis/`: workflow, tasks, specs, memory, scripts, and runtime state. +- Platform directories: `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, `.kiro/`, `.gemini/`, `.qoder/`, `.codebuddy/`, `.github/`, `.factory/`, `.pi/`, `.kilocode/`, `.agent/`, `.windsurf/`, and similar directories. +- Shared skill layer: `.agents/skills/`. + +Do not default to guiding the user to fork the Trellis CLI repository. Treat upstream source code as the operating target only when the user explicitly says they want to change Trellis upstream source, publish an npm package, or contribute a PR. + +## Local System Model + +Trellis provides three layers inside a user project: + +1. **Workflow layer**: `.trellis/workflow.md` defines phases, routing, next actions, and prompt blocks. +2. **Persistence layer**: `.trellis/tasks/`, `.trellis/spec/`, and `.trellis/workspace/` store tasks, specs, and session memory. +3. **Platform integration layer**: hooks, settings, agents, skills, commands, prompts, and workflows in platform directories connect the Trellis workflow to different AI tools. + +All three layers live inside the user project, so an AI can read and modify them directly. + +## Core Paths + +| Path | Purpose | +| --- | --- | +| `.trellis/workflow.md` | Workflow phases, skill routing, and workflow-state prompt blocks. | +| `.trellis/config.yaml` | Project configuration, task lifecycle hooks, monorepo package configuration, and journal configuration. | +| `.trellis/spec/` | The user's project-specific coding conventions and thinking guides. | +| `.trellis/tasks/` | Each task's PRD, technical notes, research files, and JSONL context. | +| `.trellis/workspace/` | Per-developer journals and cross-session memory. | +| `.trellis/scripts/` | Local Python runtime used by commands, hooks, and context injection. | +| `.trellis/.runtime/` | Session-level runtime state, such as the current task pointer. | +| `.trellis/.template-hashes.json` | Template hashes for Trellis-managed files, used by update to determine whether local files were modified by the user. | + +## AI Customization Principles + +1. **Find the local source of truth first**: Do not edit from memory. Read `.trellis/workflow.md`, `.trellis/config.yaml`, the relevant platform directory, and related task files first. +2. **Edit the user project, not the npm package cache**: Modify generated files inside the project, not `node_modules` or the global npm install directory. +3. **Keep platform files aligned with `.trellis/`**: If workflow routing changes, also check whether platform skills or commands still describe the same flow. +4. **Put project-specific rules in `.trellis/spec/` or a local skill**: Do not put team conventions into `trellis-meta`. +5. **Preserve user changes**: If a file was already modified locally, work from the current content instead of overwriting it with a default template. + +## How To Use This Directory + +- To understand which files exist after init, read `generated-files.md`. +- To change phases, routing, or next actions, read `workflow.md`. +- To change the task model, JSONL context, or active task behavior, read `task-system.md`. +- To change coding convention injection, read `spec-system.md`. +- To understand journals and cross-session memory, read `workspace-memory.md`. +- To change hooks or sub-agent context loading, read `context-injection.md`. diff --git a/.claude/skills/trellis-meta/references/local-architecture/spec-system.md b/.claude/skills/trellis-meta/references/local-architecture/spec-system.md new file mode 100644 index 0000000..1ff49f4 --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/spec-system.md @@ -0,0 +1,102 @@ +# Local Spec System + +`.trellis/spec/` is the user's project-specific engineering spec library. Trellis is not about making AI memorize conventions; it injects relevant specs or requires the AI to read them at the right time. + +## Directory Model + +A common single-repository structure: + +```text +.trellis/spec/ +├── backend/ +│ ├── index.md +│ └── ... +├── frontend/ +│ ├── index.md +│ └── ... +└── guides/ + ├── index.md + └── ... +``` + +A common monorepo structure: + +```text +.trellis/spec/ +├── cli/ +│ ├── backend/ +│ │ ├── index.md +│ │ └── ... +│ └── unit-test/ +│ ├── index.md +│ └── ... +├── docs-site/ +│ └── docs/ +│ ├── index.md +│ └── ... +└── guides/ + ├── index.md + └── ... +``` + +`index.md` is the entry point for each layer. It should list the Pre-Development Checklist and Quality Check. Specific guidelines live in other Markdown files in the same directory. + +## Package Configuration + +`.trellis/config.yaml` can declare packages: + +```yaml +packages: + cli: + path: packages/cli + docs-site: + path: docs-site + type: submodule +default_package: cli +``` + +The AI can run: + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +This command lists packages and spec layers for the current project. Use this output as the reference when configuring context JSONL. + +## How Specs Enter Tasks + +Before a task enters implementation, planning may write relevant specs into `implement.jsonl` / `check.jsonl` when the task needs spec or research context beyond the task artifacts: + +```jsonl +{"file": ".trellis/spec/cli/backend/index.md", "reason": "CLI backend conventions"} +{"file": ".trellis/spec/cli/unit-test/conventions.md", "reason": "Test expectations"} +``` + +Sub-agents or platform preludes read these JSONL files and load the referenced specs. On platforms without sub-agent support, the AI should read the relevant specs directly according to the workflow. + +## What Specs Should Contain + +Specs should contain executable engineering conventions for the project, not generic best practices: + +- Where files should live. +- How error handling should be expressed. +- Input/output contracts for APIs, hooks, and commands. +- Patterns that are forbidden. +- Cases that require tests. +- Project-specific pitfalls and how to avoid them. + +When the AI learns a new rule during implementation or debugging, it should update `.trellis/spec/` rather than only summarizing it in chat. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Add a new spec layer | `.trellis/spec/<package>/<layer>/index.md` and corresponding guideline files. | +| Change monorepo spec mapping | `packages` / `default_package` / `spec_scope` in `.trellis/config.yaml`. | +| Change which specs AI reads before implementation | The task's `implement.jsonl`. | +| Change which specs AI reads during checking | The task's `check.jsonl`. | +| Change when specs should be updated | Phase 3.3 in `.trellis/workflow.md` and the `trellis-update-spec` skill. | + +## Boundaries + +`.trellis/spec/` is the user's project specification, not a permanent copy of Trellis built-in templates. The AI should encourage the user to update it according to the actual project code instead of treating Trellis default templates as immutable documents. diff --git a/.claude/skills/trellis-meta/references/local-architecture/task-system.md b/.claude/skills/trellis-meta/references/local-architecture/task-system.md new file mode 100644 index 0000000..9dfe5bb --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/task-system.md @@ -0,0 +1,130 @@ +# Local Task System + +The Trellis task system is stored entirely under `.trellis/tasks/` in the user project. Each task is a directory containing requirements, context, research, state, and relationship information. + +## Task Directory Structure + +```text +.trellis/tasks/ +├── 04-28-example-task/ +│ ├── task.json +│ ├── prd.md +│ ├── design.md +│ ├── implement.md +│ ├── implement.jsonl +│ ├── check.jsonl +│ └── research/ +└── archive/ + └── 2026-04/ +``` + +| File | Purpose | +| --- | --- | +| `task.json` | Task metadata: status, assignee, priority, branch, parent/child tasks, and similar fields. | +| `prd.md` | Requirements, constraints, and acceptance criteria. Lightweight tasks may be PRD-only. | +| `design.md` | Technical design for complex tasks: boundaries, contracts, data flow, compatibility, tradeoffs. | +| `implement.md` | Execution plan for complex tasks: ordered checklist, validation commands, review gates, rollback points. | +| `implement.jsonl` | List of spec/research files the implement agent must read first. | +| `check.jsonl` | List of spec/research files the check agent must read first. | +| `research/` | Research artifacts. Complex findings should not live only in chat. | + +## `task.json` + +`task.json` records task status and metadata. Common fields: + +| Field | Meaning | +| --- | --- | +| `id` / `name` / `title` | Task identity and title. | +| `status` | Status such as `planning`, `in_progress`, `review`, or `completed`. | +| `priority` | `P0`, `P1`, `P2`, `P3`. | +| `creator` / `assignee` | Creator and assignee. | +| `package` | Target package in a monorepo; may be empty. | +| `branch` / `base_branch` | Working branch and PR target branch. | +| `children` / `parent` | Parent/child task relationships. | +| `commit` / `pr_url` | Commit and PR information after completion. | +| `meta` | Extension fields. | + +## Parent / Child Task Trees + +Parent/child task relationships are for work structure. A parent task groups related deliverables under one source requirement set; it is not a dependency scheduler and does not replace the child task's own planning artifacts. + +Use a parent task when a request has multiple independently verifiable deliverables. The parent owns: + +- Source requirements and user-facing scope. +- The map of child tasks and their responsibility boundaries. +- Cross-child acceptance criteria and final integration review. + +Use child tasks for deliverables that can move through planning, implementation, check, and archive independently. If one child depends on another, write that dependency in the child `prd.md` / `implement.md`; do not rely on tree position to imply ordering. + +Create new children with: + +```bash +python ./.trellis/scripts/task.py create "<child title>" --slug <child-slug> --parent <parent-dir> +``` + +Link or unlink existing tasks with: + +```bash +python ./.trellis/scripts/task.py add-subtask <parent-dir> <child-dir> +python ./.trellis/scripts/task.py remove-subtask <parent-dir> <child-dir> +``` + +`children` on the parent is a historical list. When a child is archived, Trellis keeps that child name in the parent so progress like `[2/3 done]` remains meaningful after completed children move to `archive/`. + +The AI should not treat phase numbers as task status. Task progress is mainly determined by `status`, artifact presence (`prd.md`, optional `design.md` / `implement.md`), whether JSONL context is configured for sub-agent mode, and the phase descriptions in `workflow.md`. + +## Active Task + +The user sees a "current task," but Trellis stores active task state per session. + +```text +.trellis/.runtime/sessions/<context-key>.json +``` + +`task.py start` writes the task path into the runtime session file for the current session. `task.py current --source` shows the current task and where it came from. Different AI windows can point to different tasks without overwriting each other. + +If the platform or shell environment has no stable session identity, `task.py start` may be unable to set the active task. The AI should read the error, inspect the platform hook/session environment, and not fall back to a shared global pointer. + +## JSONL Context + +`implement.jsonl` and `check.jsonl` are context manifests for sub-agents to read first. They do not replace `implement.md`; `implement.md` is the human-readable execution plan. + +Format: + +```jsonl +{"file": ".trellis/spec/cli/backend/index.md", "reason": "Backend conventions"} +{"file": ".trellis/tasks/04-28-example/research/api.md", "reason": "API research"} +``` + +Rules: + +- Include spec and research files. +- Do not include code files that are about to be modified. +- Do not treat temporary conclusions in chat as the only context. +- Seed rows have no `file` field; they only prompt the AI to fill in real entries. + +## Common Commands + +```bash +python ./.trellis/scripts/task.py create "<title>" --slug <slug> +python ./.trellis/scripts/task.py start <task> +python ./.trellis/scripts/task.py current --source +python ./.trellis/scripts/task.py add-context <task> implement <file> <reason> +python ./.trellis/scripts/task.py validate <task> +python ./.trellis/scripts/task.py finish +python ./.trellis/scripts/task.py archive <task> +``` + +When modifying the task system, the AI should prefer script commands to maintain structure. Edit JSON/Markdown directly only when scripts do not cover the need. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Change the default task template | `.trellis/scripts/common/task_store.py` and task creation instructions. | +| Change status semantics | `.trellis/workflow.md`, workflow-state hook logic, and task usage conventions. | +| Add task lifecycle actions | `hooks.after_*` in `.trellis/config.yaml`. | +| Change context rules | Planning artifact guidance in `.trellis/workflow.md` and related platform agent/hook instructions. | +| Change archive policy | `.trellis/scripts/common/task_store.py` / `task_utils.py`. | + +These are local files in the user project. Do not default to editing Trellis CLI source code unless the user wants to contribute upstream. diff --git a/.claude/skills/trellis-meta/references/local-architecture/workflow.md b/.claude/skills/trellis-meta/references/local-architecture/workflow.md new file mode 100644 index 0000000..f0659ff --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/workflow.md @@ -0,0 +1,75 @@ +# Local Workflow System + +`.trellis/workflow.md` is the Trellis workflow source of truth inside the user project. An AI does not need Trellis source code to understand how the current project should move tasks forward; this file is enough. + +## File Responsibilities + +`.trellis/workflow.md` has three responsibilities: + +1. **Explain workflow phases**: Plan, Execute, Finish. +2. **Define skill routing**: which skill or agent the AI should use when the user expresses a certain intent. +3. **Provide workflow-state prompt blocks**: hooks can inject the prompt block for the current state into the conversation. + +## Current Phase Model + +```text +Phase 1: Plan -> clarify what to build, produce prd.md and required research +Phase 2: Execute -> implement against the PRD and specs, then check +Phase 3: Finish -> final verification, preserve lessons, and wrap up +``` + +Each phase contains numbered steps, such as `1.3 Configure context`. These numbers are not runtime fields in `task.json`; they are workflow structure for AI and humans to read. + +## Skill Routing + +`workflow.md` separates routing by platform capability: + +- Platforms with sub-agent support: dispatch `trellis-implement` by default for implementation and `trellis-check` for checking. +- Platforms without sub-agent support: the main session reads skills such as `trellis-before-dev`, then executes directly. + +When changing local AI behavior, update the routing descriptions in `workflow.md` first, then check whether the corresponding platform skill, command, or agent files need to stay in sync. + +## Workflow-State Prompt Blocks + +The bottom of `workflow.md` can contain state blocks like this: + +```text +[workflow-state:no_task] +... +[/workflow-state:no_task] +``` + +Hooks choose the right block based on current task status and inject it into the conversation. Common states include: + +| State | Meaning | +| --- | --- | +| `no_task` | The current session has no active task. | +| `planning` | The task is still in requirements, research, or context configuration. | +| `in_progress` | The task has entered implementation and checking. | +| `completed` | The task is complete and waiting for wrap-up or archive. | + +If the user wants to change policies such as "whether to create a task when there is no task," "when task creation may be skipped," or "whether sub-agents are required," edit these state blocks and the routing table above them. + +## Local Modification Patterns + +Common changes: + +| Goal | Edit point | +| --- | --- | +| Add a phase | Update the Phase Index, phase body, routing, and state blocks. | +| Change task creation policy | Update the `no_task` state block and Phase 1 description. | +| Change the default implementation/check path | Update Phase 2 and skill routing. | +| Change the wrap-up flow | Update Phase 3 and `finish-work` related descriptions. Note the current split: Phase 3.4 = AI-driven code commits (batched, user-confirmed), Phase 3.5 = `/finish-work` (archive + record session). `/finish-work` refuses to run if the working tree is dirty. | +| Change platform differences | Update routing descriptions grouped by platform. | + +After editing, make the AI reread `.trellis/workflow.md`; do not assume the flow from the old conversation is still valid. + +## Relationship To Platform Files + +`workflow.md` is the semantic center of the local workflow, but each platform can also have its own entry files: + +- skills, such as `trellis-brainstorm` and `trellis-check`. +- commands/prompts/workflows, such as continue and finish-work. +- hooks, such as session-start or workflow-state injection. + +If only `workflow.md` changes, platform entry files may still contain old language. When the user wants to change "what the AI actually does," also inspect the relevant platform directory. diff --git a/.claude/skills/trellis-meta/references/local-architecture/workspace-memory.md b/.claude/skills/trellis-meta/references/local-architecture/workspace-memory.md new file mode 100644 index 0000000..92d29f4 --- /dev/null +++ b/.claude/skills/trellis-meta/references/local-architecture/workspace-memory.md @@ -0,0 +1,71 @@ +# Local Workspace Memory System + +`.trellis/workspace/` stores cross-session memory. Its purpose is to let AI and humans understand what happened before across different windows and different days. + +## Directory Structure + +```text +.trellis/workspace/ +├── index.md +└── <developer>/ + ├── index.md + ├── journal-1.md + └── journal-2.md +``` + +| File | Purpose | +| --- | --- | +| `.trellis/.developer` | Current developer identity. | +| `.trellis/workspace/index.md` | Global workspace overview. | +| `.trellis/workspace/<developer>/index.md` | Session index for a developer. | +| `.trellis/workspace/<developer>/journal-N.md` | Session journal. | + +## Developer Identity + +Run this the first time: + +```bash +python ./.trellis/scripts/init_developer.py <name> +``` + +This creates `.trellis/.developer` and the corresponding workspace directory. The AI should not change developer identity casually; if the identity is wrong, first confirm who is using the current project. + +## Journal + +`journal-N.md` records completed or partially completed work from each session. By default, each journal holds about 2000 lines; after that it rotates to the next file. + +Common command for recording a session: + +```bash +python ./.trellis/scripts/add_session.py \ + --title "Session title" \ + --summary "What changed" \ + --commit "abc1234" +``` + +Planning or review work without a commit can also be recorded by using `--no-commit` or an empty commit value. + +## Relationship Between Workspace Memory And Tasks + +| System | What it stores | +| --- | --- | +| `.trellis/tasks/` | Requirements, design, research, and state for a specific task. | +| `.trellis/workspace/` | Work records across tasks and sessions. | +| `.trellis/spec/` | Engineering knowledge preserved as long-term conventions. | + +If information is only useful for the current task, put it in the task directory. +If information describes what happened in the current session, put it in the workspace journal. +If information should be followed every time code is written in the future, put it in spec. + +## Local Customization Points + +| Need | Edit location | +| --- | --- | +| Change maximum journal lines | `max_journal_lines` in `.trellis/config.yaml`. | +| Change session auto-commit message | `session_commit_message` in `.trellis/config.yaml`. | +| Change session content format | `.trellis/scripts/add_session.py`. | +| Change how workspace is displayed in context | `.trellis/scripts/common/session_context.py`. | + +## AI Usage Rules + +The AI should not treat workspace as the only source of truth. When resuming a task, read the current task first, then use workspace for background. After a task is complete, record important process notes in workspace; if long-term rules emerged, update spec. diff --git a/.claude/skills/trellis-meta/references/platform-files/agents.md b/.claude/skills/trellis-meta/references/platform-files/agents.md new file mode 100644 index 0000000..3976987 --- /dev/null +++ b/.claude/skills/trellis-meta/references/platform-files/agents.md @@ -0,0 +1,80 @@ +# Agents + +Trellis agent files define specialized roles. Common Trellis agents in a user project are: + +- `trellis-research` +- `trellis-implement` +- `trellis-check` + +File locations and formats differ by platform, but responsibility boundaries should stay consistent. + +## Agent Responsibilities + +| Agent | Responsibility | +| --- | --- | +| `trellis-research` | Investigate the question and write findings into the current task's `research/`. | +| `trellis-implement` | Implement against `prd.md`, optional `design.md` / `implement.md`, `implement.jsonl`, and related spec/research. | +| `trellis-check` | Review changes, fix discovered issues, and run necessary checks. | + +Agent files should not become generic chat prompts. They should define input sources, write boundaries, whether code may be changed, and how results are reported. + +## Common Paths + +| Platform | Agent path | +| --- | --- | +| Claude Code | `.claude/agents/trellis-*.md` | +| Cursor | `.cursor/agents/trellis-*.md` | +| OpenCode | `.opencode/agents/trellis-*.md` | +| Codex | `.codex/agents/trellis-*.toml` | +| Kiro | `.kiro/agents/trellis-*.json` | +| Gemini CLI | `.gemini/agents/trellis-*.md` | +| Qoder | `.qoder/agents/trellis-*.md` | +| CodeBuddy | `.codebuddy/agents/trellis-*.md` | +| Factory Droid | `.factory/droids/trellis-*.md` | +| Pi Agent | `.pi/agents/trellis-*.md` | + +GitHub Copilot agent/prompt support is provided by a combination of directories such as `.github/agents/`, `.github/prompts/`, and `.github/skills/`; inspect the files actually generated in the user project. + +Main-session workflow platforms such as Kilo, Antigravity, and Windsurf may not have Trellis sub-agent files. They usually rely on workflows/skills to guide the main session. + +## Two Context Loading Modes + +### hook push + +The platform hook injects task context before the agent starts. The agent file itself can focus more on responsibilities and boundaries. + +Common on platforms that support agent hooks. + +### agent pull + +The agent file instructs the agent to read after startup: + +- `python ./.trellis/scripts/task.py current --source` +- `implement.jsonl` or `check.jsonl` +- spec/research files referenced by JSONL +- current task `prd.md` +- `design.md` if present +- `implement.md` if present + +This mode fits platforms whose hooks cannot reliably rewrite sub-agent prompts. + +## Local Change Scenarios + +| User need | Edit location | +| --- | --- | +| Implement agent must follow extra restrictions | The platform's `trellis-implement` agent file. | +| Check agent must run project-specific commands | `trellis-check` agent file, and `.trellis/spec/` if needed. | +| Research agent must output a fixed format | `trellis-research` agent file. | +| Agent cannot read task context | Agent prelude or `inject-subagent-context` hook. | +| Add a project-specific agent | Platform agent directory + related workflow/command/skill entry point. | + +## Modification Principles + +1. **Keep responsibilities single-purpose**. Do not mix research, implement, and check responsibilities into one agent. +2. **Specify the read order**. Agents must know to start from the active task, read jsonl/spec context, then read `prd.md`, `design.md` if present, and `implement.md` if present. +3. **Specify write boundaries**. Research usually only writes `research/`; implement can write code; check can fix issues. +4. **Keep semantics synchronized in multi-platform projects**. If the user configured Claude, Codex, and Cursor together, decide whether changes to one platform's agent also need to be applied to others. + +## Do Not Default To Editing Upstream Templates + +Local AI should default to modifying platform agent files inside the user project. Discuss upstream template source only when the user explicitly wants to contribute the change back to Trellis. diff --git a/.claude/skills/trellis-meta/references/platform-files/hooks-and-settings.md b/.claude/skills/trellis-meta/references/platform-files/hooks-and-settings.md new file mode 100644 index 0000000..94156a8 --- /dev/null +++ b/.claude/skills/trellis-meta/references/platform-files/hooks-and-settings.md @@ -0,0 +1,69 @@ +# Hooks And Settings + +Hooks/settings are the entry layer that connects a platform to Trellis. They decide which scripts, plugins, or extensions a platform runs for which events. + +## Settings Responsibilities + +settings/config files usually register: + +- session-start hook: injects a Trellis overview when a new session starts or context resets. +- workflow-state hook: parses `[workflow-state:STATUS]` blocks from `.trellis/workflow.md` and emits the body matching the current task `status` on each user input. Parser-only; the script does not embed fallback content. +- sub-agent context hook: injects task context when implementation/check/research agents start. +- shell/session bridge: lets shell commands see the same Trellis session identity. +- platform plugin or extension entry points. + +Common files: + +| Platform | settings/config | +| --- | --- | +| Claude Code | `.claude/settings.json` | +| Cursor | `.cursor/hooks.json` | +| Codex | `.codex/hooks.json`, `.codex/config.toml` | +| OpenCode | `.opencode/package.json`, `.opencode/plugins/*` | +| Kiro | `.kiro/hooks/` + platform config | +| Gemini CLI | `.gemini/settings.json` | +| Qoder | `.qoder/settings.json` | +| CodeBuddy | `.codebuddy/settings.json` | +| GitHub Copilot | `.github/copilot/hooks.json` | +| Factory Droid | `.factory/settings.json` | +| Pi Agent | `.pi/settings.json`, `.pi/extensions/trellis/` | + +Whether these files exist in a project depends on which `trellis init --<platform>` flags the user ran. + +## Hook Script Types + +| Script | Purpose | +| --- | --- | +| `session-start.py` | Generates session-start context. | +| `inject-workflow-state.py` | Parses `[workflow-state:STATUS]` blocks in `.trellis/workflow.md` and emits the body matching the current task status. Falls back to `Refer to workflow.md for current step.` when no matching block exists. | +| `inject-subagent-context.py` | Injects PRD, JSONL context, and related spec/research into sub-agents. | +| `inject-shell-session-context.py` | Lets shell commands inherit Trellis session identity. | + +Not every platform has every hook. Do not copy files from another platform just because a platform lacks a hook; first confirm whether that platform supports the corresponding event. + +## Local Change Scenarios + +| User need | Edit location | +| --- | --- | +| AI should see more/less context in a new session | Platform `session-start` hook. | +| Per-turn hint policy should change | `[workflow-state:STATUS]` block in `.trellis/workflow.md`. The hook parses workflow.md verbatim — no script edit required. | +| Sub-agent cannot read PRD/spec | `inject-subagent-context` hook or agent prelude. | +| `task.py current` in shell has no active task | Shell/session bridge hook or platform environment variable configuration. | +| Disable an automatic injection | The corresponding hook registration in settings/config. | + +## Modification Principles + +1. **Settings wire things up; hooks define behavior**. If only the hook changes, the platform may never call it. If only settings change, behavior may not change. +2. **Confirm platform event names first**. Different platforms use different names for SessionStart, UserPromptSubmit, AgentSpawn, shell execution, and similar events. +3. **Hooks read local `.trellis/`, not upstream source**. `.trellis/scripts/` and `.trellis/workflow.md` in the user project are the default targets. +4. **Errors must be visible**. Hook failures should tell the user what was not injected instead of silently leaving the AI without context. + +## Troubleshooting Path + +If the user says "AI did not read Trellis state": + +1. Check whether the platform settings register the hook. +2. Check whether the hook file exists. +3. Manually run the `.trellis/scripts/get_context.py` or `task.py current --source` command that the hook depends on. +4. Check whether active task state exists in `.trellis/.runtime/sessions/`. +5. Check whether the platform shell passes session identity. diff --git a/.claude/skills/trellis-meta/references/platform-files/overview.md b/.claude/skills/trellis-meta/references/platform-files/overview.md new file mode 100644 index 0000000..60ae1df --- /dev/null +++ b/.claude/skills/trellis-meta/references/platform-files/overview.md @@ -0,0 +1,59 @@ +# Platform Files Overview + +Trellis connects the same local architecture to different AI tools. `.trellis/` stores the shared runtime; platform directories store adapter files that define how each AI tool enters Trellis. + +When a local AI modifies Trellis, it should distinguish two file categories first: + +- **Shared files**: `.trellis/workflow.md`, `.trellis/tasks/`, `.trellis/spec/`, `.trellis/scripts/`. +- **Platform files**: `.claude/`, `.codex/`, `.cursor/`, `.opencode/`, `.kiro/`, `.gemini/`, `.qoder/`, `.codebuddy/`, `.github/`, `.factory/`, `.pi/`, `.kilocode/`, `.agent/`, `.windsurf/`, and similar directories. + +Platform files do not store business state. They let the corresponding AI tool read Trellis state, call Trellis scripts, and load Trellis skills/agents/hooks. + +## Platform File Categories + +| Category | Common paths | Purpose | +| --- | --- | --- | +| settings/config | `.claude/settings.json`, `.codex/hooks.json`, `.qoder/settings.json` | Register hooks, plugins, extensions, or platform behavior. | +| hooks/plugins/extensions | `.claude/hooks/`, `.opencode/plugins/`, `.pi/extensions/` | Inject context at session start, user input, agent startup, shell execution, and similar events. | +| agents | `.claude/agents/`, `.codex/agents/`, `.kiro/agents/` | Define `trellis-research`, `trellis-implement`, and `trellis-check`. | +| skills | `.claude/skills/`, `.agents/skills/`, `.qoder/skills/` | Capability descriptions that auto-trigger or can be read on demand. | +| commands/prompts/workflows | `.cursor/commands/`, `.github/prompts/`, `.windsurf/workflows/` | Entry points explicitly invoked by the user. | + +## Three Platform Integration Modes + +### 1. Hook / Extension Driven + +These platforms can trigger scripts or plugins on specific events and actively inject Trellis context into AI. + +Common capabilities: + +- session-start injection of a `.trellis/` overview. +- workflow-state hints for each user turn. +- PRD/spec/research injection when sub-agents start. +- Shell commands inheriting session identity. + +To change "when the AI knows what," inspect hooks/plugins/extensions and settings first. + +### 2. Agent Prelude / Pull-Based + +Some platforms cannot reliably let hooks rewrite sub-agent prompts, so the agent file itself instructs the agent to read the active task, PRD, and JSONL context after startup. + +To change how sub-agents load context, inspect the agent files themselves. + +### 3. Main-Session Workflow + +Some platforms do not have Trellis sub-agent or hook capabilities. They rely on workflows/skills/commands to guide the main-session AI to read files, run scripts, and move tasks forward. + +To change behavior, inspect platform workflows/skills/commands and `.trellis/workflow.md`. + +## Local Modification Order + +When the user asks to customize behavior for a platform, the AI should inspect files in this order: + +1. Read `.trellis/workflow.md` to confirm the shared flow. +2. Read the target platform's settings/config to see which hooks/agents/skills/commands are registered. +3. Read the target platform's agents/skills/commands/hooks. +4. Modify the local file closest to the user's need. +5. If the change affects the shared flow, synchronize `.trellis/workflow.md` or `.trellis/spec/`. + +Do not modify only platform files and forget the shared workflow. Do not modify only `.trellis/workflow.md` and forget that platform entry points may still contain old descriptions. diff --git a/.claude/skills/trellis-meta/references/platform-files/platform-map.md b/.claude/skills/trellis-meta/references/platform-files/platform-map.md new file mode 100644 index 0000000..b5576f4 --- /dev/null +++ b/.claude/skills/trellis-meta/references/platform-files/platform-map.md @@ -0,0 +1,74 @@ +# Platform File Map + +This page lists common Trellis file locations in a user project by platform. Whether a platform directory exists in an actual project depends on which `trellis init --<platform>` commands the user ran. + +## Matrix + +| Platform | CLI flag | Main directory | Skill directory | Agent directory | Hooks/extensions | +| --- | --- | --- | --- | --- | --- | +| Claude Code | `--claude` | `.claude/` | `.claude/skills/` | `.claude/agents/` | `.claude/hooks/` + `.claude/settings.json` | +| Cursor | `--cursor` | `.cursor/` | `.cursor/skills/` | `.cursor/agents/` | `.cursor/hooks.json` + `.cursor/hooks/` | +| OpenCode | `--opencode` | `.opencode/` | `.opencode/skills/` | `.opencode/agents/` | `.opencode/plugins/` | +| Codex | `--codex` | `.codex/` | `.agents/skills/` | `.codex/agents/` | `.codex/hooks/` + `.codex/hooks.json` | +| Kilo | `--kilo` | `.kilocode/` | `.kilocode/skills/` | Usually none | `.kilocode/workflows/` | +| Kiro | `--kiro` | `.kiro/` | `.kiro/skills/` | `.kiro/agents/` | `.kiro/hooks/` | +| Gemini CLI | `--gemini` | `.gemini/` | `.agents/skills/` | `.gemini/agents/` | `.gemini/settings.json` + `.gemini/hooks/` | +| Antigravity | `--antigravity` | `.agent/` | `.agent/skills/` | Usually none | `.agent/workflows/` | +| Windsurf | `--windsurf` | `.windsurf/` | `.windsurf/skills/` | Usually none | `.windsurf/workflows/` | +| Qoder | `--qoder` | `.qoder/` | `.qoder/skills/` | `.qoder/agents/` | `.qoder/hooks/` + `.qoder/settings.json` | +| CodeBuddy | `--codebuddy` | `.codebuddy/` | `.codebuddy/skills/` | `.codebuddy/agents/` | `.codebuddy/hooks/` + `.codebuddy/settings.json` | +| GitHub Copilot | `--copilot` | `.github/` | `.github/skills/` | `.github/agents/` | `.github/copilot/hooks/` + prompts | +| Factory Droid | `--droid` | `.factory/` | `.factory/skills/` | `.factory/droids/` | `.factory/hooks/` + settings | +| Pi Agent | `--pi` | `.pi/` | `.pi/skills/` | `.pi/agents/` | `.pi/extensions/trellis/` + `.pi/settings.json` | + +## Capability Groups + +### Trellis Sub-Agent Support + +These platforms usually have `trellis-research`, `trellis-implement`, and `trellis-check` files: + +- Claude Code +- Cursor +- OpenCode +- Codex +- Kiro +- Gemini CLI +- Qoder +- CodeBuddy +- GitHub Copilot +- Factory Droid +- Pi Agent + +When changing implementation/check/research behavior, look for the corresponding platform agent files first. + +### Main-Session Workflow Platforms + +These platforms rely more on workflows/skills to guide the main session: + +- Kilo +- Antigravity +- Windsurf + +When changing behavior, inspect workflows and skills first. Do not assume Trellis sub-agents exist. + +### Shared `.agents/skills/` + +Codex writes the shared `.agents/skills/` layer. Some tools that support agentskills.io can also read this directory. If the user wants multiple compatible tools to share one skill, consider `.agents/skills/` first, but do not assume every platform reads it. + +## Decision Rules When Modifying Platform Files + +1. User specified a platform: modify only that platform directory unless shared workflow/spec files must also change. +2. User says "all platforms should do this": synchronize equivalent entry points platform by platform; do not modify only one directory. +3. User only says "my AI": inspect the configuration directories that actually exist in the project and infer the current AI platform. +4. User wants project rules: prefer `.trellis/spec/` or a project-local skill. +5. User wants Trellis behavior: edit `.trellis/workflow.md` plus platform hooks/agents/skills/commands. + +## When Paths Differ + +Platform ecosystems change, and user projects may already be customized. If this table disagrees with local files, use the actual settings/config in the user project as authoritative: + +- Check the hook that settings registers. +- Check the script that a command/prompt/workflow points to. +- Judge behavior by the read rules currently written in the agent file. + +Do not delete a custom file just because it is not listed in this path table. diff --git a/.claude/skills/trellis-meta/references/platform-files/skills-and-commands.md b/.claude/skills/trellis-meta/references/platform-files/skills-and-commands.md new file mode 100644 index 0000000..816c666 --- /dev/null +++ b/.claude/skills/trellis-meta/references/platform-files/skills-and-commands.md @@ -0,0 +1,83 @@ +# Skills, Commands, Prompts, And Workflows + +Skills and commands are textual entry points for user interaction with Trellis. Different platforms use different names, but their core purpose is the same: tell the AI how to enter the Trellis flow when the user expresses a certain intent. + +## Conceptual Differences + +| Type | Trigger mode | Best for | +| --- | --- | --- | +| skill | AI auto-match or explicit user mention | Long-term capabilities, workflow rules, modification guides. | +| command | Explicit user invocation | Clear operation entry points such as continue and finish-work. | +| prompt | Explicit user invocation or platform selection | Similar to command, but in a platform prompt format. | +| workflow | Explicit user selection or platform auto-match | Guides the main session when no sub-agent/hook exists. | + +Trellis workflow skills usually share one semantic set: brainstorm, before-dev, check, update-spec, break-loop. Multi-file built-in skills such as `trellis-meta` use layered references. + +## Common Paths + +| Platform | Common entries | +| --- | --- | +| Claude Code | `.claude/skills/`, `.claude/commands/` | +| Cursor | `.cursor/skills/`, `.cursor/commands/` | +| OpenCode | `.opencode/skills/`, `.opencode/commands/` | +| Codex | `.agents/skills/`, `.codex/skills/` | +| Kilo | `.kilocode/skills/`, `.kilocode/workflows/` | +| Kiro | `.kiro/skills/` | +| Gemini CLI | `.agents/skills/`, `.gemini/commands/` | +| Antigravity | `.agent/skills/`, `.agent/workflows/` | +| Windsurf | `.windsurf/skills/`, `.windsurf/workflows/` | +| Qoder | `.qoder/skills/`, `.qoder/commands/` | +| CodeBuddy | `.codebuddy/skills/`, `.codebuddy/commands/` | +| GitHub Copilot | `.github/skills/`, `.github/prompts/` | +| Factory Droid | `.factory/skills/`, `.factory/commands/` | +| Pi Agent | `.pi/skills/` | + +In a user project, use the files actually generated by init as authoritative. + +## Skill Structure + +A common skill is a directory: + +```text +trellis-meta/ +├── SKILL.md +└── references/ +``` + +`SKILL.md` should tell the AI: + +- When to use this skill. +- Which reference to read first for the current task. +- What not to do. + +References hold longer explanations so the entry file does not contain everything. + +## Command/Prompt/Workflow Structure + +Commands, prompts, and workflows are usually single files. Their content should include: + +- When to use it. +- Which `.trellis/` files to read. +- Which scripts to run. +- How to report after completion. + +They should not store task state; task state belongs in `.trellis/tasks/` and `.trellis/.runtime/`. + +## Local Change Scenarios + +| User need | Edit location | +| --- | --- | +| Change AI auto-trigger rules | The corresponding skill's frontmatter description. | +| Change user command behavior | The corresponding command/prompt/workflow file. | +| Add a project-local skill | Platform skill directory, or shared `.agents/skills/`. | +| Let multiple platforms share one capability | Write equivalent skills in each platform skill directory, or use the `.agents/skills/` shared layer on platforms that support it. | +| Change finish/continue entry points | Platform commands/prompts/workflows. | + +## Modification Principles + +1. **Keep entry files short; references carry long content**. This matters especially for multi-file skills like `trellis-meta`. +2. **Make trigger descriptions specific**. A description that is too broad can mis-trigger; one that is too narrow may not trigger. +3. **Keep the same semantics consistent across platforms**. File formats can differ, but behavior descriptions should match. +4. **Put project-specific capabilities in local skills**. Do not put team-private flows into public `trellis-meta`. + +If the user only wants local AI to know one more project rule, usually create a project-local skill or update `.trellis/spec/` instead of changing a Trellis built-in workflow skill. diff --git a/.claude/skills/trellis-session-insight/SKILL.md b/.claude/skills/trellis-session-insight/SKILL.md new file mode 100644 index 0000000..3670739 --- /dev/null +++ b/.claude/skills/trellis-session-insight/SKILL.md @@ -0,0 +1,81 @@ +--- +name: trellis-session-insight +description: "Reach into past AI conversation history through the `trellis mem` CLI. Use whenever the user asks 'how did we solve X last time', 'have we discussed this before', 'what was the decision on X', 'remind me what we did in this task', '上次怎么解的', '之前讨论过吗', '想起一段对话', or when starting a brainstorm that overlaps prior work, debugging a familiar bug, continuing a task across sessions, or doing a finish-work review. Returns raw past dialogue; decide for the moment whether to update spec, append to task notes, quote inline in the answer, or just internalize." +--- + +# Trellis Session Insight + +This skill teaches an AI **how to call `trellis mem`** — the project's cross-session memory feedstock — and **when reaching for it is the right move**. + +It is intentionally a **capability skill, not a workflow**. There is no fixed output file, no required write-back step, no "always run after finish-work" rule. What to do with what `mem` returns is a judgement call made in the moment of the conversation. The skill exists so the AI knows the capability is there and can decide. + +## What `trellis mem` is + +A local CLI that indexes the user's past Claude Code and Codex conversation logs (the JSONL files each platform stores under `~/.claude/projects/` and `~/.codex/sessions/`) and lets you list, search, slice by Trellis task boundaries, and dump cleaned dialogue from them. OpenCode logs are not yet indexable (provider adapter pending) — when an OpenCode session is the obvious target, surface that limitation rather than guessing. + +Nothing in `mem` is uploaded. All reads are local. + +## When to reach for it + +The bar is "would a senior teammate ask 'didn't we already talk about this?'" — those are the moments. Some concrete patterns: + +- **Brainstorm rerun risk.** Starting a new task that touches an area the user has been in before, and you want to check whether a decision was already made — before re-asking the user. +- **Familiar-bug debugging.** The current bug pattern feels like one the user reported / fixed before. Pulling the relevant past session can save a full debugging loop. +- **Cross-session continuation.** The user resumes work after a gap and says "where were we" / "继续上次的" without being specific. +- **Decision retrieval.** The user references "the decision we made about X" but the decision lives in an old brainstorm, not in any `prd.md` / `spec/`. +- **Finish-work retrospective.** When the user explicitly asks for a wrap-up of what was decided / what hurt / what surprised them in this task — not as a forced step on every finish-work. +- **Pattern-spotting across past work.** The user asks "do I keep making the same mistake on X" / "我每次都踩这个坑吗" — search across sessions answers that. + +If none of these apply, don't call `mem`. It is a tool, not a ceremony. + +## When NOT to reach for it + +- The relevant context is already in the current turn, `prd.md`, `design.md`, recent `git log`, or the open files. `mem` is for stuff that has fallen out of immediate reach. +- The user is asking about a fact in the code, not a fact from a past conversation. `git log -p` / `grep` / reading the file directly is faster and more authoritative. +- You are in a sub-agent (`trellis-implement` / `trellis-check`) whose dispatch prompt already includes the curated `implement.jsonl` / `check.jsonl` context. Adding `mem` on top usually just clutters. +- The user has explicitly said "don't dig through history, just answer what I asked". + +## What to do with what `mem` returns + +Treat the output as **raw material**, not a deliverable. Once you have it, decide based on the live conversation: + +- **Quote inline in your reply** if a specific past exchange answers the user's current question — and cite the session-id / phase so the user can verify. +- **Update `<task>/prd.md` or `<task>/design.md`** if `mem` surfaced a load-bearing decision that should have been written down but wasn't. Surface the proposed edit to the user first. +- **Append to a task-local notes file** (e.g. `<task>/notes.md` or extending an existing one) if the finding belongs to the current task's record but doesn't fit the PRD. +- **Update `.trellis/spec/`** if the finding is a project-wide convention or gotcha that would help future tasks. Run the `trellis-update-spec` skill for that — `session-insight` ends at the discovery. +- **Just absorb it** for the next few turns and answer better, without writing anything. This is often the right move for one-off recall. + +Trellis does not prescribe a single destination. Forcing every recall into a fixed file makes the file grow into noise. Let the situation decide. + +## How to call it + +Full CLI reference is in `references/cli-quick-reference.md`. The 80% case is one of: + +```bash +# Find sessions whose contents mention a keyword (project-scope is default; +# add --global to search every project on this machine). +trellis mem search "<keyword>" + +# Dump dialogue from one session, optionally filtered by phase or keyword. +trellis mem extract <session-id> --phase brainstorm +trellis mem extract <session-id> --grep "<keyword>" + +# Drill into a session: top-N hit turns + surrounding context. +trellis mem context <session-id> --turns 3 --around 2 + +# When you do not know the session id yet, start with list + filter. +trellis mem list --task <task-dir> +trellis mem projects # → list active project cwds, then narrow +``` + +Phase slicing (`--phase brainstorm|implement|all`) cuts the session at `task.py create` and `task.py start` boundaries. For a finish-work review of the current task, `--phase brainstorm` recovers the planning discussion and `--phase implement` recovers the execution loop. Default is `all`. + +## Triggering patterns + +`references/triggering-patterns.md` lists more verbatim user phrasings (English + Chinese) that should make you think "reach for `mem`" — keep that handy when training instinct. + +## Out of scope + +- `mem` does not edit code or update files. Any write-back is your decision in the moment. +- `mem` is read-only on the platform JSONL stores. It does not push or sync to remote. +- This skill does not replace `trellis-update-spec` (which is the right tool for promoting a finding into project-wide guidance) or the platform-native task / spec workflow. diff --git a/.claude/skills/trellis-session-insight/references/cli-quick-reference.md b/.claude/skills/trellis-session-insight/references/cli-quick-reference.md new file mode 100644 index 0000000..3d5f95c --- /dev/null +++ b/.claude/skills/trellis-session-insight/references/cli-quick-reference.md @@ -0,0 +1,66 @@ +# `trellis mem` CLI Reference + +Full flag reference for the five subcommands. Pin this as the authoritative source — `trellis mem help` prints the same content at runtime, so anything here that drifts is a bug. + +## Subcommands + +| Command | Purpose | +|---|---| +| `list` | List sessions. Default subcommand when none is given. | +| `search <keyword>` | Find sessions whose contents match a keyword. | +| `context <session-id>` | Drill into one session: top-N hit turns + surrounding context. Pair with `--grep` for keyword anchoring. | +| `extract <session-id>` | Dump cleaned dialogue. Combine with `--phase` / `--grep` to slice. | +| `projects` | List active project `cwd` values with session counts. Use this to discover which `--cwd` to pass to other subcommands. | + +## Flags (apply where meaningful) + +| Flag | Subcommands | Meaning | +|---|---|---| +| `--platform claude\|codex\|opencode\|all` | all | Default `all`. OpenCode adapter is currently a stub on `0.6.0-beta.*` — see "Caveats" below. | +| `--since YYYY-MM-DD` | list / search | Inclusive lower date bound. | +| `--until YYYY-MM-DD` | list / search | Inclusive upper date bound. | +| `--global` | list / search | Include sessions from every project on this machine. Default is the current project `cwd`. | +| `--cwd <path>` | list / search | Force a specific project cwd instead of inferring from where you are. | +| `--limit N` | list / search | Cap output rows. Default `50`. | +| `--grep KW` | extract / context | Filter turns by keyword. Multi-token AND when whitespace-separated. | +| `--phase brainstorm\|implement\|all` | extract | Slice session by Trellis task boundaries. `brainstorm` = `[task.py create, task.py start)`. `implement` = `[task.py start, task.py finish)` window. Default `all`. | +| `--turns N` | context | Number of hit turns to return. Default `3`. | +| `--around N` | context | Surrounding turns to include per hit. Default `1`. | +| `--max-chars N` | context | Total character budget. Default `6000` (~1500 tokens). | +| `--include-children` | search / context | Merge OpenCode sub-agent sessions into their parent session. | +| `--json` | all | Emit machine-parseable JSON instead of human-readable output. | +| `--task <task-dir>` | list | Narrow to sessions whose context-key resolved to a given task directory (uses `.trellis/.runtime/sessions/*.json`). | + +## Common one-liners + +```bash +# What past sessions discussed "deadlock" anywhere on this machine? +trellis mem search "deadlock" --global --limit 20 + +# Inside a specific session, surface the top 5 turns that mention "lock contention" +# plus 2 turns of surrounding context. +trellis mem context 5842592d --grep "lock contention" --turns 5 --around 2 + +# Recover the brainstorm window for a session — useful when continuing a task +# the user started a week ago. +trellis mem extract 5842592d --phase brainstorm + +# List every project this machine has Trellis sessions for, with counts. +trellis mem projects +``` + +## Output shapes + +- **Default human output** (no `--json`): wrapped to a terminal, with session ids highlighted and turn markers visible. Suitable to read inline but messy to paste into a markdown file. +- **`--json`**: stable schema, safe to parse and process. When piping `mem` output into a follow-up step (e.g. summarizing for a Lessons section), prefer `--json`. + +## Caveats + +- **OpenCode adapter is a stub on `0.6.0-beta.*`.** When `--platform` resolves to OpenCode (or `all` and OpenCode would be included), `mem` prints a one-line "reader unavailable" notice and continues with the other platforms. Don't promise OpenCode coverage in your reply until the adapter ships. +- **`--phase` slicing depends on `task.py create` / `task.py start` invocations appearing in the recorded bash calls of the session.** Sessions where the user ran `task.py` from a different terminal — outside the recorded AI loop — will not have phase boundaries. `--phase all` is the safe fallback. +- **`mem` indexes platform JSONL files directly.** If the user has cleared their Claude / Codex session storage, `mem` cannot recover what is no longer on disk. +- **`mem` is read-only.** No remote sync, no edits to platform JSONL. Any write you do based on `mem` findings is your own follow-up call into the editing tools available to you. + +## When you need more than this reference + +Run `trellis mem help` in the user's shell. The runtime help is authoritative and will be ahead of this reference during fast-moving beta releases. diff --git a/.claude/skills/trellis-session-insight/references/triggering-patterns.md b/.claude/skills/trellis-session-insight/references/triggering-patterns.md new file mode 100644 index 0000000..66021ca --- /dev/null +++ b/.claude/skills/trellis-session-insight/references/triggering-patterns.md @@ -0,0 +1,93 @@ +# Triggering Patterns + +Verbatim user phrasings that should make an AI reach for `trellis mem`. Calibrate instinct against these — if a user message hits one of these patterns and you do not reach for `mem`, you probably missed an obvious recall. + +Patterns are grouped by the *intent* behind the phrasing, not the surface words. The same intent shows up in different languages and registers. + +## Past-solution recall + +The user is asking "how did we (or I) solve this before". Past dialogue holds the answer; the codebase shows the result but not the reasoning. + +- "How did we solve this last time?" +- "What did we end up doing about X?" +- "We dealt with this once already, didn't we?" +- "上次怎么解的?" +- "之前是怎么搞定 X 的?" +- "我记得以前修过类似的" + +Reach: `trellis mem search "<symptom keyword>" --global --limit 10`, then `context` into the hit that looks closest. + +## Decision retrieval + +The user is referencing a decision that lives in old dialogue, not in any committed file. Look in brainstorm windows. + +- "What was the decision on X?" +- "Did we decide to use Postgres or SQLite?" +- "The rationale for choosing X over Y was…?" +- "我们当时为啥选了 X 而不是 Y?" +- "关于 X 我们之前是怎么定的?" +- "之前讨论过 X 的方案吗?" + +Reach: `trellis mem search "<decision keyword>"` to find the session, then `extract <id> --phase brainstorm` to recover the discussion. + +## Cross-session continuation + +The user resumed work after a gap and the context is implicit. + +- "Where were we?" +- "Continue from last time." +- "Pick up where we left off." +- "继续上次的" +- "我们上次做到哪了" +- "接着昨天那个任务" + +Reach: `trellis mem list --task <current-task-dir>` to find the most recent sessions tied to the active task, then `extract` the last one. + +## Familiar-bug debugging + +The current bug feels like one already seen. Past sessions probably hold the resolution path. + +- "I feel like I've hit this before." +- "Doesn't this look like that bug from last month?" +- "Same kind of timeout I had in X." +- "这个错好像之前见过" +- "这个 bug 是不是上次那个?" +- "怎么又是这个 error?" + +Reach: `trellis mem search "<error message fragment>" --global`. Anchor on a short, distinctive token from the actual error string. + +## Self-pattern spotting + +The user is asking whether they keep repeating the same kind of mistake or decision. + +- "Do I always make this mistake?" +- "How often have I run into X?" +- "Is this a recurring thing for me?" +- "我每次都踩这个坑吗?" +- "我老犯这个错?" +- "这类问题之前出现过几次?" + +Reach: `trellis mem search "<topic>" --global --limit 50` and scan the dates / projects in the listing. Optionally `extract` two or three for comparison. + +## Finish-work retrospective (on demand) + +The user explicitly wants to look back at this task — not as a forced step, only when they ask. + +- "Summarize what we did in this task." +- "What were the key decisions / surprises?" +- "Write up the lessons from this round." +- "总结一下这次的经验" +- "记一下这次踩的坑" +- "复盘下这个任务" + +Reach: identify the current task's session id (from `.trellis/.runtime/sessions/*.json` or `mem list --task <task-dir>`), then `extract <id> --phase brainstorm` and `--phase implement`. Present a summary — surface concrete file:line citations where possible. Whether to also write the summary somewhere (PRD, spec, notes file) is the user's call; offer, don't auto-write. + +## Anti-patterns: do NOT reach for `mem` here + +- "What does this function do?" → read the file. +- "Why is this test failing?" → read the test output and the file. +- "What's the right pattern for X in our codebase?" → grep / read spec files. +- "What's the latest npm version of Y?" → call `npm view`. +- "Fix this bug." → debug. Reach for `mem` only if you suspect prior context exists; otherwise it is noise. + +The bar stays: would a senior teammate ask "didn't we already talk about this?" before answering? If yes, reach for `mem`. If no, don't. diff --git a/.claude/skills/trellis-spec-bootstrap/SKILL.md b/.claude/skills/trellis-spec-bootstrap/SKILL.md new file mode 100644 index 0000000..e1650df --- /dev/null +++ b/.claude/skills/trellis-spec-bootstrap/SKILL.md @@ -0,0 +1,41 @@ +--- +name: trellis-spec-bootstrap +description: "Bootstrap project-specific Trellis coding specs with a platform-neutral single-agent workflow. Use when creating or refreshing .trellis/spec guidelines, analyzing a codebase with GitNexus, ABCoder, or source inspection, decomposing package/layer spec work, and writing real codebase-backed spec docs without placeholder text." +--- + +# Trellis Spec Bootstrap + +Use this skill to create or refresh `.trellis/spec/` guidelines from the real codebase. One capable agent owns the full loop: analyze the repository, choose the spec boundaries, write the docs, and verify the result. The workflow does not depend on a specific host, CLI, or agent brand. + +## Workflow + +1. Confirm Trellis is initialized and inspect the current `.trellis/spec/` tree. +2. Analyze the repository architecture with the best available tools: GitNexus, ABCoder, language tooling, and direct source reads. +3. Decompose the spec work by package and layer only when that reflects the actual codebase. +4. Fill or reshape the spec files with concrete patterns, file paths, examples, and anti-patterns from the project. +5. Verify that the final specs are internally consistent and contain no template placeholders. + +## Reference Routing + +| Need | Read | +|------|------| +| Repository architecture analysis | [references/repository-analysis.md](references/repository-analysis.md) | +| Spec work decomposition and task planning | [references/spec-task-planning.md](references/spec-task-planning.md) | +| Writing high-signal Trellis spec files | [references/spec-writing.md](references/spec-writing.md) | +| GitNexus and ABCoder MCP setup | [references/mcp-setup.md](references/mcp-setup.md) | + +## Operating Rules + +- Treat templates as starting points, not contracts. Delete, rename, split, or add spec files when the repository calls for it. +- Prefer source-backed rules over generic advice. Every important recommendation should point at a real file or repeated local pattern. +- Keep execution single-owner by default. Optional helper agents are an implementation detail, not a requirement or user-visible dependency. +- Do not write platform-specific instructions unless the target project already standardizes on that platform. +- Do not leave placeholder text, empty headings, or copied boilerplate in `.trellis/spec/`. + +## Done Criteria + +- `.trellis/spec/` describes the project as it exists now. +- Each relevant package or layer has practical coding guidance with real examples. +- Non-applicable template sections are removed. +- `index.md` files match the final spec file set. +- Any required setup or analysis assumptions are documented in the relevant spec or task notes. diff --git a/.claude/skills/trellis-spec-bootstrap/references/mcp-setup.md b/.claude/skills/trellis-spec-bootstrap/references/mcp-setup.md new file mode 100644 index 0000000..629fcbd --- /dev/null +++ b/.claude/skills/trellis-spec-bootstrap/references/mcp-setup.md @@ -0,0 +1,90 @@ +# MCP Setup + +GitNexus and ABCoder are recommended when bootstrapping Trellis specs because they expose architecture and AST context to the agent. They are tool choices, not platform requirements. Configure them through whatever MCP mechanism your agent host provides. + +## GitNexus + +GitNexus builds a code knowledge graph from the repository. Use it for module boundaries, execution flows, dependency relationships, blast radius, and graph queries. + +### Install and Index + +```bash +# Run from the repository root. +npx gitnexus analyze + +# Check index status. +npx gitnexus status + +# Re-index after code changes when the analysis is stale. +npx gitnexus analyze +``` + +The index is written to `.gitnexus/`. Keep embeddings only if the project already uses them; otherwise a normal index is enough for spec bootstrapping. + +### MCP Server Command + +Use this server command in the host's MCP configuration: + +```bash +npx -y gitnexus mcp +``` + +### Useful Tools + +| Tool | Purpose | +|------|---------| +| `gitnexus_query` | Find execution flows and functional areas by concept | +| `gitnexus_context` | Inspect callers, callees, references, and process participation for a symbol | +| `gitnexus_impact` | Understand blast radius before changing a symbol | +| `gitnexus_detect_changes` | Check changed symbols and affected flows before finishing | +| `gitnexus_cypher` | Run direct graph queries | +| `gitnexus_list_repos` | List indexed repositories | + +## ABCoder + +ABCoder parses code into UniAST and gives precise package, file, and node-level structure. Use it for signatures, type shapes, implementations, dependencies, and reverse references. + +### Install + +```bash +go install github.com/cloudwego/abcoder@latest +abcoder --help +``` + +### Parse Repositories + +```bash +abcoder parse /absolute/path/to/package \ + --lang typescript \ + --name package-name \ + --output ~/abcoder-asts +``` + +For monorepos, parse each package with a stable `--name` so task notes can reference the same repository names. + +### MCP Server Command + +Use this server command in the host's MCP configuration: + +```bash +abcoder mcp ~/abcoder-asts +``` + +### Useful Tools + +| Tool | Layer | Purpose | +|------|-------|---------| +| `list_repos` | 1 | List parsed repositories | +| `get_repo_structure` | 2 | Inspect packages and files | +| `get_package_structure` | 3 | Inspect nodes within a package | +| `get_file_structure` | 3 | Inspect functions, classes, types, and signatures in a file | +| `get_ast_node` | 4 | Retrieve code, dependencies, references, and implementations | + +## Verification + +After configuration, verify from the agent host that both MCP servers are visible. Then run one simple query against each server before starting the spec writing pass. + +```bash +ls .gitnexus/meta.json +ls ~/abcoder-asts/*.json +``` diff --git a/.claude/skills/trellis-spec-bootstrap/references/repository-analysis.md b/.claude/skills/trellis-spec-bootstrap/references/repository-analysis.md new file mode 100644 index 0000000..1309d29 --- /dev/null +++ b/.claude/skills/trellis-spec-bootstrap/references/repository-analysis.md @@ -0,0 +1,59 @@ +# Repository Analysis + +The goal is to discover the project's real architecture before writing rules. Do not start from generic spec templates and fill blanks. Start from the code, then let the spec structure follow. + +## Analysis Order + +1. Read the existing `.trellis/spec/` tree and note which files are templates, outdated, or already project-specific. +2. Inspect package manifests, build scripts, workspace config, and top-level documentation to identify packages and runtime layers. +3. Use GitNexus for execution flows, module clusters, dependency hubs, and impact-sensitive areas. +4. Use ABCoder or language-native tooling for exact signatures, types, class boundaries, and implementation examples. +5. Read representative source and test files directly before turning any finding into a spec rule. + +## What To Capture + +| Area | Questions | +|------|-----------| +| Package boundaries | What does each package own? What imports cross boundaries? | +| Runtime layers | Which code is CLI, backend, frontend, worker, shared library, test-only, or tooling? | +| Core abstractions | Which types, services, stores, commands, routes, or adapters define the system shape? | +| Data flow | Where does user input enter, how is it validated, and where does state persist? | +| Error handling | How are failures represented, logged, surfaced, and tested? | +| Configuration | Where do defaults, environment config, generated files, and templates live? | +| Tests | Which test styles are trusted examples for new work? | + +## GitNexus Usage + +Start broad, then inspect specific symbols: + +```text +gitnexus_query({query: "CLI command execution flow"}) +gitnexus_query({query: "template generation and migration"}) +gitnexus_context({name: "SymbolName"}) +gitnexus_cypher({query: "MATCH (n)-[r]->(m) RETURN n.name, type(r), m.name LIMIT 30"}) +``` + +Use GitNexus results to find important files and flows. Do not quote graph output as the final authority until you have checked the relevant source files. + +## ABCoder Usage + +Use ABCoder when the spec needs exact code shapes: + +```text +list_repos() +get_repo_structure({repo_name: "package-name"}) +get_file_structure({repo_name: "package-name", file_path: "src/example.ts"}) +get_ast_node({repo_name: "package-name", node_ids: [{mod_path: "...", pkg_path: "...", name: "SymbolName"}]}) +``` + +ABCoder is most valuable for documenting constructor patterns, function signatures, type contracts, and reference chains. + +## Analysis Notes + +Keep short notes while analyzing. The notes should include: + +- Package or layer name. +- Files that define the local pattern. +- Rules the spec should teach. +- Anti-patterns found in old code, comments, tests, or migration paths. +- Spec files that should be created, deleted, renamed, or merged. diff --git a/.claude/skills/trellis-spec-bootstrap/references/spec-task-planning.md b/.claude/skills/trellis-spec-bootstrap/references/spec-task-planning.md new file mode 100644 index 0000000..dca2687 --- /dev/null +++ b/.claude/skills/trellis-spec-bootstrap/references/spec-task-planning.md @@ -0,0 +1,61 @@ +# Spec Task Planning + +Use a single agent as the default execution model. The agent may create Trellis tasks for traceability, but the skill should not require a specific platform, CLI, or parallel worker model. + +## Decomposition + +Create spec work units around real ownership boundaries: + +- One package when a package has its own conventions. +- One layer when the same package has distinct frontend, backend, CLI, worker, or shared-library rules. +- One cross-cutting guide when a pattern spans packages and is not owned by one layer. + +Avoid artificial decomposition. A small library usually needs one focused spec pass, not several tasks. + +## Task Shape + +When a Trellis task is useful, write a concise PRD with these sections: + +```markdown +# Fill <package-or-layer> Trellis Specs + +## Goal +Write project-specific `.trellis/spec/` guidance for <scope>. + +## Scope +- Spec directory: +- Source directories to inspect: +- Tests to inspect: +- Out of scope: + +## Architecture Context +Summarize the concrete findings from repository analysis. + +## Files To Create Or Update +- `.trellis/spec/.../index.md` +- `.trellis/spec/.../<topic>.md` + +## Rules +- Adapt the spec file set to the real codebase. +- Use real source examples with file paths. +- Remove template-only sections that do not apply. +- Do not modify product source code unless the task explicitly asks for it. + +## Acceptance Criteria +- [ ] Specs contain concrete examples and anti-patterns from the repository. +- [ ] No placeholder text remains. +- [ ] Index files match the final spec files. +- [ ] Claims are backed by source files, tests, or project docs. +``` + +## Optional Helper Agents + +If the host supports subagents, helpers can inspect independent packages or run verification. They are optional. The main agent still owns integration and final quality. + +Helper tasks must have clear ownership: + +- Read-only research tasks may inspect any source needed for the assigned scope. +- Write tasks should own disjoint spec directories. +- Verification tasks should check placeholder removal, broken links, and consistency. + +Do not encode helper-agent names, vendor-specific commands, or platform-specific routing in the skill. Put only the required work and acceptance criteria in the task. diff --git a/.claude/skills/trellis-spec-bootstrap/references/spec-writing.md b/.claude/skills/trellis-spec-bootstrap/references/spec-writing.md new file mode 100644 index 0000000..6bc7dec --- /dev/null +++ b/.claude/skills/trellis-spec-bootstrap/references/spec-writing.md @@ -0,0 +1,70 @@ +# Spec Writing + +Trellis specs are coding guidance for future agents. They should explain how to work in this repository, not how a generic project might be organized. + +## Write From Evidence + +Each important rule should be backed by one of these: + +- A source file that demonstrates the preferred pattern. +- A test file that shows expected behavior. +- A project document that defines the convention. +- A repeated pattern across multiple files. + +Use short snippets only when they make the rule clearer. Prefer linking to the file path and naming the symbol or behavior. + +## File Structure + +Keep the spec tree aligned with the project: + +- Keep `index.md` as the navigation file for the spec directory. +- Split topics when developers would look for them independently. +- Merge topics when separate files would repeat the same rule. +- Delete template files that do not apply. +- Add new files for important local patterns the template missed. + +## Content Standards + +Good spec sections include: + +- When the rule applies. +- The local pattern to follow. +- The source or test files that prove the pattern. +- Common mistakes or anti-patterns. +- Verification commands or checks when they are specific and reliable. + +Avoid: + +- Placeholder prose. +- Generic framework advice. +- Tool instructions that only work in one agent host. +- Long copied code blocks. +- Rules based on a single accidental implementation detail. + +## Example Shape + +```markdown +## Command Handlers + +Command handlers should keep argument parsing, validation, and side effects separate. The local pattern is: + +- Parse CLI flags at the command boundary. +- Convert raw inputs into typed task options before invoking core logic. +- Keep filesystem writes in the command or service layer, not in template helpers. + +Reference files: +- `packages/cli/src/commands/example.ts` +- `packages/cli/test/commands/example.test.ts` + +Avoid passing raw `process.argv` or unvalidated config objects into shared helpers. +``` + +## Final Pass + +Before finishing: + +```bash +grep -R "To be filled\\|TODO: fill\\|placeholder" .trellis/spec +``` + +Also check links, index files, and whether any spec still describes a template rather than this repository. diff --git a/.claude/skills/trellis-update-spec/SKILL.md b/.claude/skills/trellis-update-spec/SKILL.md new file mode 100644 index 0000000..557bc4e --- /dev/null +++ b/.claude/skills/trellis-update-spec/SKILL.md @@ -0,0 +1,356 @@ +--- +name: trellis-update-spec +description: "Captures executable contracts and coding conventions into .trellis/spec/ documents. Use when learning something valuable from debugging, implementing, or discussion that should be preserved for future sessions." +--- + +# Update Code-Spec - Capture Executable Contracts + +When you learn something valuable (from debugging, implementing, or discussion), use this to update the relevant code-spec documents. + +**Timing**: After completing a task, fixing a bug, or discovering a new pattern + +--- + +## Code-Spec First Rule (CRITICAL) + +In this project, "spec" for implementation work means **code-spec**: +- Executable contracts (not principle-only text) +- Concrete signatures, payload fields, env keys, and boundary behavior +- Testable validation/error behavior + +If the change touches infra or cross-layer contracts, code-spec depth is mandatory. + +### Mandatory Triggers + +Apply code-spec depth when the change includes any of: +- New/changed command or API signature +- Cross-layer request/response contract change +- Database schema/migration change +- Infra integration (storage, queue, cache, secrets, env wiring) + +### Mandatory Output (7 Sections) + +For triggered tasks, include all sections below: +1. Scope / Trigger +2. Signatures (command/API/DB) +3. Contracts (request/response/env) +4. Validation & Error Matrix +5. Good/Base/Bad Cases +6. Tests Required (with assertion points) +7. Wrong vs Correct (at least one pair) + +--- + +## When to Update Code-Specs + +| Trigger | Example | Target Spec | +|---------|---------|-------------| +| **Implemented a feature** | Added a new integration or module | Relevant spec file | +| **Made a design decision** | Chose extensibility pattern over simplicity | Relevant spec + "Design Decisions" section | +| **Fixed a bug** | Found a subtle issue with error handling | Relevant spec (e.g., error-handling docs) | +| **Discovered a pattern** | Found a better way to structure code | Relevant spec file | +| **Hit a gotcha** | Learned that X must be done before Y | Relevant spec + "Common Mistakes" section | +| **Established a convention** | Team agreed on naming pattern | Quality guidelines | +| **New thinking trigger** | "Don't forget to check X before doing Y" | `guides/*.md` (as a checklist item) | + +**Key Insight**: Code-spec updates are NOT just for problems. Every feature implementation contains design decisions and contracts that future AI/developers need to execute safely. + +--- + +## Spec Structure Overview + +``` +.trellis/spec/ +├── <layer>/ # Per-layer coding standards (e.g., backend/, frontend/, api/) +│ ├── index.md # Overview and links +│ └── *.md # Topic-specific guidelines +└── guides/ # Thinking checklists (NOT coding specs!) + ├── index.md # Guide index + └── *.md # Topic-specific guides +``` + +### CRITICAL: Code-Spec vs Guide - Know the Difference + +| Type | Location | Purpose | Content Style | +|------|----------|---------|---------------| +| **Code-Spec** | `<layer>/*.md` | Tell AI "how to implement safely" | Signatures, contracts, matrices, cases, test points | +| **Guide** | `guides/*.md` | Help AI "what to think about" | Checklists, questions, pointers to specs | + +**Decision Rule**: Ask yourself: + +- "This is **how to write** the code" → Put in a spec layer directory +- "This is **what to consider** before writing" → Put in `guides/` + +**Example**: + +| Learning | Wrong Location | Correct Location | +|----------|----------------|------------------| +| "Use API X not API Y for this task" | ❌ `guides/` (too specific for a thinking guide) | ✅ Relevant spec file (concrete convention) | +| "Remember to check X when doing Y" | ❌ Spec file (too abstract for a spec) | ✅ `guides/` (thinking checklist) | + +**Guides should be short checklists that point to specs**, not duplicate the detailed rules. + +--- + +## Update Process + +### Step 1: Identify What You Learned + +Answer these questions: + +1. **What did you learn?** (Be specific) +2. **Why is it important?** (What problem does it prevent?) +3. **Where does it belong?** (Which spec file?) + +### Step 2: Classify the Update Type + +| Type | Description | Action | +|------|-------------|--------| +| **Design Decision** | Why we chose approach X over Y | Add to "Design Decisions" section | +| **Project Convention** | How we do X in this project | Add to relevant section with examples | +| **New Pattern** | A reusable approach discovered | Add to "Patterns" section | +| **Forbidden Pattern** | Something that causes problems | Add to "Anti-patterns" or "Don't" section | +| **Common Mistake** | Easy-to-make error | Add to "Common Mistakes" section | +| **Convention** | Agreed-upon standard | Add to relevant section | +| **Gotcha** | Non-obvious behavior | Add warning callout | + +### Step 3: Read the Target Code-Spec + +Before editing, read the current code-spec to: +- Understand existing structure +- Avoid duplicating content +- Find the right section for your update + +```bash +cat .trellis/spec/<category>/<file>.md +``` + +### Step 4: Make the Update + +Follow these principles: + +1. **Be Specific**: Include concrete examples, not just abstract rules +2. **Explain Why**: State the problem this prevents +3. **Show Contracts**: Add signatures, payload fields, and error behavior +4. **Show Code**: Add code snippets for key patterns +5. **Keep it Short**: One concept per section + +### Step 5: Update the Index (if needed) + +If you added a new section or the code-spec status changed, update the category's `index.md`. + +--- + +## Update Templates + +### Mandatory Template for Infra/Cross-Layer Work + +```markdown +## Scenario: <name> + +### 1. Scope / Trigger +- Trigger: <why this requires code-spec depth> + +### 2. Signatures +- Backend command/API/DB signature(s) + +### 3. Contracts +- Request fields (name, type, constraints) +- Response fields (name, type, constraints) +- Environment keys (required/optional) + +### 4. Validation & Error Matrix +- <condition> -> <error> + +### 5. Good/Base/Bad Cases +- Good: ... +- Base: ... +- Bad: ... + +### 6. Tests Required +- Unit/Integration/E2E with assertion points + +### 7. Wrong vs Correct +#### Wrong +... +#### Correct +... +``` + +### Adding a Design Decision + +```markdown +### Design Decision: [Decision Name] + +**Context**: What problem were we solving? + +**Options Considered**: +1. Option A - brief description +2. Option B - brief description + +**Decision**: We chose Option X because... + +**Example**: +\`\`\`typescript +// How it's implemented +code example +\`\`\` + +**Extensibility**: How to extend this in the future... +``` + +### Adding a Project Convention + +```markdown +### Convention: [Convention Name] + +**What**: Brief description of the convention. + +**Why**: Why we do it this way in this project. + +**Example**: +\`\`\`typescript +// How to follow this convention +code example +\`\`\` + +**Related**: Links to related conventions or specs. +``` + +### Adding a New Pattern + +```markdown +### Pattern Name + +**Problem**: What problem does this solve? + +**Solution**: Brief description of the approach. + +**Example**: +\`\`\` +// Good +code example + +// Bad +code example +\`\`\` + +**Why**: Explanation of why this works better. +``` + +### Adding a Forbidden Pattern + +```markdown +### Don't: Pattern Name + +**Problem**: +\`\`\` +// Don't do this +bad code example +\`\`\` + +**Why it's bad**: Explanation of the issue. + +**Instead**: +\`\`\` +// Do this instead +good code example +\`\`\` +``` + +### Adding a Common Mistake + +```markdown +### Common Mistake: Description + +**Symptom**: What goes wrong + +**Cause**: Why this happens + +**Fix**: How to correct it + +**Prevention**: How to avoid it in the future +``` + +### Adding a Gotcha + +```markdown +> **Warning**: Brief description of the non-obvious behavior. +> +> Details about when this happens and how to handle it. +``` + +--- + +## Interactive Mode + +If you're unsure what to update, answer these prompts: + +1. **What did you just finish?** + - [ ] Fixed a bug + - [ ] Implemented a feature + - [ ] Refactored code + - [ ] Had a discussion about approach + +2. **What did you learn or decide?** + - Design decision (why X over Y) + - Project convention (how we do X) + - Non-obvious behavior (gotcha) + - Better approach (pattern) + +3. **Would future AI/developers need to know this?** + - To understand how the code works → Yes, update spec + - To maintain or extend the feature → Yes, update spec + - To avoid repeating mistakes → Yes, update spec + - Purely one-off implementation detail → Maybe skip + +4. **Which area does it relate to?** + - [ ] Backend code + - [ ] Frontend code + - [ ] Cross-layer data flow + - [ ] Code organization/reuse + - [ ] Quality/testing + +--- + +## Quality Checklist + +Before finishing your code-spec update: + +- [ ] Is the content specific and actionable? +- [ ] Did you include a code example? +- [ ] Did you explain WHY, not just WHAT? +- [ ] Did you include executable signatures/contracts? +- [ ] Did you include validation and error matrix? +- [ ] Did you include Good/Base/Bad cases? +- [ ] Did you include required tests with assertion points? +- [ ] Is it in the right code-spec file? +- [ ] Does it duplicate existing content? +- [ ] Would a new team member understand it? + +--- + +## Relationship to Other Commands + +``` +Development Flow: + Learn something → /trellis:update-spec → Knowledge captured + ↑ ↓ + /trellis:break-loop ←──────────────────── Future sessions benefit + (deep bug analysis) +``` + +- `/trellis:break-loop` - Analyzes bugs deeply, often reveals spec updates needed +- `/trellis:update-spec` - Actually makes the updates +- `/trellis:finish-work` - Reminds you to check if specs need updates + +--- + +## Core Philosophy + +> **Code-specs are living documents. Every debugging session, every "aha moment" is an opportunity to make the implementation contract clearer.** + +The goal is **institutional memory**: +- What one person learns, everyone benefits from +- What AI learns in one session, persists to future sessions +- Mistakes become documented guardrails diff --git a/.codex/agents/trellis-check.toml b/.codex/agents/trellis-check.toml new file mode 100644 index 0000000..820f50c --- /dev/null +++ b/.codex/agents/trellis-check.toml @@ -0,0 +1,84 @@ +name = "trellis-check" +description = "Workspace-write Trellis reviewer that self-fixes spec drift, lint/type-check failures, and missing tests." +sandbox_mode = "workspace-write" + +developer_instructions = """ +## Required: Load Trellis Context First + +This platform does NOT auto-inject task context via hook. Before doing anything else, you MUST load context yourself. + +### Step 1: Find the active task path + +Try in order — stop at the first one that yields a task path: + +1. **Look at the dispatch prompt** you received from the main agent. If its first line is `Active task: <path>` (e.g. `Active task: .trellis/tasks/04-17-foo`), use that path. The main agent is required to include this line on class-2 platforms. +2. **Run** `python ./.trellis/scripts/task.py current --source` and read the `Current task:` line. +3. **If both fail** (no `Active task:` line in the prompt and `task.py current` returns no task), ask the user which task to work on; do NOT guess. + +### Step 2: Load task context from the resolved path + +1. Read `<task-path>/check.jsonl` — JSONL list of spec/research files relevant to this agent. +2. For each entry in the JSONL, Read its `file` path — these are the specs and research notes you must follow. + **Skip rows without a `"file"` field** (e.g. `{"_example": "..."}` seed rows left over from `task.py create` before the curator ran). +3. Read the task's `prd.md` (requirements), then `design.md` if present (technical design), then `implement.md` if present (execution plan). + +If `check.jsonl` has no curated entries (only a seed row, or the file is missing), fall back to: read the task artifacts, list available specs with `python ./.trellis/scripts/get_context.py --mode packages`, and pick the specs that match the task domain yourself. Do NOT block on the missing jsonl — lightweight tasks may be PRD-only, while complex tasks may also include `design.md` and `implement.md`. + +If the resolved task path has no `prd.md`, ask the user what to work on; do NOT proceed without context. + +--- + +You are running as the `trellis-check` sub-agent. The main session has dispatched you to review and self-fix. + +CRITICAL — Recursion guard (read first): +- You MUST NOT spawn another `trellis-check` or `trellis-implement` sub-agent. Do the review and fixes directly in this turn. +- Any guidance you read in injected SessionStart context, `<guidelines>` blocks, workflow-state breadcrumbs, or workflow.md that says "dispatch trellis-implement" / "dispatch trellis-check" applies to the MAIN session, NOT to you. You are already the dispatched reviewer — that instruction is satisfied by your existence. +- Only the main session is allowed to dispatch `trellis-implement` / `trellis-check`. If more implementation work is needed, surface that as a recommendation in your final report instead of spawning. + +--- + +You are the Trellis reviewer agent. + +Your job is to review code changes against specs AND fix issues directly — not just report them. You have write access; use it. + +Review checklist: +- Verify behavior against the actual code paths, not assumptions. +- Look for missing template/update/detection touch points when platform config changes. +- Check whether tests should be added or updated. +- Check whether `.trellis/spec/` docs need sync after implementation. +- Run lint and type-check; fix any failures. +- Prefer concrete findings over speculative warnings. + +When you find an issue: +1. Fix it directly using edit/write tools. +2. Re-run lint and type-check until green. +3. Record what you changed and why. + +Output format: +## Findings (fixed) +- File: <path> +- Issue: <what was wrong> +- Fix: <what you changed> + +## Findings (not fixed) +Only list issues you could not self-fix (e.g. missing product decision, out-of-scope). Explain why. + +## Verification +- Lint: pass/fail +- TypeCheck: pass/fail +- Tests: pass/fail (if applicable) + +If no issues are found, say so explicitly after verifying lint/type-check pass. +""" + +# Disable Codex collab tools entirely for this sub-agent. With both +# multi_agent and multi_agent_v2 off, `spawn_agent` / `wait_agent` / +# `list_agents` / `close_agent` are not registered in the sub-agent's tool +# list at all — the model literally cannot call them. This is the structural +# fix for the wait_agent self-deadlock when the parent inherits its +# transcript via Codex's default `fork_turns="all"` (#240 follow-up, #241). +[features] +multi_agent = false + +[features.multi_agent_v2] +enabled = false diff --git a/.codex/agents/trellis-implement.toml b/.codex/agents/trellis-implement.toml new file mode 100644 index 0000000..e4b753b --- /dev/null +++ b/.codex/agents/trellis-implement.toml @@ -0,0 +1,65 @@ +name = "trellis-implement" +description = "Workspace-write Trellis implementer that follows specs and keeps generated templates in sync." +sandbox_mode = "workspace-write" + +developer_instructions = """ +## Required: Load Trellis Context First + +This platform does NOT auto-inject task context via hook. Before doing anything else, you MUST load context yourself. + +### Step 1: Find the active task path + +Try in order — stop at the first one that yields a task path: + +1. **Look at the dispatch prompt** you received from the main agent. If its first line is `Active task: <path>` (e.g. `Active task: .trellis/tasks/04-17-foo`), use that path. The main agent is required to include this line on class-2 platforms. +2. **Run** `python ./.trellis/scripts/task.py current --source` and read the `Current task:` line. +3. **If both fail** (no `Active task:` line in the prompt and `task.py current` returns no task), ask the user which task to work on; do NOT guess. + +### Step 2: Load task context from the resolved path + +1. Read `<task-path>/implement.jsonl` — JSONL list of spec/research files relevant to this agent. +2. For each entry in the JSONL, Read its `file` path — these are the specs and research notes you must follow. + **Skip rows without a `"file"` field** (e.g. `{"_example": "..."}` seed rows left over from `task.py create` before the curator ran). +3. Read the task's `prd.md` (requirements), then `design.md` if present (technical design), then `implement.md` if present (execution plan). + +If `implement.jsonl` has no curated entries (only a seed row, or the file is missing), fall back to: read the task artifacts, list available specs with `python ./.trellis/scripts/get_context.py --mode packages`, and pick the specs that match the task domain yourself. Do NOT block on the missing jsonl — lightweight tasks may be PRD-only, while complex tasks may also include `design.md` and `implement.md`. + +If the resolved task path has no `prd.md`, ask the user what to work on; do NOT proceed without context. + +--- + +You are running as the `trellis-implement` sub-agent. The main session has dispatched you to do the work. + +CRITICAL — Recursion guard (read first): +- You MUST NOT spawn another `trellis-implement` or `trellis-check` sub-agent. Do the implementation work directly in this turn. +- Any guidance you read in injected SessionStart context, `<guidelines>` blocks, workflow-state breadcrumbs, or workflow.md that says "dispatch trellis-implement" / "dispatch trellis-check" applies to the MAIN session, NOT to you. You are already the dispatched implementer — that instruction is satisfied by your existence. +- Only the main session is allowed to dispatch `trellis-implement` / `trellis-check`. If more parallel work is needed, surface that as a recommendation in your final report instead of spawning. + +--- + +You are the Trellis implementer agent. + +Rules: +- Read before write. Follow `.trellis/spec/` guidance relevant to the task. +- Keep changes focused on the requested scope. +- When touching platform registries or template lists, search first so you do not miss mirrored update paths. +- If you modify `.trellis/scripts/`, keep `packages/cli/src/templates/trellis/scripts/` in sync. +- Do not make destructive git changes unless explicitly asked. + +Before finishing, summarize: +- Files changed +- Tests/checks run +- Remaining risks or follow-ups +""" + +# Disable Codex collab tools entirely for this sub-agent. With both +# multi_agent and multi_agent_v2 off, `spawn_agent` / `wait_agent` / +# `list_agents` / `close_agent` are not registered in the sub-agent's tool +# list at all — the model literally cannot call them. This is the structural +# fix for the wait_agent self-deadlock when the parent inherits its +# transcript via Codex's default `fork_turns="all"` (#240 follow-up, #241). +[features] +multi_agent = false + +[features.multi_agent_v2] +enabled = false diff --git a/.codex/agents/trellis-research.toml b/.codex/agents/trellis-research.toml new file mode 100644 index 0000000..69227da --- /dev/null +++ b/.codex/agents/trellis-research.toml @@ -0,0 +1,73 @@ +name = "trellis-research" +description = "Trellis researcher for specs, code patterns, and affected files. Writes findings into {TASK_DIR}/research/ — read-only elsewhere." +sandbox_mode = "workspace-write" + +developer_instructions = """ +You are the Trellis researcher agent. + +## Core principle + +Conversations get compacted; files don't. Every research topic MUST be +persisted to `{TASK_DIR}/research/<topic>.md`. Returning findings only +through the chat reply is a failure. + +## Workflow + +1. Run `python ./.trellis/scripts/task.py current --source` to get the + active task path and source. If no active task is set, ask the user + where to write output; do not guess. +2. Run `mkdir -p <TASK_DIR>/research` to ensure the directory exists. +3. Read `.trellis/workflow.md`, relevant `.trellis/spec/` files, and + target code before forming an opinion. +4. For each research topic, write `<TASK_DIR>/research/<slug>.md` with: + - Query, scope, date + - Files found (path + one-line description) + - Code patterns (cite file:line) + - External references (docs, versions) + - Related specs + - Caveats / not-found notes +5. Reply with only: list of files written, one-line summary per file, + any critical caveats. Do not paste full research into the reply. + +## Scope limits + +Write allowed ONLY in `{TASK_DIR}/research/`. + +Write forbidden everywhere else: +- Code files (`src/`, `lib/`, …) +- Spec files (`.trellis/spec/`) — use `update-spec` skill instead +- `.trellis/scripts/`, `.trellis/workflow.md`, platform config +- Other task directories +- Any git operation + +If the user asks you to edit code, decline and tell them to spawn the +`implement` agent. + +## Output format for each research file + +``` +# Research: <topic> + +- Query: ... +- Scope: internal / external / mixed +- Date: YYYY-MM-DD + +## Findings +... + +## Caveats / Not Found +... +``` +""" + +# Disable Codex collab tools entirely for this sub-agent. With both +# multi_agent and multi_agent_v2 off, `spawn_agent` / `wait_agent` / +# `list_agents` / `close_agent` are not registered in the sub-agent's tool +# list at all — the model literally cannot call them. This is the structural +# fix for the wait_agent self-deadlock when the parent inherits its +# transcript via Codex's default `fork_turns="all"` (#240 follow-up, #241). +[features] +multi_agent = false + +[features.multi_agent_v2] +enabled = false diff --git a/.codex/hooks.json b/.codex/hooks.json new file mode 100644 index 0000000..6432832 --- /dev/null +++ b/.codex/hooks.json @@ -0,0 +1,15 @@ +{ + "hooks": { + "UserPromptSubmit": [ + { + "hooks": [ + { + "type": "command", + "command": "python -X utf8 .codex/hooks/inject-workflow-state.py", + "timeout": 15 + } + ] + } + ] + } +} diff --git a/.codex/hooks/inject-workflow-state.py b/.codex/hooks/inject-workflow-state.py new file mode 100644 index 0000000..fda556b --- /dev/null +++ b/.codex/hooks/inject-workflow-state.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Trellis per-turn breadcrumb hook (UserPromptSubmit / BeforeAgent equivalent). + +Runs on every user prompt. Resolves the active task through Trellis' +session-aware active task resolver and emits a short <workflow-state> +block reminding the main AI what task is active and its expected flow. + +The emitted ``hookEventName`` field is platform-aware: most hosts expect +``UserPromptSubmit`` (Claude Code naming, also accepted by Cursor / Qoder / +CodeBuddy / Droid / Codex / Copilot wiring), but Gemini CLI 0.40.x renamed +its per-turn event to ``BeforeAgent`` and its schema validator rejects the +legacy name. ``_detect_platform`` picks the right value at runtime. +Breadcrumb text is pulled exclusively from workflow.md +[workflow-state:STATUS] tag blocks — workflow.md is the single source of +truth. There are no fallback dicts in this script: when workflow.md is +missing or a tag is absent, the breadcrumb degrades to a generic +"Refer to workflow.md for current step." line so users see (and fix) +the broken state instead of the hook silently masking it. + +Shared across all hook-capable platforms (Claude, Cursor, Codex, Qoder, +CodeBuddy, Droid, Gemini, Copilot). Kiro is not wired (no per-turn +hook entry point). Written to each platform's hooks directory via +writeSharedHooks() at init time. + +Silent exit 0 cases (no output): + - No .trellis/ directory found (not a Trellis project) + - task.json malformed or missing status +""" +from __future__ import annotations + +import json +import os +import re +import sys +from pathlib import Path + +# Force UTF-8 on stdin/stdout/stderr on Windows. Default codepage there is +# cp936 / cp1252 / etc. — non-ASCII content (Chinese task names, prd snippets) +# both in stdin (hook payload from host CLI) and stdout (our emitted blocks) +# raises UnicodeDecodeError / UnicodeEncodeError. Equivalent to `python -X utf8` +# but applied per-stream so we don't depend on host CLI's command wiring. +if sys.platform.startswith("win"): + import io as _io + for _stream_name in ("stdin", "stdout", "stderr"): + _stream = getattr(sys, _stream_name, None) + if _stream is None: + continue + if hasattr(_stream, "reconfigure"): + try: + _stream.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] + except Exception: + pass + elif hasattr(_stream, "detach"): + try: + setattr(sys, _stream_name, _io.TextIOWrapper(_stream.detach(), encoding="utf-8", errors="replace")) + except Exception: + pass +from typing import Optional + + +# Bootstrap notice for Codex while the session has no active task. Codex does not +# get the full SessionStart overview; this short reminder points the main session +# at the start skill once and leaves the per-turn state block compact. +CODEX_NO_TASK_BOOTSTRAP_NOTICE = """<trellis-bootstrap> +If you have not already loaded Trellis context this session, read the `trellis-start` skill once. +</trellis-bootstrap>""" + + +# --------------------------------------------------------------------------- +# CWD-robust Trellis root discovery (fixes hook-path-robustness for this hook) +# --------------------------------------------------------------------------- + +def find_trellis_root(start: Path) -> Optional[Path]: + """Walk up from start to find directory containing .trellis/. + + Handles CWD drift: subdirectory launches, monorepo packages, etc. + Returns None if no .trellis/ found (silent no-op). + """ + cur = start.resolve() + while cur != cur.parent: + if (cur / ".trellis").is_dir(): + return cur + cur = cur.parent + return None + + +# --------------------------------------------------------------------------- +# Active task discovery +# --------------------------------------------------------------------------- + +def _detect_platform(input_data: dict) -> str | None: + if isinstance(input_data.get("cursor_version"), str): + return "cursor" + env_map = { + "CLAUDE_PROJECT_DIR": "claude", + "CURSOR_PROJECT_DIR": "cursor", + "CODEBUDDY_PROJECT_DIR": "codebuddy", + "FACTORY_PROJECT_DIR": "droid", + "GEMINI_PROJECT_DIR": "gemini", + "QODER_PROJECT_DIR": "qoder", + "KIRO_PROJECT_DIR": "kiro", + "COPILOT_PROJECT_DIR": "copilot", + } + for env_name, platform in env_map.items(): + if os.environ.get(env_name): + return platform + script_parts = set(Path(sys.argv[0]).parts) + if ".claude" in script_parts: + return "claude" + if ".cursor" in script_parts: + return "cursor" + if ".codex" in script_parts: + return "codex" + if ".gemini" in script_parts: + return "gemini" + if ".qoder" in script_parts: + return "qoder" + if ".codebuddy" in script_parts: + return "codebuddy" + if ".factory" in script_parts: + return "droid" + if ".kiro" in script_parts: + return "kiro" + return None + + +def _resolve_active_task(root: Path, input_data: dict): + scripts_dir = root / ".trellis" / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + from common.active_task import resolve_active_task # type: ignore[import-not-found] + + return resolve_active_task(root, input_data, platform=_detect_platform(input_data)) + + +def get_active_task(root: Path, input_data: dict) -> Optional[tuple[str, str, str]]: + """Return (task_id, status, source) from the current active task.""" + active = _resolve_active_task(root, input_data) + if not active.task_path: + return None + + task_dir = Path(active.task_path) + if not task_dir.is_absolute(): + task_dir = root / task_dir + if active.stale: + return task_dir.name, f"stale_{active.source_type}", active.source + + task_json = task_dir / "task.json" + if not task_json.is_file(): + return None + try: + data = json.loads(task_json.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError): + return None + + task_id = data.get("id") or task_dir.name + status = data.get("status", "") + if not isinstance(status, str) or not status: + return None + return task_id, status, active.source + + +# --------------------------------------------------------------------------- +# Breadcrumb loading: parse workflow.md, fall back to hardcoded defaults +# --------------------------------------------------------------------------- + +# Supports STATUS values with letters, digits, underscores, hyphens +# (so "in-review" / "blocked-by-team" work alongside "in_progress"). +_TAG_RE = re.compile( + r"\[workflow-state:([A-Za-z0-9_-]+)\]\s*\n(.*?)\n\s*\[/workflow-state:\1\]", + re.DOTALL, +) + +def load_breadcrumbs(root: Path) -> dict[str, str]: + """Parse workflow.md for [workflow-state:STATUS] blocks. + + Returns {status: body_text}. workflow.md is the single source of + truth — there are no fallback dicts in this script. Missing tags + (or a missing/unreadable workflow.md) fall back to a generic line + in build_breadcrumb so users see the broken state and fix + workflow.md, rather than the hook silently masking the issue. + """ + workflow = root / ".trellis" / "workflow.md" + if not workflow.is_file(): + return {} + try: + content = workflow.read_text(encoding="utf-8") + except OSError: + return {} + + result: dict[str, str] = {} + for match in _TAG_RE.finditer(content): + status = match.group(1) + body = match.group(2).strip() + if body: + result[status] = body + return result + + +def _read_trellis_config(root: Path) -> dict: + """Load .trellis/config.yaml via the bundled trellis_config helper. + + The helper lives in .trellis/scripts/common; the hook lives outside the + scripts tree, so we extend sys.path before importing. + """ + scripts_dir = root / ".trellis" / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + try: + from common.trellis_config import read_trellis_config # type: ignore[import-not-found] + except Exception: + return {} + try: + return read_trellis_config(root) + except Exception: + return {} + + +def _codex_mode_banner(config: dict) -> str: + """Emit a `<codex-mode>` banner for the additionalContext payload. + + Reads `codex.dispatch_mode` from .trellis/config.yaml; defaults to + `inline` when missing or invalid because Codex sub-agents run with + `fork_turns="none"` isolation and can't inherit the parent session's + task context. The banner makes the active mode explicit to Codex AI + per turn, complementing the workflow-state body which is per-status. + Mode tells AI which dispatch protocol to follow; workflow-state tells + AI what step it's at. + """ + mode = "inline" + if isinstance(config, dict): + codex_cfg = config.get("codex") + if isinstance(codex_cfg, dict): + cfg_mode = codex_cfg.get("dispatch_mode") + if cfg_mode in ("inline", "sub-agent"): + mode = cfg_mode + if mode == "sub-agent": + meaning = ( + "sub-agent: implement/check work defaults to Trellis sub-agents; " + "the main session still coordinates, clarifies, updates specs, commits, and finishes." + ) + else: + meaning = ( + "inline: the main session implements/checks directly; " + "do not dispatch implement/check sub-agents." + ) + return f"<codex-mode>{meaning}</codex-mode>" + + +def resolve_breadcrumb_key( + status: str, platform: str | None, config: dict +) -> str: + """Pick the breadcrumb tag key based on Codex dispatch_mode. + + Codex defaults to ``inline`` because sub-agents run with ``fork_turns="none"`` + isolation and can't inherit the parent session's task context. Users can + opt into ``codex.dispatch_mode: sub-agent`` in ``.trellis/config.yaml`` + to use the parallel ``<status>-inline`` tag → ``<status>`` flip. Invalid + or missing values fall back to inline. + + Non-codex platforms return the plain status unchanged. + """ + if platform == "codex": + mode = "inline" + if isinstance(config, dict): + codex_cfg = config.get("codex") + if isinstance(codex_cfg, dict): + cfg_mode = codex_cfg.get("dispatch_mode") + if cfg_mode in ("inline", "sub-agent"): + mode = cfg_mode + return f"{status}-inline" if mode == "inline" else status + return status + + +def build_breadcrumb( + task_id: Optional[str], + status: str, + templates: dict[str, str], + source: str | None = None, + breadcrumb_key: str | None = None, +) -> str: + """Build the <workflow-state>...</workflow-state> block. + + - Known status (tag present in workflow.md) → detailed template body + - Unknown status (no tag, or workflow.md missing) → generic + "Refer to workflow.md for current step." line + - `no_task` pseudo-status (task_id is None) → header omits task info + """ + lookup_key = breadcrumb_key or status + body = templates.get(lookup_key) + if body is None and lookup_key != status: + body = templates.get(status) + if body is None: + body = "Refer to workflow.md for current step." + header = f"Status: {status}" if task_id is None else f"Task: {task_id} ({status})" + return f"<workflow-state>\n{header}\n{body}\n</workflow-state>" + + +# --------------------------------------------------------------------------- +# Entry +# --------------------------------------------------------------------------- + +def main() -> int: + if os.environ.get("TRELLIS_HOOKS") == "0" or os.environ.get("TRELLIS_DISABLE_HOOKS") == "1": + return 0 + + try: + data = json.load(sys.stdin) + except (json.JSONDecodeError, ValueError): + data = {} + + cwd_str = data.get("cwd") or os.getcwd() + cwd = Path(cwd_str) + + root = find_trellis_root(cwd) + if root is None: + return 0 # not a Trellis project + + templates = load_breadcrumbs(root) + platform = _detect_platform(data) + config = _read_trellis_config(root) + task = get_active_task(root, data) + if task is None: + # No active task — still emit a breadcrumb nudging AI toward + # trellis-brainstorm + task.py create when user describes real work. + no_task_key = resolve_breadcrumb_key("no_task", platform, config) + breadcrumb = build_breadcrumb( + None, "no_task", templates, breadcrumb_key=no_task_key + ) + else: + task_id, status, source = task + status_key = resolve_breadcrumb_key(status, platform, config) + source_for_breadcrumb = None if platform == "codex" else source + breadcrumb = build_breadcrumb( + task_id, status, templates, source_for_breadcrumb, breadcrumb_key=status_key + ) + if platform == "codex": + parts: list[str] = [] + if task is None: + parts.append(CODEX_NO_TASK_BOOTSTRAP_NOTICE) + parts.append(_codex_mode_banner(config)) + parts.append(breadcrumb) + breadcrumb = "\n\n".join(parts) + + # Gemini CLI 0.40.x rejects "UserPromptSubmit" — its per-turn event is + # named "BeforeAgent". Other platforms (Claude/Cursor/Qoder/CodeBuddy/ + # Droid/Codex/Copilot) accept the original Claude-style name. + hook_event_name = ( + "BeforeAgent" if platform == "gemini" else "UserPromptSubmit" + ) + + output = { + "hookSpecificOutput": { + "hookEventName": hook_event_name, + "additionalContext": breadcrumb, + } + } + print(json.dumps(output)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.codex/hooks/session-start.py b/.codex/hooks/session-start.py new file mode 100644 index 0000000..4f56599 --- /dev/null +++ b/.codex/hooks/session-start.py @@ -0,0 +1,545 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Codex Session Start Hook - Inject Trellis context into Codex sessions. + +Output format follows Codex hook protocol: + stdout JSON → { hookSpecificOutput: { hookEventName: "SessionStart", additionalContext: "..." } } +""" + +from __future__ import annotations + +import json +import os +import re +import subprocess +import sys +import warnings +from io import StringIO +from pathlib import Path + +# Force UTF-8 on stdin/stdout/stderr on Windows. Default codepage there is +# cp936 / cp1252 / etc. — non-ASCII content (Chinese task names, prd snippets) +# both in stdin (hook payload from host CLI) and stdout (our emitted blocks) +# raises UnicodeDecodeError / UnicodeEncodeError. Equivalent to `python -X utf8` +# but applied per-stream so we don't depend on host CLI's command wiring. +if sys.platform.startswith("win"): + import io as _io + for _stream_name in ("stdin", "stdout", "stderr"): + _stream = getattr(sys, _stream_name, None) + if _stream is None: + continue + if hasattr(_stream, "reconfigure"): + try: + _stream.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] + except Exception: + pass + elif hasattr(_stream, "detach"): + try: + setattr(sys, _stream_name, _io.TextIOWrapper(_stream.detach(), encoding="utf-8", errors="replace")) + except Exception: + pass + + +def _normalize_windows_shell_path(path_str: str) -> str: + """Normalize Unix-style shell paths to real Windows paths. + + On Windows, shells like Git Bash / MSYS2 / Cygwin may report paths like + `/d/Users/...` or `/cygdrive/d/Users/...`. `Path.resolve()` will misinterpret + these as `D:/d/Users...` on drive D: (or similar), breaking repo root + detection. + + This function is intentionally conservative: it only rewrites patterns that + unambiguously represent a drive letter mount. + """ + if not isinstance(path_str, str) or not path_str: + return path_str + + # Only relevant on Windows; keep other platforms untouched. + if not sys.platform.startswith("win"): + return path_str + + p = path_str.strip() + + # Already a Windows drive path (C:\... or C:/...) + if re.match(r"^[A-Za-z]:[\/]", p): + return p + + # MSYS/Git-Bash style: /c/Users/... or /d/Work/... + m = re.match(r"^/([A-Za-z])/(.*)", p) + if m: + drive, rest = m.group(1).upper(), m.group(2) + rest = rest.replace('/', '\\') + return f"{drive}:\\{rest}" + + # Cygwin style: /cygdrive/c/Users/... + m = re.match(r"^/cygdrive/([A-Za-z])/(.*)", p) + if m: + drive, rest = m.group(1).upper(), m.group(2) + rest = rest.replace('/', '\\') + return f"{drive}:\\{rest}" + + # WSL mounted drive (sometimes leaked into env): /mnt/c/Users/... + m = re.match(r"^/mnt/([A-Za-z])/(.*)", p) + if m: + drive, rest = m.group(1).upper(), m.group(2) + rest = rest.replace('/', '\\') + return f"{drive}:\\{rest}" + + return path_str + + +warnings.filterwarnings("ignore") + +FIRST_REPLY_NOTICE = """<first-reply-notice> +On the first visible assistant reply in this session, begin with exactly one short Chinese sentence: +Trellis SessionStart 已注入：workflow、当前任务状态、开发者身份、git 状态、active tasks、spec 索引已加载。 +Then continue directly with the user's request. This notice is one-shot: do not repeat it after the first assistant reply in the same session. +</first-reply-notice>""" + +def should_skip_injection() -> bool: + if os.environ.get("TRELLIS_HOOKS") == "0": + return True + if os.environ.get("TRELLIS_DISABLE_HOOKS") == "1": + return True + return os.environ.get("CODEX_NON_INTERACTIVE") == "1" + + +def configure_project_encoding(project_dir: Path) -> None: + """Reuse Trellis' shared Windows stdio encoding helper before JSON output.""" + scripts_dir = project_dir / ".trellis" / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + + try: + from common import configure_encoding # type: ignore[import-not-found] + + configure_encoding() + except Exception: + pass + + +def _has_curated_jsonl_entry(jsonl_path: Path) -> bool: + """Return True iff jsonl has at least one row with a ``file`` field. + + A freshly seeded jsonl only contains a ``{"_example": ...}`` row (no + ``file`` key) — that is NOT "ready". Readiness requires at least one + curated entry. Matches the contract used by ``inject-subagent-context.py``. + """ + try: + for line in jsonl_path.read_text(encoding="utf-8").splitlines(): + line = line.strip() + if not line: + continue + try: + row = json.loads(line) + except json.JSONDecodeError: + continue + if isinstance(row, dict) and row.get("file"): + return True + except (OSError, UnicodeDecodeError): + return False + return False + + +def read_file(path: Path, fallback: str = "") -> str: + try: + return path.read_text(encoding="utf-8") + except (FileNotFoundError, PermissionError): + return fallback + + +def _resolve_context_key(project_dir: Path, hook_input: dict) -> str | None: + scripts_dir = project_dir / ".trellis" / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + try: + from common.active_task import resolve_context_key # type: ignore[import-not-found] + except Exception: + return None + return resolve_context_key(hook_input, platform="codex") + + +def _resolve_active_task(trellis_dir: Path, hook_input: dict): + scripts_dir = trellis_dir / "scripts" + if str(scripts_dir) not in sys.path: + sys.path.insert(0, str(scripts_dir)) + from common.active_task import resolve_active_task # type: ignore[import-not-found] + + return resolve_active_task(trellis_dir.parent, hook_input, platform="codex") + + +def run_script(script_path: Path, context_key: str | None = None) -> str: + try: + env = os.environ.copy() + env["PYTHONIOENCODING"] = "utf-8" + if context_key: + env["TRELLIS_CONTEXT_ID"] = context_key + cmd = [sys.executable, "-W", "ignore", str(script_path)] + result = subprocess.run( + cmd, + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=5, + cwd=str(script_path.parent.parent.parent), + env=env, + ) + return result.stdout if result.returncode == 0 else "No context available" + except (subprocess.TimeoutExpired, FileNotFoundError, PermissionError): + return "No context available" + + +def _normalize_task_ref(task_ref: str) -> str: + normalized = task_ref.strip() + if not normalized: + return "" + + path_obj = Path(normalized) + if path_obj.is_absolute(): + return str(path_obj) + + normalized = normalized.replace("\\", "/") + while normalized.startswith("./"): + normalized = normalized[2:] + + if normalized.startswith("tasks/"): + return f".trellis/{normalized}" + + return normalized + + +def _resolve_task_dir(trellis_dir: Path, task_ref: str) -> Path: + normalized = _normalize_task_ref(task_ref) + path_obj = Path(normalized) + if path_obj.is_absolute(): + return path_obj + if normalized.startswith(".trellis/"): + return trellis_dir.parent / path_obj + return trellis_dir / "tasks" / path_obj + + +def _get_task_status(trellis_dir: Path, hook_input: dict) -> str: + active = _resolve_active_task(trellis_dir, hook_input) + if not active.task_path: + return ( + "Status: NO ACTIVE TASK\n" + "Next: Classify the current turn and ask for task-creation consent " + "before creating any Trellis task." + ) + + task_ref = active.task_path + task_dir = _resolve_task_dir(trellis_dir, task_ref) + if active.stale or not task_dir.is_dir(): + return ( + f"Status: STALE POINTER\nTask: {task_ref}\n" + "Next: Task directory not found. Run: python ./.trellis/scripts/task.py finish" + ) + + task_json_path = task_dir / "task.json" + task_data: dict = {} + if task_json_path.is_file(): + try: + task_data = json.loads(task_json_path.read_text(encoding="utf-8")) + except (json.JSONDecodeError, PermissionError): + pass + + task_title = task_data.get("title", task_ref) + task_status = task_data.get("status", "unknown") + + if task_status == "completed": + return ( + f"Status: COMPLETED\nTask: {task_title}\n" + f"Next: Archive with `python ./.trellis/scripts/task.py archive {task_dir.name}` " + "or start a new task." + ) + + has_prd = (task_dir / "prd.md").is_file() + has_design = (task_dir / "design.md").is_file() + has_implement = (task_dir / "implement.md").is_file() + present = [ + name + for name in ("prd.md", "design.md", "implement.md", "implement.jsonl", "check.jsonl") + if (task_dir / name).is_file() + ] + present_line = ", ".join(present) if present else "none" + + if not has_prd: + return ( + f"Status: PLANNING\nTask: {task_title}\nPresent: {present_line}\n" + "Next: Load trellis-brainstorm and write prd.md. Stay in planning." + ) + + if task_status == "planning": + if has_design and has_implement: + next_action = "Review planning artifacts with the user before `task.py start`." + else: + next_action = ( + "Lightweight task can ask for start review with PRD-only; " + "complex task must add design.md and implement.md before `task.py start`." + ) + return ( + f"Status: PLANNING\nTask: {task_title}\nPresent: {present_line}\n" + f"Next: {next_action}" + ) + + return ( + f"Status: {task_status.upper()}\nTask: {task_title}\nPresent: {present_line}\n" + "Next: Follow the matching per-turn workflow-state. Context order is jsonl entries, " + "prd.md, design.md if present, implement.md if present." + ) + + +def _run_git(repo_root: Path, args: list[str]) -> str: + try: + result = subprocess.run( + ["git", *args], + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=3, + cwd=str(repo_root), + ) + except (subprocess.TimeoutExpired, FileNotFoundError, PermissionError): + return "" + if result.returncode != 0: + return "" + return result.stdout.strip() + + +def _format_git_state(repo_root: Path) -> str: + branch = _run_git(repo_root, ["branch", "--show-current"]) or "(detached)" + dirty_lines = [ + line for line in _run_git(repo_root, ["status", "--porcelain"]).splitlines() + if line.strip() + ] + dirty_text = "clean" if not dirty_lines else f"dirty {len(dirty_lines)} paths" + return f"Git: branch {branch}; {dirty_text}." + + +def _repo_relative(repo_root: Path, path: Path) -> str: + try: + return path.relative_to(repo_root).as_posix() + except ValueError: + return str(path) + + +def _collect_spec_index_paths(trellis_dir: Path) -> list[str]: + paths: list[str] = [] + guides_index = trellis_dir / "spec" / "guides" / "index.md" + if guides_index.is_file(): + paths.append(".trellis/spec/guides/index.md") + + spec_dir = trellis_dir / "spec" + if not spec_dir.is_dir(): + return paths + + for sub in sorted(spec_dir.iterdir()): + if not sub.is_dir() or sub.name.startswith(".") or sub.name == "guides": + continue + index_file = sub / "index.md" + if index_file.is_file(): + paths.append(f".trellis/spec/{sub.name}/index.md") + continue + for nested in sorted(sub.iterdir()): + if not nested.is_dir(): + continue + nested_index = nested / "index.md" + if nested_index.is_file(): + paths.append(f".trellis/spec/{sub.name}/{nested.name}/index.md") + + return paths + + +def _build_compact_current_state( + trellis_dir: Path, + hook_input: dict, + spec_index_paths: list[str], +) -> str: + repo_root = trellis_dir.parent + lines: list[str] = [] + + try: + from common.paths import get_active_journal_file, get_developer, get_tasks_dir, count_lines # type: ignore[import-not-found] + from common.tasks import iter_active_tasks # type: ignore[import-not-found] + except Exception: + get_active_journal_file = None # type: ignore[assignment] + get_developer = None # type: ignore[assignment] + get_tasks_dir = None # type: ignore[assignment] + count_lines = None # type: ignore[assignment] + iter_active_tasks = None # type: ignore[assignment] + + developer = get_developer(repo_root) if get_developer else None + lines.append(f"Developer: {developer or '(not initialized)'}") + lines.append(_format_git_state(repo_root)) + + active = _resolve_active_task(trellis_dir, hook_input) + if active.task_path: + task_dir = _resolve_task_dir(trellis_dir, active.task_path) + status = "unknown" + task_json = task_dir / "task.json" + if task_json.is_file(): + try: + data = json.loads(task_json.read_text(encoding="utf-8")) + if isinstance(data, dict): + status = str(data.get("status") or "unknown") + except (json.JSONDecodeError, OSError): + pass + lines.append(f"Current task: {_repo_relative(repo_root, task_dir)}; status={status}.") + else: + lines.append("Current task: none.") + + if get_tasks_dir and iter_active_tasks: + try: + task_count = sum(1 for _ in iter_active_tasks(get_tasks_dir(repo_root))) + lines.append( + f"Active tasks: {task_count} total. Use `python ./.trellis/scripts/task.py list --mine` only if needed." + ) + except Exception: + pass + + if get_active_journal_file and count_lines: + journal = get_active_journal_file(repo_root) + if journal: + lines.append( + f"Journal: {_repo_relative(repo_root, journal)}, {count_lines(journal)} / 2000 lines." + ) + + if spec_index_paths: + lines.append(f"Spec indexes: {len(spec_index_paths)} available.") + + return "\n".join(lines) + + +def _extract_range(content: str, start_header: str, end_header: str) -> str: + """Extract lines starting at `## start_header` up to (but excluding) `## end_header`.""" + lines = content.splitlines() + start: "int | None" = None + end: int = len(lines) + start_match = f"## {start_header}" + end_match = f"## {end_header}" + for i, line in enumerate(lines): + stripped = line.strip() + if start is None and stripped == start_match: + start = i + continue + if start is not None and stripped == end_match: + end = i + break + if start is None: + return "" + return "\n".join(lines[start:end]).rstrip() + + +_BREADCRUMB_TAG_RE = re.compile( + r"\[workflow-state:([A-Za-z0-9_-]+)\]\s*\n.*?\n\s*\[/workflow-state:\1\]", + re.DOTALL, +) + + +def _strip_breadcrumb_tag_blocks(content: str) -> str: + stripped = _BREADCRUMB_TAG_RE.sub("", content) + stripped = re.sub(r"<!--.*?-->", "", stripped, flags=re.DOTALL) + stripped = re.sub(r"^\[(?!/?workflow-state:)/?[^\]\n]+\]\s*\n?", "", stripped, flags=re.MULTILINE) + return re.sub(r"\n{3,}", "\n\n", stripped).strip() + + +def _build_workflow_toc(workflow_path: Path) -> str: + """Inject only the compact Phase Index summary for SessionStart.""" + content = read_file(workflow_path) + if not content: + return "No workflow.md found" + + out_lines = [ + "# Development Workflow - Session Summary", + "Full guide: .trellis/workflow.md. Step detail: `python ./.trellis/scripts/get_context.py --mode phase --step <X.Y>`.", + "", + ] + + phases = _extract_range(content, "Phase Index", "Phase 1: Plan") + if phases: + out_lines.append(_strip_breadcrumb_tag_blocks(phases).rstrip()) + + return "\n".join(out_lines).rstrip() + + +def main() -> None: + if should_skip_injection(): + sys.exit(0) + + # Read hook input from stdin + try: + hook_input = json.loads(sys.stdin.read()) + if not isinstance(hook_input, dict): + hook_input = {} + project_dir = Path(_normalize_windows_shell_path(hook_input.get("cwd", "."))).resolve() + except (json.JSONDecodeError, KeyError): + hook_input = {} + project_dir = Path(".").resolve() + + configure_project_encoding(project_dir) + + trellis_dir = project_dir / ".trellis" + spec_index_paths = _collect_spec_index_paths(trellis_dir) + + output = StringIO() + + output.write("""<session-context> +Trellis compact SessionStart context. Use it to orient the session; load details on demand. +</session-context> + +""") + output.write(FIRST_REPLY_NOTICE) + output.write("\n\n") + + output.write("<current-state>\n") + output.write(_build_compact_current_state(trellis_dir, hook_input, spec_index_paths)) + output.write("\n</current-state>\n\n") + + output.write("<trellis-workflow>\n") + output.write(_build_workflow_toc(trellis_dir / "workflow.md")) + output.write("\n</trellis-workflow>\n\n") + + output.write("<guidelines>\n") + output.write( + "Task context order for implementation/check: jsonl entries -> `prd.md` -> " + "`design.md if present` -> `implement.md if present`. Missing optional artifacts " + "are skipped for lightweight tasks.\n\n" + ) + + if spec_index_paths: + output.write("## Available indexes (read on demand)\n") + for p in spec_index_paths: + output.write(f"- {p}\n") + output.write("\n") + + output.write( + "Discover more via: " + "`python ./.trellis/scripts/get_context.py --mode packages`\n" + ) + output.write("</guidelines>\n\n") + + task_status = _get_task_status(trellis_dir, hook_input) + output.write(f"<task-status>\n{task_status}\n</task-status>\n\n") + + output.write("""<ready> +Context loaded. Follow <task-status>. Load workflow/spec/task details only when needed. +</ready>""") + + context = output.getvalue() + result = { + "suppressOutput": True, + "systemMessage": f"Trellis context injected ({len(context)} chars)", + "hookSpecificOutput": { + "hookEventName": "SessionStart", + "additionalContext": context, + }, + } + + print(json.dumps(result, ensure_ascii=False), flush=True) + + +if __name__ == "__main__": + main() diff --git a/.gitignore b/.gitignore index 2e70ae4..53d4ffb 100644 --- a/.gitignore +++ b/.gitignore @@ -19,4 +19,20 @@ config.toml rt.json free_rt.json -legacy/ \ No newline at end of file +legacy/ + +# Local browser/test/build artifacts +.playwright-cli/ +artifacts/ +cpa_governor_plugin/dist/ +cpa_governor_plugin/go/*.sqlite* +cpa_key_policy_plus_plugin/dist/ +cpa_key_policy_plus_plugin/go/build/ +cpa_key_policy_plus_plugin/go/*.sqlite* +cpa_key_policy_plus_plugin/go/*.so +cpa_key_policy_plus_plugin/go/*.h +cpa_codexcont_executor_plugin/dist/ +cpa_codexcont_executor_plugin/go/build/ +cpa_codexcont_executor_plugin/go/*.sqlite* +cpa_codexcont_executor_plugin/go/*.so +cpa_codexcont_executor_plugin/go/*.h diff --git a/.trellis/.gitignore b/.trellis/.gitignore new file mode 100644 index 0000000..5a991ea --- /dev/null +++ b/.trellis/.gitignore @@ -0,0 +1,32 @@ +# Developer identity (local only) +.developer + +# Current task pointer (each dev works on different task) +.current-task + +# Session/window scoped runtime state +.runtime/ + +# Ralph Loop state file +.ralph-state.json + +# Agent runtime files +.agents/ +.agent-log +.session-id + +# Task directory runtime files +.plan-log + +# Atomic update temp files +*.tmp + +# Update backup directories +.backup-* + +# Conflict resolution temp files +*.new + +# Python cache +**/__pycache__/ +**/*.pyc diff --git a/.trellis/.template-hashes.json b/.trellis/.template-hashes.json new file mode 100644 index 0000000..9ace9ed --- /dev/null +++ b/.trellis/.template-hashes.json @@ -0,0 +1,127 @@ +{ + "__version": 2, + "hashes": { + ".claude/agents/trellis-check.md": "4e4d849d91918228a288752c1196a8ad91ee090f760f04a6680319baf1f8aee5", + ".claude/agents/trellis-implement.md": "650bfb5f6bef4bdac138cde68e676631063afdf251938ec69d8f3f1504687407", + ".claude/agents/trellis-research.md": "f95e69d638266056713e79c884ead1e99d376d70284f66255b6dd139a3e712be", + ".claude/settings.json": "01226db3027908dac1260955e205877ee46c1d410912172d8bae9c53527b3b0f", + ".claude/hooks/inject-subagent-context.py": "4e7889074f93d668c6a312a7e9ddc65c8c619b17c2454bf72e5c563371c6bb98", + ".claude/hooks/inject-workflow-state.py": "f7fa9389ed7aa264597fff5de6277bec186e89a3ef539192997c6d026d88d5ec", + ".claude/hooks/session-start.py": "922691b5df26f6e9482796776dcd9d50ff3ea0aa9d0c8aa9abd2d7254b7ea38c", + ".claude/commands/trellis/continue.md": "d513aabcdb4d2d1afc75dc860bc66d1a971d64c542aa3956ec7ca583379a6ebc", + ".claude/commands/trellis/finish-work.md": "f11f661cff6d5d26dccb5e9574c3d2c7873a9dfaed7962d471ff5ea2fd48d691", + ".claude/skills/trellis-before-dev/SKILL.md": "8f897d8dd76c1eeb532cf15ba784360f45a93229f8dcd0acfc14282f014f92a0", + ".claude/skills/trellis-brainstorm/SKILL.md": "d698b386edf46abed6d0e6a14362b81287272625e06c3255e9d8d6cb810ef85a", + ".claude/skills/trellis-break-loop/SKILL.md": "35afb53fef42cd494e566f1ef170dbf442ec2be7e19931f28a14079b4dda753f", + ".claude/skills/trellis-check/SKILL.md": "2be18b6665b4da497c554acd5d76ef038cc960d25f3d4216f03e047202f2eadc", + ".claude/skills/trellis-update-spec/SKILL.md": "d975db7af166578488958751ae2c56edb827a68bddb569aa27acc3453f64e610", + ".claude/skills/trellis-meta/references/customize-local/add-project-local-conventions.md": "ef3380e71aa9f5103d37b467b1f725a8033ac516e4de31e4d790be02ec2c39e8", + ".claude/skills/trellis-meta/references/customize-local/change-agents.md": "7f2982162463f107f8b1a4fa1a41fee2bc7dbd0cc8e90c48559aba30c3ea403c", + ".claude/skills/trellis-meta/references/customize-local/change-context-loading.md": "e6aa7d938741c08b864f54e9bda8e4e68e04bc60cb46682f6acabaf37f9b2da2", + ".claude/skills/trellis-meta/references/customize-local/change-hooks.md": "c8b35dda1530de521cf6bb043188f0cbbea0c9180b1aa44e64e31e20433ef4ca", + ".claude/skills/trellis-meta/references/customize-local/change-skills-or-commands.md": "b3009ef20a4f24e5d8b196109dc9bab6bd30fc030dbc4fb796afdd2ca912e1ea", + ".claude/skills/trellis-meta/references/customize-local/change-spec-structure.md": "1a712408217ee9cc6a916d874e3d6ed1ba7a3bdc1b9dc4bb1c64393f0df1eb98", + ".claude/skills/trellis-meta/references/customize-local/change-task-lifecycle.md": "148b7442ef8106de907afd06f9d1ca96f7ec074caedced3dd4175b3a26698ca2", + ".claude/skills/trellis-meta/references/customize-local/change-workflow.md": "f7855f2db1bcb213ba843c38776ccdc1f4616ed687f84e977da2f5e6cf7195eb", + ".claude/skills/trellis-meta/references/customize-local/overview.md": "465db9cecf085b37f7aed2fc5240c92c638e937f7960ca35b0f05a780dd4fdc9", + ".claude/skills/trellis-meta/references/local-architecture/context-injection.md": "8497289bf333b3aa456f317039d1239b7ece79254aa0eb62cfc647714c866084", + ".claude/skills/trellis-meta/references/local-architecture/generated-files.md": "4356517517cef0ba7f3ba01965a4ba8953505702e4085f0797d3e36817c9669f", + ".claude/skills/trellis-meta/references/local-architecture/overview.md": "45ffd4ee95020f58201adc885f3dfc89b26483c2b350d96ca7f2f57f94d5ff5f", + ".claude/skills/trellis-meta/references/local-architecture/spec-system.md": "55f3c95033a3cbed6c06e225187071502d7dcf3f153f4becbec6aa79acc782f5", + ".claude/skills/trellis-meta/references/local-architecture/task-system.md": "25dbd2be6a2271591274b56616b853b16e6fd9f44f43073350688e89e87295a6", + ".claude/skills/trellis-meta/references/local-architecture/workflow.md": "cfcdc6e4468a5d9c816e929fcca01640cd41cfdaaa4824118b40a8e460c927b6", + ".claude/skills/trellis-meta/references/local-architecture/workspace-memory.md": "79786a1ca2980b1785a36aba8142f9d879459c47dc000c999f638e5c864d04d3", + ".claude/skills/trellis-meta/references/platform-files/agents.md": "8af9722fa637bd0cd8addd28ae5da6812a5413711924259fd4af0c2ebb447c5b", + ".claude/skills/trellis-meta/references/platform-files/hooks-and-settings.md": "6e2d6d88719c2779fe34004f63d36cff203d8f64e7fb620f7cb1cde15c37c462", + ".claude/skills/trellis-meta/references/platform-files/overview.md": "6479cd2393166b4b369b511c44b78cbc64975c8b1df96ee1d4d1bd06b75cd48d", + ".claude/skills/trellis-meta/references/platform-files/platform-map.md": "ded6751c06f31d0a701d33c9dd69c482a583539ad3ed464aaad9e705f793b212", + ".claude/skills/trellis-meta/references/platform-files/skills-and-commands.md": "85435eb8bb6921283575bca51268fc534c22fd3ca33782e841ee5c76140ae48f", + ".claude/skills/trellis-meta/SKILL.md": "942e898a6fd769a93a3ca6f43f9fe0412d0adae011654fd384e9cacbd2af4f34", + ".claude/skills/trellis-session-insight/references/cli-quick-reference.md": "b64803cee3898d70a8a9f79d701e81ebf6e96bd43ba36c0b33e2ac9182b28a5c", + ".claude/skills/trellis-session-insight/references/triggering-patterns.md": "121ecd23be83d1567e8ce15c366a81073d7a2b1d3ad616fce235c07ca1f1cc20", + ".claude/skills/trellis-session-insight/SKILL.md": "d2893a294da1fa2d920e784f7f8169bf5134d0204e6a41406f94447c498dbc8d", + ".claude/skills/trellis-spec-bootstrap/references/mcp-setup.md": "df542fc8f279edd38046d26a7c8151804b708f57b24d4aa2733cea587a88c65e", + ".claude/skills/trellis-spec-bootstrap/references/repository-analysis.md": "0dae98d774f6e34559b9f3442888ac43e3a8af110c37cbefc49ce256986858b6", + ".claude/skills/trellis-spec-bootstrap/references/spec-task-planning.md": "ef493d028c3b0807a8a534bb71fb92a68129f273db763ad27ceb464a522e799d", + ".claude/skills/trellis-spec-bootstrap/references/spec-writing.md": "e9800fe9ed4a4cd87062ea1829cf2caa8d170ec15e141678a6a30e74c497f47d", + ".claude/skills/trellis-spec-bootstrap/SKILL.md": "97bfa68c06cebb558eb4464bc1b81f7d2d56040d75baa8de1ee5ad90cca0196a", + ".agents/skills/trellis-continue/SKILL.md": "54f3148f1a15a95b149b33eb040cb818501cad4969ef75654bd149fea3ee56b6", + ".agents/skills/trellis-finish-work/SKILL.md": "79e6d165358253a7379cae647bbd50b6bf174a1107f451b768a2bb4ba7cc0b87", + ".agents/skills/trellis-before-dev/SKILL.md": "8f897d8dd76c1eeb532cf15ba784360f45a93229f8dcd0acfc14282f014f92a0", + ".agents/skills/trellis-brainstorm/SKILL.md": "d698b386edf46abed6d0e6a14362b81287272625e06c3255e9d8d6cb810ef85a", + ".agents/skills/trellis-break-loop/SKILL.md": "35afb53fef42cd494e566f1ef170dbf442ec2be7e19931f28a14079b4dda753f", + ".agents/skills/trellis-check/SKILL.md": "2be18b6665b4da497c554acd5d76ef038cc960d25f3d4216f03e047202f2eadc", + ".agents/skills/trellis-update-spec/SKILL.md": "003ce08a3404aeb50998029392c4d4e57b626edf526d3ebd585032bb92dcbb96", + ".agents/skills/trellis-meta/references/customize-local/add-project-local-conventions.md": "ef3380e71aa9f5103d37b467b1f725a8033ac516e4de31e4d790be02ec2c39e8", + ".agents/skills/trellis-meta/references/customize-local/change-agents.md": "7f2982162463f107f8b1a4fa1a41fee2bc7dbd0cc8e90c48559aba30c3ea403c", + ".agents/skills/trellis-meta/references/customize-local/change-context-loading.md": "e6aa7d938741c08b864f54e9bda8e4e68e04bc60cb46682f6acabaf37f9b2da2", + ".agents/skills/trellis-meta/references/customize-local/change-hooks.md": "c8b35dda1530de521cf6bb043188f0cbbea0c9180b1aa44e64e31e20433ef4ca", + ".agents/skills/trellis-meta/references/customize-local/change-skills-or-commands.md": "b3009ef20a4f24e5d8b196109dc9bab6bd30fc030dbc4fb796afdd2ca912e1ea", + ".agents/skills/trellis-meta/references/customize-local/change-spec-structure.md": "1a712408217ee9cc6a916d874e3d6ed1ba7a3bdc1b9dc4bb1c64393f0df1eb98", + ".agents/skills/trellis-meta/references/customize-local/change-task-lifecycle.md": "148b7442ef8106de907afd06f9d1ca96f7ec074caedced3dd4175b3a26698ca2", + ".agents/skills/trellis-meta/references/customize-local/change-workflow.md": "f7855f2db1bcb213ba843c38776ccdc1f4616ed687f84e977da2f5e6cf7195eb", + ".agents/skills/trellis-meta/references/customize-local/overview.md": "465db9cecf085b37f7aed2fc5240c92c638e937f7960ca35b0f05a780dd4fdc9", + ".agents/skills/trellis-meta/references/local-architecture/context-injection.md": "8497289bf333b3aa456f317039d1239b7ece79254aa0eb62cfc647714c866084", + ".agents/skills/trellis-meta/references/local-architecture/generated-files.md": "4356517517cef0ba7f3ba01965a4ba8953505702e4085f0797d3e36817c9669f", + ".agents/skills/trellis-meta/references/local-architecture/overview.md": "45ffd4ee95020f58201adc885f3dfc89b26483c2b350d96ca7f2f57f94d5ff5f", + ".agents/skills/trellis-meta/references/local-architecture/spec-system.md": "55f3c95033a3cbed6c06e225187071502d7dcf3f153f4becbec6aa79acc782f5", + ".agents/skills/trellis-meta/references/local-architecture/task-system.md": "25dbd2be6a2271591274b56616b853b16e6fd9f44f43073350688e89e87295a6", + ".agents/skills/trellis-meta/references/local-architecture/workflow.md": "cfcdc6e4468a5d9c816e929fcca01640cd41cfdaaa4824118b40a8e460c927b6", + ".agents/skills/trellis-meta/references/local-architecture/workspace-memory.md": "79786a1ca2980b1785a36aba8142f9d879459c47dc000c999f638e5c864d04d3", + ".agents/skills/trellis-meta/references/platform-files/agents.md": "8af9722fa637bd0cd8addd28ae5da6812a5413711924259fd4af0c2ebb447c5b", + ".agents/skills/trellis-meta/references/platform-files/hooks-and-settings.md": "6e2d6d88719c2779fe34004f63d36cff203d8f64e7fb620f7cb1cde15c37c462", + ".agents/skills/trellis-meta/references/platform-files/overview.md": "6479cd2393166b4b369b511c44b78cbc64975c8b1df96ee1d4d1bd06b75cd48d", + ".agents/skills/trellis-meta/references/platform-files/platform-map.md": "ded6751c06f31d0a701d33c9dd69c482a583539ad3ed464aaad9e705f793b212", + ".agents/skills/trellis-meta/references/platform-files/skills-and-commands.md": "85435eb8bb6921283575bca51268fc534c22fd3ca33782e841ee5c76140ae48f", + ".agents/skills/trellis-meta/SKILL.md": "942e898a6fd769a93a3ca6f43f9fe0412d0adae011654fd384e9cacbd2af4f34", + ".agents/skills/trellis-session-insight/references/cli-quick-reference.md": "b64803cee3898d70a8a9f79d701e81ebf6e96bd43ba36c0b33e2ac9182b28a5c", + ".agents/skills/trellis-session-insight/references/triggering-patterns.md": "121ecd23be83d1567e8ce15c366a81073d7a2b1d3ad616fce235c07ca1f1cc20", + ".agents/skills/trellis-session-insight/SKILL.md": "d2893a294da1fa2d920e784f7f8169bf5134d0204e6a41406f94447c498dbc8d", + ".agents/skills/trellis-spec-bootstrap/references/mcp-setup.md": "df542fc8f279edd38046d26a7c8151804b708f57b24d4aa2733cea587a88c65e", + ".agents/skills/trellis-spec-bootstrap/references/repository-analysis.md": "0dae98d774f6e34559b9f3442888ac43e3a8af110c37cbefc49ce256986858b6", + ".agents/skills/trellis-spec-bootstrap/references/spec-task-planning.md": "ef493d028c3b0807a8a534bb71fb92a68129f273db763ad27ceb464a522e799d", + ".agents/skills/trellis-spec-bootstrap/references/spec-writing.md": "e9800fe9ed4a4cd87062ea1829cf2caa8d170ec15e141678a6a30e74c497f47d", + ".agents/skills/trellis-spec-bootstrap/SKILL.md": "97bfa68c06cebb558eb4464bc1b81f7d2d56040d75baa8de1ee5ad90cca0196a", + ".agents/skills/trellis-start/SKILL.md": "ddcb643c43e967dc7e435cb815c9eb00ddab8913190d837d8b9504424f47a81a", + ".codex/agents/trellis-check.toml": "a64bf083a35146c9b0074b98d20efcdfa9dc448f1696677bab12ea390354c1b0", + ".codex/agents/trellis-implement.toml": "719b3c332d9bec179be5cbd2728994d1add7dbfc9585944df04e58a3f434fbc5", + ".codex/agents/trellis-research.toml": "73bf9654d99ee60cec9f6d77fe60fe2e32afbdd7d3f01c8f759c131364bf3c31", + ".codex/hooks/session-start.py": "a1f2de75eb17eb419ece4854d78f1d2255e0e732e672ec8fbfb439b71a09bf11", + ".codex/hooks/inject-workflow-state.py": "f7fa9389ed7aa264597fff5de6277bec186e89a3ef539192997c6d026d88d5ec", + ".codex/hooks.json": "7bad6065612c5bd0d4e0bb587bf3c9f3950c8060f9a52f4defd7247dd9ba8aec", + ".codex/config.toml": "4224eb7df6802a623cb1bee522aed0a23ba6be862b90f1b597a313fc16864b06", + "AGENTS.md": "6cacfe99748b435d0660c2463c697bc323d53798aecf3492283ca8eac1b29682", + ".trellis/agents/check.md": "edb4f57361407249a53bf5998ebf91c40d2b969e826a2c5e1b4e813a08bcb175", + ".trellis/agents/implement.md": "66e25ad046c94869442834bc3cdfbd5a9a7412d3ff54561d64d2886552c27e87", + ".trellis/config.yaml": "3e295bf4310763240647f40b3aeee7a7c6d134142cdc826e02d850ca2407fc43", + ".trellis/scripts/add_session.py": "f26b66a539d160c739d4b88fd926b3d7f6745be326cd57131e5ef17a7b011fbe", + ".trellis/scripts/common/active_task.py": "6c88ed40ef7289bca0f6d2ecba0f8b8aef46cd58788080fbeeea88de138a431f", + ".trellis/scripts/common/cli_adapter.py": "cd844d1e84b1a09b373b3a7609e4d5606ee9d4825154c002cc9bb3f54c8e2fb9", + ".trellis/scripts/common/config.py": "25c5a53ad20d6909be5209222e4208a84528805316a4d78350529459a364edb1", + ".trellis/scripts/common/developer.py": "b2141b0145a41f8cedb4f9a24c925796edb2f0f6fde7c86b559513ec30499368", + ".trellis/scripts/common/git.py": "e14817be7de122d3a106f509c2825aeb9669d962ba73ba241642d2931cfdf1d6", + ".trellis/scripts/common/git_context.py": "fa30ced454f1a91ffc9f8b2abeb32225e3447cbdc90bad783797374eba07265d", + ".trellis/scripts/common/io.py": "6480b181f2bc505323b28ed7a66963d7b7edc96251e83b4c8e7a45907cc721c8", + ".trellis/scripts/common/log.py": "471df6895cfac80f995edebbf9974f6b7440634b7a688f28b8331c868bc0f3cf", + ".trellis/scripts/common/packages_context.py": "efe158d7c99c2268851d0216fbb08de22836e418a8dbeb73575b8cc249eed7b7", + ".trellis/scripts/common/paths.py": "05898ef136cc7c4d861b05fbf2b16d53ddd3e6f311a231d4fcfcb81bde7c45ee", + ".trellis/scripts/common/safe_commit.py": "8789bff4b30a9065469210f2efab3f59f03dddd77bef4e4b6a5bb641f93539f4", + ".trellis/scripts/common/session_context.py": "11e336b77a42e8ae080ebcea3fe45938eb8b0d1d279e968eededa57de6006db8", + ".trellis/scripts/common/tasks.py": "4436a8b0b53c270a35989e26d9dbd92669408c6562d88c02083a404562da85fe", + ".trellis/scripts/common/task_context.py": "d174684d417bbe2fafc26b6afcddb264c7dc519527bb24d2055cd27daaad9b55", + ".trellis/scripts/common/task_queue.py": "0be61f713462b1fe4574927c82fc4704e678afe72dcb9813543aedf2f9e9e0c5", + ".trellis/scripts/common/task_store.py": "b6d5089ae823fee9d53fec3d4e20449e670b7968d31ef9eac4561dd23661b64c", + ".trellis/scripts/common/task_utils.py": "f5ef4af87ba3e11d8b19630c0c96d009de1811fc9be56c2027a9c96e21ed103e", + ".trellis/scripts/common/trellis_config.py": "0839dcf90ebbd77712c276930a89335b3313927051650c91d220fb51ca2a6a3c", + ".trellis/scripts/common/types.py": "9962081cc2608fb9d1deb32c6880e336f62cdca6b338e7ae813304701e155ee9", + ".trellis/scripts/common/workflow_phase.py": "3141c0aa55109b883886221a95878fac7d0a1aedd25fb9a963c47add7383db4e", + ".trellis/scripts/common/__init__.py": "3d5e9347141f0296319a5beb29d69ae714c5a474b9078caeb3edd7c5f6562e22", + ".trellis/scripts/get_context.py": "af3ea7cd563a453227cf2cb4ab04d667390046b7febfac2217348d0892781f4b", + ".trellis/scripts/get_developer.py": "84c27076323c3e0f2c9c8ed16e8aa865e225d902a187c37e20ee1a46e7142d8f", + ".trellis/scripts/hooks/linear_sync.py": "cfc270b7ff775caa5b2434823c45414a3b37f9ba2aa1e293a26daef9fd2e577a", + ".trellis/scripts/init_developer.py": "0943f1c240993649ab89b91a2c5b379e84daa8c53b35f0490774bff05a552873", + ".trellis/scripts/task.py": "b3a43f6ef149ec8f5a288be53fe7be0a94955b78872ad7f77d0db4a1aecfa87b", + ".trellis/scripts/__init__.py": "1242be5b972094c2e141aecbe81a4efd478f6534e3d5e28306374e6a18fcf46c", + ".trellis/workflow.md": "28190a1db4ad4533c545187a8647f40003ab7b763e7bd75b3ee1c2d77909bc5e" + } +} \ No newline at end of file diff --git a/.trellis/.version b/.trellis/.version new file mode 100644 index 0000000..4124bbd --- /dev/null +++ b/.trellis/.version @@ -0,0 +1 @@ +0.6.0-beta.23 \ No newline at end of file diff --git a/.trellis/agents/check.md b/.trellis/agents/check.md new file mode 100644 index 0000000..6c1bf13 --- /dev/null +++ b/.trellis/agents/check.md @@ -0,0 +1,70 @@ +--- +name: check +description: | + Code quality auditor for the Trellis channel runtime. Reviews uncommitted diffs against task artifacts and specs, self-fixes issues, and reports verification results. +provider: claude +labels: [trellis, check] +--- + +# Check Agent (channel runtime) + +You are the Check Agent spawned by `trellis channel spawn --agent check` inside the Trellis channel runtime. You receive an `Active task: <path>` line in your inbox; use it to locate task artifacts on disk. + +## Context + +Before reviewing, read in this order: + +1. `<task-path>/check.jsonl` if present — spec manifest curated for this turn; read every listed file +2. `<task-path>/prd.md` — requirements +3. `<task-path>/design.md` if present — technical design +4. `<task-path>/implement.md` if present — execution plan +5. `.trellis/spec/` — project-wide guidelines (load only what is relevant to the diff under review) + +## Core Responsibilities + +1. **Get the diff** — `git diff` / `git diff --staged` for uncommitted changes +2. **Review against task artifacts** — does the diff satisfy `prd.md` (and `design.md` / `implement.md` if present)? +3. **Review against specs** — naming, structure, type safety, error handling, conventions in `.trellis/spec/` +4. **Self-fix** — when an issue is mechanical and small, fix it directly with the editing tools you have +5. **Run verification** — project lint and typecheck on the changed scope +6. **Report** — concrete findings with `file:line` citations and what was fixed vs. what is open + +## Forbidden Operations + +- `git commit` +- `git push` +- `git merge` + +The supervising main session owns commits. Report the post-fix state; do not commit on its behalf. + +## Workflow + +1. Run `git diff --name-only` and `git diff` to scope the changes +2. Read the task artifacts and relevant spec files +3. For each issue: + - If mechanical (lint nit, missing type, wrong import, dead branch) → fix in-place + - If a design/judgment issue → record and report, do not silently rewrite +4. Run the project's lint and typecheck on the changed scope after self-fixes +5. Report + +## Report Format + +``` +## Self-Check Complete + +### Files Checked +- <path> + +### Issues Found and Fixed +1. `<file>:<line>` — <what was wrong> → <what you changed> + +### Issues Not Fixed +- `<file>:<line>` — <issue> — <why deferred to the main session> + +### Verification Results +- TypeCheck: <pass|fail|skipped + reason> +- Lint: <pass|fail|skipped + reason> + +### Summary +Checked <N> files, found <X> issues, fixed <Y>, <X-Y> open. +``` diff --git a/.trellis/agents/implement.md b/.trellis/agents/implement.md new file mode 100644 index 0000000..3262f79 --- /dev/null +++ b/.trellis/agents/implement.md @@ -0,0 +1,71 @@ +--- +name: implement +description: | + Code implementation expert for the Trellis channel runtime. Understands specs and task artifacts, then implements features. No git commit allowed. +provider: claude +labels: [trellis, implement] +--- + +# Implement Agent (channel runtime) + +You are the Implement Agent spawned by `trellis channel spawn --agent implement` inside the Trellis channel runtime. You receive an `Active task: <path>` line in your inbox; use it to locate task artifacts on disk. + +## Context + +Before implementing, read in this order: + +1. `<task-path>/implement.jsonl` if present — spec manifest curated for this turn; read every listed file +2. `<task-path>/prd.md` — requirements +3. `<task-path>/design.md` if present — technical design +4. `<task-path>/implement.md` if present — execution plan +5. `.trellis/spec/` — project-wide guidelines (load only what is relevant to the diff you are about to write) + +## Core Responsibilities + +1. **Understand specs** — read relevant spec files in `.trellis/spec/` +2. **Understand task artifacts** — read the artifacts listed above +3. **Implement features** — write code that follows specs and existing patterns +4. **Self-check** — run lint and typecheck on the changed scope before reporting + +## Forbidden Operations + +- `git commit` +- `git push` +- `git merge` + +The supervising main session owns commits. Report what changed; do not commit on its behalf. + +## Workflow + +1. Read relevant specs based on task type and the files in `implement.jsonl` if present +2. Read the task's `prd.md`, `design.md` if present, and `implement.md` if present +3. Implement features following specs and existing patterns +4. Run the project's lint and typecheck commands on the changed scope +5. Report files touched, key decisions, and verification results back to the channel + +## Code Standards + +- Follow existing code patterns +- Don't add unnecessary abstractions +- Only do what the PRD asks for; no speculative scope expansion +- Surface uncertainty back to the channel rather than guessing + +## Report Format + +``` +## Implementation Complete + +### Files Modified +- <path> — <one-line description> + +### Implementation Summary +1. <step> +2. <step> + +### Verification Results +- Lint: <pass|fail|skipped + reason> +- TypeCheck: <pass|fail|skipped + reason> + +### Open Questions +- <if any, otherwise omit> +``` diff --git a/.trellis/config.yaml b/.trellis/config.yaml new file mode 100644 index 0000000..002a712 --- /dev/null +++ b/.trellis/config.yaml @@ -0,0 +1,110 @@ +# Trellis Configuration +# Project-level settings for the Trellis workflow system +# +# All values have sensible defaults. Only override what you need. + +#------------------------------------------------------------------------------- +# Session Recording +#------------------------------------------------------------------------------- + +# Commit message used when auto-committing journal/index changes +# after running add_session.py +session_commit_message: "chore: record journal" + +# Maximum lines per journal file before rotating to a new one +max_journal_lines: 2000 + +#------------------------------------------------------------------------------- +# Session Auto-Commit +#------------------------------------------------------------------------------- + +# Auto-commit behavior for session journal + task archive operations. +# - true (default): scripts auto-stage and auto-commit journal / task changes +# after add_session.py / task.py archive runs. +# - false: scripts do not touch git. Files (journal-*.md, task archive moves) +# are still written to disk; you decide whether to git add / commit. +# +# Use `false` if your project's .gitignore intentionally excludes `.trellis/` +# and you want session data kept local-only, or if you prefer to review +# staged changes manually before each commit. +# +# Accepts: true / false / yes / no / 1 / 0 / on / off (case-insensitive). +# +# session_auto_commit: true + +#------------------------------------------------------------------------------- +# Task Lifecycle Hooks +#------------------------------------------------------------------------------- + +# Shell commands to run after task lifecycle events. +# Each hook receives TASK_JSON_PATH environment variable pointing to task.json. +# Hook failures print a warning but do not block the main operation. +# +# hooks: +# after_create: +# - "echo 'Task created'" +# after_start: +# - "echo 'Task started'" +# after_finish: +# - "echo 'Task finished'" +# after_archive: +# - "echo 'Task archived'" + +#------------------------------------------------------------------------------- +# Monorepo / Packages +#------------------------------------------------------------------------------- + +# Declare packages for monorepo projects. +# Trellis auto-detects workspaces during `trellis init`, but you can also +# configure them manually here. +# +# packages: +# frontend: +# path: packages/frontend +# backend: +# path: packages/backend +# docs: +# path: docs-site +# type: submodule +# # For polyrepo / meta-repo layouts (independent .git in each subdir), +# # mark the package with `git: true`. The runtime treats it as an +# # independent repository for things like git-context display. +# webapp: +# path: ./webapp +# git: true + +# Default package used when --package is not specified. +# default_package: frontend + +#------------------------------------------------------------------------------- +# Channel worker OOM guard +#------------------------------------------------------------------------------- +# Default safeguards for `trellis channel spawn` workers. The guard runs +# at spawn time (cleans expired idle workers, then enforces the live-worker +# budget) and inside each supervisor (self-terminates a worker that stays +# continuously idle past `idle_timeout`). +# +# Precedence: CLI flag > env var (TRELLIS_CHANNEL_WORKER_IDLE_TIMEOUT / +# TRELLIS_CHANNEL_MAX_LIVE_WORKERS) > this config > built-in default. +# +# `idle_timeout: 0` disables idle cleanup (workers can sit idle forever +# unless explicitly killed or given `--timeout`). +# `max_live_workers: 0` disables the spawn-time budget check. +# +channel: + worker_guard: + idle_timeout: 5m + max_live_workers: 6 + +#------------------------------------------------------------------------------- +# Codex (dispatch behavior) +#------------------------------------------------------------------------------- +# Codex-only knob; other platforms ignore it. Default ("inline") makes the +# main Codex agent edit code directly because Codex sub-agents run with +# `fork_turns="none"` isolation and can't inherit the parent session's +# task context. Set to "sub-agent" to opt into the legacy dispatch model +# (main agent spawns trellis-implement / trellis-check / trellis-research +# sub-agents). +# +# codex: +# dispatch_mode: inline # or "sub-agent" to dispatch trellis-* sub-agents diff --git a/.trellis/scripts/__init__.py b/.trellis/scripts/__init__.py new file mode 100644 index 0000000..815a137 --- /dev/null +++ b/.trellis/scripts/__init__.py @@ -0,0 +1,5 @@ +""" +Trellis Python Scripts + +This module provides Python implementations of Trellis workflow scripts. +""" diff --git a/.trellis/scripts/add_session.py b/.trellis/scripts/add_session.py new file mode 100644 index 0000000..7149739 --- /dev/null +++ b/.trellis/scripts/add_session.py @@ -0,0 +1,547 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Add a new session to journal file and update index.md. + +Usage: + python add_session.py --title "Title" --commit "hash" --summary "Summary" [--package cli] + python add_session.py --title "Title" --branch "feat/my-branch" + + # Pipe detailed content via stdin (use --stdin to opt in): + cat << 'EOF' | python add_session.py --stdin --title "Title" --summary "Summary" + <session content here> + EOF + +Branch resolution order: + 1. --branch CLI arg (explicit) + 2. task.json branch field (from active task) + 3. git branch --show-current (auto-detect) + 4. None (omitted gracefully) +""" + +from __future__ import annotations + +import argparse +import re +import sys +from datetime import datetime +from pathlib import Path + +from common.paths import ( + FILE_JOURNAL_PREFIX, + get_repo_root, + get_current_task, + get_developer, + get_workspace_dir, +) +from common.developer import ensure_developer +from common.git import run_git +from common.safe_commit import ( + print_gitignore_warning, + safe_git_add, + safe_trellis_paths_to_add, +) +from common.tasks import load_task +from common.config import ( + get_packages, + get_session_auto_commit, + get_session_commit_message, + get_max_journal_lines, + is_monorepo, + resolve_package, + validate_package, +) + + +# ============================================================================= +# Helper Functions +# ============================================================================= + +def get_latest_journal_info(dev_dir: Path) -> tuple[Path | None, int, int]: + """Get latest journal file info. + + Returns: + Tuple of (file_path, file_number, line_count). + """ + latest_file: Path | None = None + latest_num = -1 + + for f in dev_dir.glob(f"{FILE_JOURNAL_PREFIX}*.md"): + if not f.is_file(): + continue + + match = re.search(r"(\d+)$", f.stem) + if match: + num = int(match.group(1)) + if num > latest_num: + latest_num = num + latest_file = f + + if latest_file: + lines = len(latest_file.read_text(encoding="utf-8").splitlines()) + return latest_file, latest_num, lines + + return None, 0, 0 + + +def get_current_session(index_file: Path) -> int: + """Get current session number from index.md.""" + if not index_file.is_file(): + return 0 + + content = index_file.read_text(encoding="utf-8") + for line in content.splitlines(): + if "Total Sessions" in line: + match = re.search(r":\s*(\d+)", line) + if match: + return int(match.group(1)) + return 0 + + +def _extract_journal_num(filename: str) -> int: + """Extract journal number from filename for sorting.""" + match = re.search(r"(\d+)", filename) + return int(match.group(1)) if match else 0 + + +def count_journal_files(dev_dir: Path, active_num: int) -> str: + """Count journal files and return table rows.""" + active_file = f"{FILE_JOURNAL_PREFIX}{active_num}.md" + result_lines = [] + + files = sorted( + [f for f in dev_dir.glob(f"{FILE_JOURNAL_PREFIX}*.md") if f.is_file()], + key=lambda f: _extract_journal_num(f.stem), + reverse=True + ) + + for f in files: + filename = f.name + lines = len(f.read_text(encoding="utf-8").splitlines()) + status = "Active" if filename == active_file else "Archived" + result_lines.append(f"| `{filename}` | ~{lines} | {status} |") + + return "\n".join(result_lines) + + +def create_new_journal_file( + dev_dir: Path, num: int, developer: str, today: str, max_lines: int = 2000, +) -> Path: + """Create a new journal file.""" + prev_num = num - 1 + new_file = dev_dir / f"{FILE_JOURNAL_PREFIX}{num}.md" + + content = f"""# Journal - {developer} (Part {num}) + +> Continuation from `{FILE_JOURNAL_PREFIX}{prev_num}.md` (archived at ~{max_lines} lines) +> Started: {today} + +--- + +""" + new_file.write_text(content, encoding="utf-8") + return new_file + + +def generate_session_content( + session_num: int, + title: str, + commit: str, + summary: str, + extra_content: str, + today: str, + package: str | None = None, + branch: str | None = None, +) -> str: + """Generate session content.""" + if commit and commit != "-": + commit_table = """| Hash | Message | +|------|---------|""" + for c in commit.split(","): + c = c.strip() + commit_table += f"\n| `{c}` | (see git log) |" + else: + commit_table = "(No commits - planning session)" + + package_line = f"\n**Package**: {package}" if package else "" + branch_line = f"\n**Branch**: `{branch}`" if branch else "" + + return f""" + +## Session {session_num}: {title} + +**Date**: {today} +**Task**: {title}{package_line}{branch_line} + +### Summary + +{summary} + +### Main Changes + +{extra_content} + +### Git Commits + +{commit_table} + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete +""" + + +def update_index( + index_file: Path, + dev_dir: Path, + title: str, + commit: str, + new_session: int, + active_file: str, + today: str, + branch: str | None = None, +) -> bool: + """Update index.md with new session info.""" + # Format commit for display + commit_display = "-" + if commit and commit != "-": + commit_display = re.sub(r"([a-f0-9]{7,})", r"`\1`", commit.replace(",", ", ")) + + # Get file number from active_file name + match = re.search(r"(\d+)", active_file) + active_num = int(match.group(1)) if match else 0 + files_table = count_journal_files(dev_dir, active_num) + + print(f"Updating index.md for session {new_session}...") + print(f" Title: {title}") + print(f" Commit: {commit_display}") + print(f" Active File: {active_file}") + print() + + content = index_file.read_text(encoding="utf-8") + + if "@@@auto:current-status" not in content: + print("Error: Markers not found in index.md. Please ensure markers exist.", file=sys.stderr) + return False + + # Process sections + lines = content.splitlines() + new_lines = [] + + in_current_status = False + in_active_documents = False + in_session_history = False + header_written = False + + for line in lines: + if "@@@auto:current-status" in line: + new_lines.append(line) + in_current_status = True + new_lines.append(f"- **Active File**: `{active_file}`") + new_lines.append(f"- **Total Sessions**: {new_session}") + new_lines.append(f"- **Last Active**: {today}") + continue + + if "@@@/auto:current-status" in line: + in_current_status = False + new_lines.append(line) + continue + + if "@@@auto:active-documents" in line: + new_lines.append(line) + in_active_documents = True + new_lines.append("| File | Lines | Status |") + new_lines.append("|------|-------|--------|") + new_lines.append(files_table) + continue + + if "@@@/auto:active-documents" in line: + in_active_documents = False + new_lines.append(line) + continue + + if "@@@auto:session-history" in line: + new_lines.append(line) + in_session_history = True + header_written = False + continue + + if "@@@/auto:session-history" in line: + in_session_history = False + new_lines.append(line) + continue + + if in_current_status: + continue + + if in_active_documents: + continue + + if in_session_history: + # Migrate old 4/6-column headers to 5-column Branch-only history. + if re.match( + r"^\|\s*#\s*\|\s*Date\s*\|\s*Title\s*\|\s*Commits\s*\|\s*Branch\s*\|\s*Base Branch\s*\|\s*$", + line, + ): + new_lines.append("| # | Date | Title | Commits | Branch |") + continue + if re.match(r"^\|\s*#\s*\|\s*Date\s*\|\s*Title\s*\|\s*Commits\s*\|\s*Branch\s*\|\s*$", line): + new_lines.append("| # | Date | Title | Commits | Branch |") + continue + if re.match(r"^\|\s*#\s*\|\s*Date\s*\|\s*Title\s*\|\s*Commits\s*\|\s*$", line): + new_lines.append("| # | Date | Title | Commits | Branch |") + continue + if re.match(r"^\|[-| ]+\|\s*$", line) and not header_written: + new_lines.append("|---|------|-------|---------|--------|") + new_lines.append(f"| {new_session} | {today} | {title} | {commit_display} | `{branch or '-'}` |") + header_written = True + continue + new_lines.append(line) + continue + + new_lines.append(line) + + index_file.write_text("\n".join(new_lines), encoding="utf-8") + print("[OK] Updated index.md successfully!") + return True + + +# ============================================================================= +# Main Function +# ============================================================================= + +def _auto_commit_workspace(repo_root: Path) -> None: + """Stage Trellis-owned workspace + task paths and commit. + + Path scope is restricted to specific products (journal files, index.md, + active task dirs, the archive subtree). We never `git add` the whole + `.trellis/` tree, and if `.gitignore` blocks the specific paths we + warn + skip — never retry with ``-f``. + + Honors ``session_auto_commit`` in ``.trellis/config.yaml``: when set to + ``false``, this function returns immediately without touching git + (journal/index files are still written to disk by the caller). + """ + if not get_session_auto_commit(repo_root): + print( + "[OK] session_auto_commit: false — skipping git stage/commit.", + file=sys.stderr, + ) + return + + commit_msg = get_session_commit_message(repo_root) + paths = safe_trellis_paths_to_add(repo_root) + if not paths: + print("[OK] No workspace changes to commit.", file=sys.stderr) + return + + success, _, err = safe_git_add(paths, repo_root) + if not success: + if err and "ignored by" in err.lower(): + print_gitignore_warning(paths) + else: + print( + f"[WARN] git add failed: {err.strip() if err else 'unknown error'}", + file=sys.stderr, + ) + return + + # Check if there are staged changes for the paths we just staged. + rc, _, _ = run_git( + ["diff", "--cached", "--quiet", "--", *paths], cwd=repo_root + ) + if rc == 0: + print("[OK] No workspace changes to commit.", file=sys.stderr) + return + + rc, _, commit_err = run_git(["commit", "-m", commit_msg], cwd=repo_root) + if rc == 0: + print(f"[OK] Auto-committed: {commit_msg}", file=sys.stderr) + else: + print( + f"[WARN] Auto-commit failed: {commit_err.strip()}", + file=sys.stderr, + ) + + +def add_session( + title: str, + commit: str = "-", + summary: str = "(Add summary)", + extra_content: str = "(Add details)", + auto_commit: bool = True, + package: str | None = None, + branch: str | None = None, +) -> int: + """Add a new session.""" + repo_root = get_repo_root() + ensure_developer(repo_root) + + developer = get_developer(repo_root) + if not developer: + print("Error: Developer not initialized", file=sys.stderr) + return 1 + + dev_dir = get_workspace_dir(repo_root) + if not dev_dir: + print("Error: Workspace directory not found", file=sys.stderr) + return 1 + + max_lines = get_max_journal_lines(repo_root) + + index_file = dev_dir / "index.md" + today = datetime.now().strftime("%Y-%m-%d") + + journal_file, current_num, current_lines = get_latest_journal_info(dev_dir) + current_session = get_current_session(index_file) + new_session = current_session + 1 + + session_content = generate_session_content( + new_session, title, commit, summary, extra_content, today, package, + branch, + ) + content_lines = len(session_content.splitlines()) + + print("========================================", file=sys.stderr) + print("ADD SESSION", file=sys.stderr) + print("========================================", file=sys.stderr) + print("", file=sys.stderr) + print(f"Session: {new_session}", file=sys.stderr) + print(f"Title: {title}", file=sys.stderr) + print(f"Commit: {commit}", file=sys.stderr) + print("", file=sys.stderr) + print(f"Current journal file: {FILE_JOURNAL_PREFIX}{current_num}.md", file=sys.stderr) + print(f"Current lines: {current_lines}", file=sys.stderr) + print(f"New content lines: {content_lines}", file=sys.stderr) + print(f"Total after append: {current_lines + content_lines}", file=sys.stderr) + print("", file=sys.stderr) + + target_file = journal_file + target_num = current_num + + if current_lines + content_lines > max_lines: + target_num = current_num + 1 + print(f"[!] Exceeds {max_lines} lines, creating {FILE_JOURNAL_PREFIX}{target_num}.md", file=sys.stderr) + target_file = create_new_journal_file(dev_dir, target_num, developer, today, max_lines) + print(f"Created: {target_file}", file=sys.stderr) + + # Append session content + if target_file: + with target_file.open("a", encoding="utf-8") as f: + f.write(session_content) + print(f"[OK] Appended session to {target_file.name}", file=sys.stderr) + + print("", file=sys.stderr) + + # Update index.md + active_file = f"{FILE_JOURNAL_PREFIX}{target_num}.md" + if not update_index( + index_file, + dev_dir, + title, + commit, + new_session, + active_file, + today, + branch, + ): + return 1 + + print("", file=sys.stderr) + print("========================================", file=sys.stderr) + print(f"[OK] Session {new_session} added successfully!", file=sys.stderr) + print("========================================", file=sys.stderr) + print("", file=sys.stderr) + print("Files updated:", file=sys.stderr) + print(f" - {target_file.name if target_file else 'journal'}", file=sys.stderr) + print(" - index.md", file=sys.stderr) + + # Auto-commit workspace changes + if auto_commit: + print("", file=sys.stderr) + _auto_commit_workspace(repo_root) + + return 0 + + +# ============================================================================= +# Main Entry +# ============================================================================= + +def main() -> int: + """CLI entry point.""" + parser = argparse.ArgumentParser( + description="Add a new session to journal file and update index.md" + ) + parser.add_argument("--title", required=True, help="Session title") + parser.add_argument("--commit", default="-", help="Comma-separated commit hashes") + parser.add_argument("--summary", default="(Add summary)", help="Brief summary") + parser.add_argument("--content-file", help="Path to file with detailed content") + parser.add_argument("--package", help="Package name tag (e.g., cli, docs-site)") + parser.add_argument("--branch", help="Branch name (auto-detected if omitted)") + parser.add_argument("--no-commit", action="store_true", + help="Skip auto-commit of workspace changes") + parser.add_argument("--stdin", action="store_true", + help="Read extra content from stdin (explicit opt-in)") + + args = parser.parse_args() + + extra_content = "(Add details)" + if args.content_file: + content_path = Path(args.content_file) + if content_path.is_file(): + extra_content = content_path.read_text(encoding="utf-8") + elif args.stdin: + extra_content = sys.stdin.read() + + # Load active task once — shared by package and branch resolution + repo_root = get_repo_root() + current = get_current_task(repo_root) + task_data = load_task(repo_root / current) if current else None + + package = args.package + if package: + # CLI source: fail-fast in monorepo, ignore in single-repo + if not is_monorepo(repo_root): + print("Warning: --package ignored in single-repo project", file=sys.stderr) + package = None + elif not validate_package(package, repo_root): + packages = get_packages(repo_root) + available = ", ".join(sorted(packages.keys())) if packages else "(none)" + print(f"Error: unknown package '{package}'. Available: {available}", file=sys.stderr) + return 1 + else: + # Inferred: active task's task.json.package → default_package → None + task_package = task_data.package if task_data else None + package = resolve_package(task_package, repo_root) + + # Resolve branch: CLI → task.json → git auto-detect → None + branch = args.branch + + if not branch: + if task_data and task_data.raw.get("branch"): + branch = task_data.raw["branch"] + else: + _, branch_out, _ = run_git(["branch", "--show-current"], cwd=repo_root) + detected = branch_out.strip() + if detected: + branch = detected + + return add_session( + args.title, args.commit, args.summary, extra_content, + auto_commit=not args.no_commit, + package=package, + branch=branch, + ) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.trellis/scripts/common/__init__.py b/.trellis/scripts/common/__init__.py new file mode 100644 index 0000000..6d72360 --- /dev/null +++ b/.trellis/scripts/common/__init__.py @@ -0,0 +1,92 @@ +""" +Common utilities for Trellis workflow scripts. + +This module provides shared functionality used by other Trellis scripts. +""" + +import io +import sys + +# ============================================================================= +# Windows Encoding Fix (MUST be at top, before any other output) +# ============================================================================= +# On Windows, stdout defaults to the system code page (often GBK/CP936). +# This causes UnicodeEncodeError when printing non-ASCII characters. +# +# Any script that imports from common will automatically get this fix. +# ============================================================================= + + +def _configure_stream(stream: object) -> object: + """Configure a stream for UTF-8 encoding on Windows.""" + # Try reconfigure() first (Python 3.7+, more reliable) + if hasattr(stream, "reconfigure"): + stream.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] + return stream + # Fallback: detach and rewrap with TextIOWrapper + elif hasattr(stream, "detach"): + return io.TextIOWrapper( + stream.detach(), # type: ignore[union-attr] + encoding="utf-8", + errors="replace", + ) + return stream + + +if sys.platform == "win32": + sys.stdout = _configure_stream(sys.stdout) # type: ignore[assignment] + sys.stderr = _configure_stream(sys.stderr) # type: ignore[assignment] + sys.stdin = _configure_stream(sys.stdin) # type: ignore[assignment] + + +def configure_encoding() -> None: + """ + Configure stdout/stderr/stdin for UTF-8 encoding on Windows. + + This is automatically called when importing from common, + but can be called manually for scripts that don't import common. + + Safe to call multiple times. + """ + global sys + if sys.platform == "win32": + sys.stdout = _configure_stream(sys.stdout) # type: ignore[assignment] + sys.stderr = _configure_stream(sys.stderr) # type: ignore[assignment] + sys.stdin = _configure_stream(sys.stdin) # type: ignore[assignment] + + +from .paths import ( + DIR_WORKFLOW, + DIR_WORKSPACE, + DIR_TASKS, + DIR_ARCHIVE, + DIR_SPEC, + DIR_SCRIPTS, + FILE_DEVELOPER, + FILE_CURRENT_TASK, + FILE_TASK_JSON, + FILE_JOURNAL_PREFIX, + get_repo_root, + get_developer, + check_developer, + get_tasks_dir, + get_workspace_dir, + get_active_journal_file, + count_lines, + get_current_task, + get_current_task_abs, + normalize_task_ref, + resolve_task_ref, + set_current_task, + clear_current_task, + has_current_task, + generate_task_date_prefix, +) + +from .active_task import ( + ActiveTask, + clear_active_task, + resolve_active_task, + resolve_context_key, + set_active_task, +) diff --git a/.trellis/scripts/common/active_task.py b/.trellis/scripts/common/active_task.py new file mode 100644 index 0000000..e6597e8 --- /dev/null +++ b/.trellis/scripts/common/active_task.py @@ -0,0 +1,626 @@ +#!/usr/bin/env python3 +"""Session-scoped active task resolution. + +The user-facing concept is a single "active task". Trellis stores that pointer +per AI session/window under `.trellis/.runtime/sessions/`; without a stable +session key there is no active task. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +import sys +import time +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +DIR_WORKFLOW = ".trellis" +DIR_TASKS = "tasks" +DIR_RUNTIME = ".runtime" +DIR_SESSIONS = "sessions" +DIR_CURSOR_SHELL = "cursor-shell" +CURSOR_SHELL_TICKET_TTL_SECONDS = 30 +TASK_SESSION_COMMANDS = {"start", "current", "finish"} + +_SESSION_KEYS = ("session_id", "sessionId", "sessionID") +_CONVERSATION_KEYS = ("conversation_id", "conversationId", "conversationID") +_TRANSCRIPT_KEYS = ("transcript_path", "transcriptPath", "transcript") +_NESTED_KEYS = ("input", "properties", "event", "hook_input", "hookInput") +_KNOWN_PLATFORMS = { + "claude", + "codex", + "cursor", + "opencode", + "gemini", + "droid", + "qoder", + "codebuddy", + "kiro", + "copilot", + "pi", +} + +_ENV_SESSION_KEYS: tuple[tuple[str, tuple[str, ...]], ...] = ( + ("claude", ("CLAUDE_SESSION_ID", "CLAUDE_CODE_SESSION_ID")), + ("codex", ("CODEX_SESSION_ID", "CODEX_THREAD_ID")), + ("cursor", ("CURSOR_SESSION_ID",)), + ("opencode", ("OPENCODE_SESSION_ID", "OPENCODE_SESSIONID", "OPENCODE_RUN_ID")), + ("gemini", ("GEMINI_SESSION_ID",)), + ("droid", ("FACTORY_SESSION_ID", "DROID_SESSION_ID")), + ("qoder", ("QODER_SESSION_ID",)), + ("codebuddy", ("CODEBUDDY_SESSION_ID",)), + ("kiro", ("KIRO_SESSION_ID",)), + ("copilot", ("COPILOT_SESSION_ID", "COPILOT_SESSIONID")), + ("pi", ("PI_SESSION_ID", "PI_SESSIONID")), +) +_ENV_CONVERSATION_KEYS: tuple[tuple[str, tuple[str, ...]], ...] = ( + ("cursor", ("CURSOR_CONVERSATION_ID", "CURSOR_CONVERSATIONID")), +) +_ENV_TRANSCRIPT_KEYS: tuple[tuple[str, tuple[str, ...]], ...] = ( + ("claude", ("CLAUDE_TRANSCRIPT_PATH",)), + ("codex", ("CODEX_TRANSCRIPT_PATH",)), + ("cursor", ("CURSOR_TRANSCRIPT_PATH",)), + ("gemini", ("GEMINI_TRANSCRIPT_PATH",)), + ("droid", ("FACTORY_TRANSCRIPT_PATH", "DROID_TRANSCRIPT_PATH")), + ("qoder", ("QODER_TRANSCRIPT_PATH",)), + ("codebuddy", ("CODEBUDDY_TRANSCRIPT_PATH",)), +) +_ENV_PLATFORM_ALIASES = { + "claude-code": "claude", + "factory": "droid", + "factory-ai": "droid", + "github-copilot": "copilot", +} + + +@dataclass(frozen=True) +class ActiveTask: + """Resolved active task state.""" + + task_path: str | None + source_type: str + context_key: str | None = None + stale: bool = False + + @property + def source(self) -> str: + """Human-readable source label.""" + if self.source_type == "session" and self.context_key: + return f"session:{self.context_key}" + if self.source_type == "session-fallback" and self.context_key: + return f"session-fallback:{self.context_key}" + return self.source_type + + +def normalize_task_ref(task_ref: str) -> str: + """Normalize a task ref for stable storage and comparison.""" + normalized = task_ref.strip() + if not normalized: + return "" + + path_obj = Path(normalized) + if path_obj.is_absolute(): + return str(path_obj) + + normalized = normalized.replace("\\", "/") + while normalized.startswith("./"): + normalized = normalized[2:] + + if normalized.startswith(f"{DIR_TASKS}/"): + return f"{DIR_WORKFLOW}/{normalized}" + + return normalized + + +def resolve_task_ref(task_ref: str, repo_root: Path) -> Path | None: + """Resolve a task ref to an absolute task directory.""" + normalized = normalize_task_ref(task_ref) + if not normalized: + return None + + path_obj = Path(normalized) + if path_obj.is_absolute(): + return path_obj + + if normalized.startswith(f"{DIR_WORKFLOW}/"): + return repo_root / path_obj + + return repo_root / DIR_WORKFLOW / DIR_TASKS / path_obj + + +def _runtime_sessions_dir(repo_root: Path) -> Path: + return repo_root / DIR_WORKFLOW / DIR_RUNTIME / DIR_SESSIONS + + +def _sanitize_key(raw: str) -> str: + safe = re.sub(r"[^A-Za-z0-9._-]+", "_", raw.strip()) + safe = safe.strip("._-") + return safe[:160] if safe else "" + + +def _hash_value(raw: str) -> str: + return hashlib.sha256(raw.encode("utf-8")).hexdigest()[:24] + + +def _as_dict(value: Any) -> dict[str, Any] | None: + return value if isinstance(value, dict) else None + + +def _string_value(value: Any) -> str | None: + if isinstance(value, str): + stripped = value.strip() + return stripped or None + return None + + +def _lookup_string(data: dict[str, Any], keys: tuple[str, ...]) -> str | None: + for key in keys: + value = _string_value(data.get(key)) + if value: + return value + + for nested_key in _NESTED_KEYS: + nested = _as_dict(data.get(nested_key)) + if not nested: + continue + value = _lookup_string(nested, keys) + if value: + return value + + return None + + +def _detect_platform(platform_input: dict[str, Any] | None, platform: str | None) -> str: + if platform: + return _sanitize_key(platform) or "session" + if platform_input: + for key in ("_trellis_platform", "trellis_platform", "platform", "source"): + value = _string_value(platform_input.get(key)) + if value: + return _sanitize_key(value) or "session" + if _string_value(platform_input.get("cursor_version")): + return "cursor" + return "session" + + +def _context_key(platform_name: str, kind: str, value: str) -> str: + if kind == "transcript": + return f"{platform_name}_transcript_{_hash_value(value)}" + safe_value = _sanitize_key(value) + if safe_value: + return f"{platform_name}_{safe_value}" + return f"{platform_name}_{_hash_value(value)}" + + +def _iter_env_keys( + env_keys: tuple[tuple[str, tuple[str, ...]], ...], + platform_name: str | None, +) -> tuple[tuple[str, tuple[str, ...]], ...]: + if not platform_name: + return env_keys + matched = tuple((name, keys) for name, keys in env_keys if name == platform_name) + return matched + + +def _env_platform_name(platform_name: str | None) -> str | None: + if not platform_name or platform_name == "session": + return None + return _ENV_PLATFORM_ALIASES.get(platform_name, platform_name) + + +def _lookup_env_context_key(platform_name: str | None) -> str | None: + """Resolve a context key from platform-provided environment variables. + + Hooks pass `TRELLIS_CONTEXT_ID` to subprocesses they launch, but an AI-run + shell command can only see session identity if the host platform exports it + in the command environment. These names are best-effort adapters; if none + are present, there is no session-scoped active task. + """ + env_platform_name = _env_platform_name(platform_name) + + for name, keys in _iter_env_keys(_ENV_SESSION_KEYS, env_platform_name): + for key in keys: + value = _string_value(os.environ.get(key)) + if value: + return _context_key(name, "session", value) + + for name, keys in _iter_env_keys(_ENV_CONVERSATION_KEYS, env_platform_name): + for key in keys: + value = _string_value(os.environ.get(key)) + if value: + return _context_key(name, "conversation", value) + + for name, keys in _iter_env_keys(_ENV_TRANSCRIPT_KEYS, env_platform_name): + for key in keys: + value = _string_value(os.environ.get(key)) + if value: + return _context_key(name, "transcript", value) + + return None + + +def _find_repo_root_from_cwd() -> Path | None: + current = Path.cwd().resolve() + while True: + if (current / DIR_WORKFLOW).is_dir(): + return current + if current == current.parent: + return None + current = current.parent + + +def _cursor_shell_ticket_dir(repo_root: Path) -> Path: + return repo_root / DIR_WORKFLOW / DIR_RUNTIME / DIR_CURSOR_SHELL + + +def _remove_file(path: Path) -> bool: + try: + path.unlink() + return True + except OSError: + return False + + +def _task_refs_match(left: str | None, right: str | None, repo_root: Path) -> bool: + if not left or not right: + return False + left_path = resolve_task_ref(left, repo_root) + right_path = resolve_task_ref(right, repo_root) + if left_path is not None and right_path is not None: + return left_path == right_path + return normalize_task_ref(left) == normalize_task_ref(right) + + +def _pending_ticket_matches_args(ticket: dict[str, Any], repo_root: Path) -> bool: + if Path(sys.argv[0]).name != "task.py": + return False + args = tuple(sys.argv[1:]) + if not args: + return False + + command_name = args[0] + if command_name not in TASK_SESSION_COMMANDS: + return False + + subcommands = ticket.get("subcommands") + if not isinstance(subcommands, list): + return False + + for subcommand in subcommands: + if not isinstance(subcommand, dict): + continue + if _string_value(subcommand.get("name")) != command_name: + continue + if command_name != "start": + return True + task_ref = args[1] if len(args) > 1 else None + if _task_refs_match(_string_value(subcommand.get("task_ref")), task_ref, repo_root): + return True + + return False + + +def _ticket_is_fresh(ticket: dict[str, Any], ticket_path: Path, now: float) -> bool: + expires_at = ticket.get("expires_at_epoch") + if isinstance(expires_at, (int, float)) and expires_at < now: + _remove_file(ticket_path) + return False + + created_at = ticket.get("created_at_epoch") + if isinstance(created_at, (int, float)): + if now - created_at <= CURSOR_SHELL_TICKET_TTL_SECONDS: + return True + _remove_file(ticket_path) + return False + return True + + +def _ticket_cwd_matches_repo(ticket: dict[str, Any], repo_root: Path) -> bool: + cwd = _string_value(ticket.get("cwd")) + if not cwd: + return True + try: + Path(cwd).resolve().relative_to(repo_root) + except ValueError: + return False + return True + + +def _matching_cursor_ticket_context_key( + ticket_path: Path, + repo_root: Path, + now: float, +) -> str | None: + ticket = _read_json(ticket_path) + if ticket is None or ticket.get("platform") != "cursor": + return None + if not _ticket_is_fresh(ticket, ticket_path, now): + return None + if not _ticket_cwd_matches_repo(ticket, repo_root): + return None + if not _pending_ticket_matches_args(ticket, repo_root): + return None + return _string_value(ticket.get("context_key")) + + +def _lookup_cursor_shell_ticket_context_key() -> str | None: + """Resolve Cursor conversation identity from a short-lived shell ticket. + + Cursor exposes `conversation_id` to `beforeShellExecution`, but does not + export it into the shell command environment. The Cursor hook writes a + short-lived ticket just before `task.py` runs. We accept a ticket only when + the current `task.py` subcommand matches and exactly one fresh context key + matches, which avoids cross-window pointer contamination. + """ + repo_root = _find_repo_root_from_cwd() + if repo_root is None: + return None + + ticket_dir = _cursor_shell_ticket_dir(repo_root) + if not ticket_dir.is_dir(): + return None + + now = time.time() + candidates: set[str] = set() + for ticket_path in ticket_dir.glob("*.json"): + context_key = _matching_cursor_ticket_context_key(ticket_path, repo_root, now) + if context_key: + candidates.add(context_key) + + if len(candidates) == 1: + return next(iter(candidates)) + return None + + +def resolve_context_key( + platform_input: dict[str, Any] | None = None, + platform: str | None = None, +) -> str | None: + """Resolve a stable session/window context key, if one is available. + + `TRELLIS_CONTEXT_ID` is an explicit context-key override used by CLI + scripts and subprocesses. It does not store the task itself. + """ + override = _string_value(os.environ.get("TRELLIS_CONTEXT_ID")) + if override: + return _sanitize_key(override) or _hash_value(override) + + data = _as_dict(platform_input) + platform_name = _detect_platform(data, platform) if data or platform else None + + if data: + session_id = _lookup_string(data, _SESSION_KEYS) + if session_id: + return _context_key(platform_name or "session", "session", session_id) + + conversation_id = _lookup_string(data, _CONVERSATION_KEYS) + if conversation_id: + return _context_key(platform_name or "session", "conversation", conversation_id) + + transcript_path = _lookup_string(data, _TRANSCRIPT_KEYS) + if transcript_path: + return _context_key(platform_name or "session", "transcript", transcript_path) + + env_context_key = _lookup_env_context_key(platform_name) + if env_context_key: + return env_context_key + + if platform_name in (None, "session", "cursor"): + return _lookup_cursor_shell_ticket_context_key() + return None + + +def _read_json(path: Path) -> dict[str, Any] | None: + try: + data = json.loads(path.read_text(encoding="utf-8")) + except (FileNotFoundError, json.JSONDecodeError, OSError): + return None + return data if isinstance(data, dict) else None + + +def _write_json(path: Path, data: dict[str, Any]) -> bool: + try: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text( + json.dumps(data, indent=2, ensure_ascii=False) + "\n", + encoding="utf-8", + ) + return True + except OSError: + return False + + +def _canonical_task_ref(task_path: str, repo_root: Path) -> str | None: + normalized = normalize_task_ref(task_path) + if not normalized: + return None + full_path = resolve_task_ref(normalized, repo_root) + if full_path is None or not full_path.is_dir(): + return None + try: + return full_path.relative_to(repo_root).as_posix() + except ValueError: + return str(full_path) + + +def _active_from_ref( + task_ref: str | None, + repo_root: Path, + source_type: str, + context_key: str | None = None, +) -> ActiveTask | None: + if not task_ref: + return None + resolved = resolve_task_ref(task_ref, repo_root) + stale = resolved is None or not resolved.is_dir() + return ActiveTask(task_ref, source_type, context_key, stale) + + +def _context_path(repo_root: Path, context_key: str) -> Path: + return _runtime_sessions_dir(repo_root) / f"{context_key}.json" + + +def resolve_active_task( + repo_root: Path, + platform_input: dict[str, Any] | None = None, + platform: str | None = None, +) -> ActiveTask: + """Resolve the active task from session runtime state only. + + A stale session task is returned as stale. Missing context identity or a + missing/empty session context falls back to single-session inference: if + exactly one session file exists in the runtime, return its task with + source_type="session-fallback" — covers class-2 platform sub-agents (codex, + copilot, gemini, qoder) that don't inherit the parent's session id. ≥2 + files or 0 files yield ActiveTask(None) — refuses to guess across windows. + """ + context_key = resolve_context_key(platform_input, platform) + if context_key: + context = _read_json(_context_path(repo_root, context_key)) or {} + task_ref = _string_value(context.get("current_task")) + active = _active_from_ref(task_ref, repo_root, "session", context_key) + if active: + return active + + fallback = _resolve_single_session_fallback(repo_root) + if fallback is not None: + return fallback + + return ActiveTask(None, "none", context_key) + + +def _resolve_single_session_fallback(repo_root: Path) -> ActiveTask | None: + """Return the task pointed at by the sole session file, if exactly one exists. + + Used when context-key resolution fails (typical for class-2 platform + sub-agents). Returns None if 0 or ≥2 session files are present — refuses + to pick across windows so 04-21's multi-session isolation contract holds. + """ + sessions_dir = _runtime_sessions_dir(repo_root) + if not sessions_dir.is_dir(): + return None + + session_files = sorted(sessions_dir.glob("*.json")) + if len(session_files) != 1: + return None + + session_file = session_files[0] + context = _read_json(session_file) or {} + task_ref = _string_value(context.get("current_task")) + if not task_ref: + return None + + fallback_key = session_file.stem + return _active_from_ref(task_ref, repo_root, "session-fallback", fallback_key) + + +def _utc_now() -> str: + return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z") + + +def _context_metadata( + platform_input: dict[str, Any] | None, + platform: str | None, + context_key: str | None = None, +) -> dict[str, Any]: + data = _as_dict(platform_input) or {} + platform_name = _detect_platform(data, platform) + if platform_name == "session" and context_key: + prefix = context_key.split("_", 1)[0] + if prefix in _KNOWN_PLATFORMS: + platform_name = prefix + metadata: dict[str, Any] = { + "platform": platform_name, + "last_seen_at": _utc_now(), + } + for key in (*_SESSION_KEYS, *_CONVERSATION_KEYS, *_TRANSCRIPT_KEYS): + value = _lookup_string(data, (key,)) + if value: + metadata[key] = value + return metadata + + +def set_active_task( + task_path: str, + repo_root: Path, + platform_input: dict[str, Any] | None = None, + platform: str | None = None, +) -> ActiveTask | None: + """Set the active task in session scope. + + Returns None when no context key is available; callers should surface a + user-facing error that explains how to provide session identity. + """ + canonical = _canonical_task_ref(task_path, repo_root) + if canonical is None: + return None + + context_key = resolve_context_key(platform_input, platform) + if not context_key: + return None + + context_path = _context_path(repo_root, context_key) + context = _read_json(context_path) or {} + context.update(_context_metadata(platform_input, platform, context_key)) + context["current_task"] = canonical + context.setdefault("current_run", None) + if not _write_json(context_path, context): + return None + return ActiveTask(canonical, "session", context_key) + + +def clear_active_task( + repo_root: Path, + platform_input: dict[str, Any] | None = None, + platform: str | None = None, +) -> ActiveTask: + """Clear the active task by deleting the current session context file.""" + context_key = resolve_context_key(platform_input, platform) + if not context_key: + return ActiveTask(None, "none") + + previous = resolve_active_task(repo_root, platform_input, platform) + context_path = _context_path(repo_root, context_key) + if context_path.is_file(): + _remove_file(context_path) + return previous + + +def clear_task_from_sessions(task_path: str, repo_root: Path) -> int: + """Delete all session runtime files that point at a task.""" + target = _canonical_task_ref(task_path, repo_root) or normalize_task_ref(task_path) + if not target: + return 0 + + cleared = 0 + sessions_dir = _runtime_sessions_dir(repo_root) + if not sessions_dir.is_dir(): + return cleared + + for session_path in sessions_dir.glob("*.json"): + context = _read_json(session_path) or {} + current = _string_value(context.get("current_task")) + if not current: + continue + current_ref = _canonical_task_ref(current, repo_root) or normalize_task_ref(current) + if current_ref != target: + continue + if session_path.is_file() and _remove_file(session_path): + cleared += 1 + + return cleared + + +def get_current_task_source( + repo_root: Path, + platform_input: dict[str, Any] | None = None, + platform: str | None = None, +) -> tuple[str, str | None, str | None]: + """Return (`source_type`, `context_key`, `task_path`) for compatibility.""" + active = resolve_active_task(repo_root, platform_input, platform) + return active.source_type, active.context_key, active.task_path diff --git a/.trellis/scripts/common/cli_adapter.py b/.trellis/scripts/common/cli_adapter.py new file mode 100644 index 0000000..b65f61a --- /dev/null +++ b/.trellis/scripts/common/cli_adapter.py @@ -0,0 +1,811 @@ +""" +CLI Adapter for Multi-Platform Support. + +Abstracts differences between Claude Code, OpenCode, Cursor, iFlow, Codex, Kilo, Kiro Code, Gemini CLI, Antigravity, Windsurf, Qoder, CodeBuddy, GitHub Copilot, Factory Droid, and Pi Agent interfaces. + +Supported platforms: +- claude: Claude Code (default) +- opencode: OpenCode +- cursor: Cursor IDE +- iflow: iFlow CLI +- codex: Codex CLI (skills-based) +- kilo: Kilo CLI +- kiro: Kiro Code (skills-based) +- gemini: Gemini CLI +- antigravity: Antigravity (workflow-based) +- windsurf: Windsurf (workflow-based) +- qoder: Qoder +- codebuddy: CodeBuddy +- copilot: GitHub Copilot (VS Code) +- droid: Factory Droid (commands-based) +- pi: Pi Agent (extension-backed) + +Usage: + from common.cli_adapter import CLIAdapter + + adapter = CLIAdapter("opencode") + cmd = adapter.build_run_command( + agent="dispatch", + session_id="abc123", + prompt="Start the pipeline" + ) +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import ClassVar, Literal + +Platform = Literal[ + "claude", + "opencode", + "cursor", + "iflow", + "codex", + "kilo", + "kiro", + "gemini", + "antigravity", + "windsurf", + "qoder", + "codebuddy", + "copilot", + "droid", + "pi", +] + + +@dataclass +class CLIAdapter: + """Adapter for different AI coding CLI tools.""" + + platform: Platform + + # ========================================================================= + # Agent Name Mapping + # ========================================================================= + + # OpenCode has built-in agents that cannot be overridden + # See: https://github.com/sst/opencode/issues/4271 + # Note: Class-level constant, not a dataclass field + _AGENT_NAME_MAP: ClassVar[dict[Platform, dict[str, str]]] = { + "claude": {}, # No mapping needed + "opencode": { + "plan": "trellis-plan", # 'plan' is built-in in OpenCode + }, + } + + def get_agent_name(self, agent: str) -> str: + """Get platform-specific agent name. + + Args: + agent: Original agent name (e.g., 'plan', 'dispatch') + + Returns: + Platform-specific agent name (e.g., 'trellis-plan' for OpenCode) + """ + mapping = self._AGENT_NAME_MAP.get(self.platform, {}) + return mapping.get(agent, agent) + + # ========================================================================= + # Agent Path + # ========================================================================= + + @property + def config_dir_name(self) -> str: + """Get platform-specific config directory name. + + Returns: + Directory name ('.claude', '.opencode', '.cursor', '.iflow', '.codex', '.kilocode', '.kiro', '.gemini', '.agent', '.windsurf', '.qoder', '.codebuddy', '.github/copilot', '.factory', or '.pi') + """ + if self.platform == "opencode": + return ".opencode" + elif self.platform == "cursor": + return ".cursor" + elif self.platform == "iflow": + return ".iflow" + elif self.platform == "codex": + return ".codex" + elif self.platform == "kilo": + return ".kilocode" + elif self.platform == "kiro": + return ".kiro" + elif self.platform == "gemini": + return ".gemini" + elif self.platform == "antigravity": + return ".agent" + elif self.platform == "windsurf": + return ".windsurf" + elif self.platform == "qoder": + return ".qoder" + elif self.platform == "codebuddy": + return ".codebuddy" + elif self.platform == "copilot": + return ".github/copilot" + elif self.platform == "droid": + return ".factory" + elif self.platform == "pi": + return ".pi" + else: + return ".claude" + + def get_config_dir(self, project_root: Path) -> Path: + """Get platform-specific config directory. + + Args: + project_root: Project root directory + + Returns: + Path to config directory (.claude, .opencode, .cursor, .iflow, .codex, .kilocode, .kiro, .gemini, .agent, .windsurf, .qoder, .codebuddy, .github/copilot, .factory, or .pi) + """ + return project_root / self.config_dir_name + + def get_agent_path(self, agent: str, project_root: Path) -> Path: + """Get path to agent definition file. + + Args: + agent: Agent name (original, before mapping) + project_root: Project root directory + + Returns: + Path to agent definition file (.md for most platforms, .toml for Codex) + """ + mapped_name = self.get_agent_name(agent) + if self.platform == "codex": + return self.get_config_dir(project_root) / "agents" / f"{mapped_name}.toml" + return self.get_config_dir(project_root) / "agents" / f"{mapped_name}.md" + + def get_commands_path(self, project_root: Path, *parts: str) -> Path: + """Get path to commands directory or specific command file. + + Args: + project_root: Project root directory + *parts: Additional path parts (e.g., 'trellis', 'finish-work.md') + + Returns: + Path to commands directory or file + + Note: + Cursor uses prefix naming: .cursor/commands/trellis-<name>.md + Antigravity uses workflow directory: .agent/workflows/<name>.md + Windsurf uses workflow directory: .windsurf/workflows/trellis-<name>.md + Copilot uses prompt files: .github/prompts/<name>.prompt.md + Pi uses prompt templates: .pi/prompts/trellis-<name>.md + Claude/OpenCode use subdirectory: .claude/commands/trellis/<name>.md + """ + if self.platform == "pi": + prompts_dir = self.get_config_dir(project_root) / "prompts" + if not parts: + return prompts_dir + if len(parts) >= 2 and parts[0] == "trellis": + filename = parts[-1] + if filename.endswith(".md"): + filename = filename[:-3] + return prompts_dir / f"trellis-{filename}.md" + return prompts_dir / Path(*parts) + + if self.platform == "windsurf": + workflow_dir = self.get_config_dir(project_root) / "workflows" + if not parts: + return workflow_dir + if len(parts) >= 2 and parts[0] == "trellis": + filename = parts[-1] + return workflow_dir / f"trellis-{filename}" + return workflow_dir / Path(*parts) + + if self.platform in ("antigravity", "kilo"): + workflow_dir = self.get_config_dir(project_root) / "workflows" + if not parts: + return workflow_dir + if len(parts) >= 2 and parts[0] == "trellis": + filename = parts[-1] + return workflow_dir / filename + return workflow_dir / Path(*parts) + + if self.platform == "copilot": + prompts_dir = project_root / ".github" / "prompts" + if not parts: + return prompts_dir + if len(parts) >= 2 and parts[0] == "trellis": + filename = parts[-1] + if filename.endswith(".md"): + filename = filename[:-3] + return prompts_dir / f"{filename}.prompt.md" + return prompts_dir / Path(*parts) + + if not parts: + return self.get_config_dir(project_root) / "commands" + + # Cursor uses prefix naming instead of subdirectory + if self.platform == "cursor" and len(parts) >= 2 and parts[0] == "trellis": + # Convert trellis/<name>.md to trellis-<name>.md + filename = parts[-1] + return ( + self.get_config_dir(project_root) / "commands" / f"trellis-{filename}" + ) + + return self.get_config_dir(project_root) / "commands" / Path(*parts) + + def get_trellis_command_path(self, name: str) -> str: + """Get relative path to a trellis command file. + + Args: + name: Command name without extension (e.g., 'finish-work', 'check') + + Returns: + Relative path string for use in JSONL entries + + Note: + Cursor: .cursor/commands/trellis-<name>.md + Codex: .agents/skills/trellis-<name>/SKILL.md + Kiro: .kiro/skills/trellis-<name>/SKILL.md + Gemini: .gemini/commands/trellis/<name>.toml + Antigravity: .agent/workflows/<name>.md + Windsurf: .windsurf/workflows/trellis-<name>.md + Pi: .pi/prompts/trellis-<name>.md + Others: .{platform}/commands/trellis/<name>.md + """ + if self.platform == "cursor": + return f".cursor/commands/trellis-{name}.md" + elif self.platform == "codex": + # 0.5.0-beta.0 renamed all skill dirs to add the `trellis-` prefix + # (see that release's manifest for the 60+ rename entries). + return f".agents/skills/trellis-{name}/SKILL.md" + elif self.platform == "kiro": + return f".kiro/skills/trellis-{name}/SKILL.md" + elif self.platform == "gemini": + return f".gemini/commands/trellis/{name}.toml" + elif self.platform == "antigravity": + return f".agent/workflows/{name}.md" + elif self.platform == "windsurf": + return f".windsurf/workflows/trellis-{name}.md" + elif self.platform == "kilo": + return f".kilocode/workflows/{name}.md" + elif self.platform == "copilot": + return f".github/prompts/{name}.prompt.md" + elif self.platform == "droid": + return f".factory/commands/trellis/{name}.md" + elif self.platform == "pi": + return f".pi/prompts/trellis-{name}.md" + else: + return f"{self.config_dir_name}/commands/trellis/{name}.md" + + # ========================================================================= + # Environment Variables + # ========================================================================= + + def get_non_interactive_env(self) -> dict[str, str]: + """Get environment variables for non-interactive mode. + + Returns: + Dict of environment variables to set + """ + if self.platform == "opencode": + return {"OPENCODE_NON_INTERACTIVE": "1"} + elif self.platform == "iflow": + return {"IFLOW_NON_INTERACTIVE": "1"} + elif self.platform == "codex": + return {"CODEX_NON_INTERACTIVE": "1"} + elif self.platform == "kiro": + return {"KIRO_NON_INTERACTIVE": "1"} + elif self.platform == "gemini": + return {} # Gemini CLI doesn't have a non-interactive env var + elif self.platform == "antigravity": + return {} + elif self.platform == "windsurf": + return {} + elif self.platform == "qoder": + return {} + elif self.platform == "codebuddy": + return {} + elif self.platform == "copilot": + return {} + elif self.platform == "droid": + return {} + elif self.platform == "pi": + return {} + else: + return {"CLAUDE_NON_INTERACTIVE": "1"} + + # ========================================================================= + # CLI Command Building + # ========================================================================= + + def build_run_command( + self, + agent: str, + prompt: str, + session_id: str | None = None, + skip_permissions: bool = True, + verbose: bool = True, + json_output: bool = True, + ) -> list[str]: + """Build CLI command for running an agent. + + Args: + agent: Agent name (will be mapped if needed) + prompt: Prompt to send to the agent + session_id: Optional session ID (Claude Code only for creation) + skip_permissions: Whether to skip permission prompts + verbose: Whether to enable verbose output + json_output: Whether to use JSON output format + + Returns: + List of command arguments + """ + mapped_agent = self.get_agent_name(agent) + + if self.platform == "opencode": + cmd = ["opencode", "run"] + cmd.extend(["--agent", mapped_agent]) + + # Note: OpenCode 'run' mode is non-interactive by default + # No equivalent to Claude Code's --dangerously-skip-permissions + # See: https://github.com/anomalyco/opencode/issues/9070 + + if json_output: + cmd.extend(["--format", "json"]) + + if verbose: + cmd.extend(["--log-level", "DEBUG", "--print-logs"]) + + # Note: OpenCode doesn't support --session-id on creation + # Session ID must be extracted from logs after startup + + cmd.append(prompt) + + elif self.platform == "iflow": + cmd = ["iflow", "-y", "-p"] + cmd.append(f"${mapped_agent} {prompt}") + elif self.platform == "codex": + cmd = ["codex", "exec"] + cmd.append(prompt) + elif self.platform == "kiro": + cmd = ["kiro", "run", prompt] + elif self.platform == "gemini": + cmd = ["gemini"] + cmd.append(prompt) + elif self.platform == "antigravity": + raise ValueError( + "Antigravity workflows are UI slash commands; CLI agent run is not supported." + ) + elif self.platform == "windsurf": + raise ValueError( + "Windsurf workflows are UI slash commands; CLI agent run is not supported." + ) + elif self.platform == "qoder": + cmd = ["qodercli", "-p", prompt] + elif self.platform == "codebuddy": + raise ValueError( + "CodeBuddy does not support non-interactive mode (no CLI agent)" + ) + elif self.platform == "copilot": + raise ValueError( + "GitHub Copilot is IDE-only; CLI agent run is not supported." + ) + elif self.platform == "droid": + raise ValueError( + "Factory Droid CLI agent run is not yet supported." + ) + elif self.platform == "pi": + cmd = ["pi", "-p", prompt] + + else: # claude + cmd = ["claude", "-p"] + cmd.extend(["--agent", mapped_agent]) + + if session_id: + cmd.extend(["--session-id", session_id]) + + if skip_permissions: + cmd.append("--dangerously-skip-permissions") + + if json_output: + cmd.extend(["--output-format", "stream-json"]) + + if verbose: + cmd.append("--verbose") + + cmd.append(prompt) + + return cmd + + def build_resume_command(self, session_id: str) -> list[str]: + """Build CLI command for resuming a session. + + Args: + session_id: Session ID to resume (ignored for iFlow) + + Returns: + List of command arguments + """ + if self.platform == "opencode": + return ["opencode", "run", "--session", session_id] + elif self.platform == "iflow": + # iFlow uses -c to continue most recent conversation + # session_id is ignored as iFlow doesn't support session IDs + return ["iflow", "-c"] + elif self.platform == "codex": + return ["codex", "resume", session_id] + elif self.platform == "kiro": + return ["kiro", "resume", session_id] + elif self.platform == "gemini": + return ["gemini", "--resume", session_id] + elif self.platform == "antigravity": + raise ValueError( + "Antigravity workflows are UI slash commands; CLI resume is not supported." + ) + elif self.platform == "windsurf": + raise ValueError( + "Windsurf workflows are UI slash commands; CLI resume is not supported." + ) + elif self.platform == "qoder": + return ["qodercli", "--resume", session_id] + elif self.platform == "codebuddy": + raise ValueError( + "CodeBuddy does not support non-interactive mode (no CLI agent)" + ) + elif self.platform == "copilot": + raise ValueError( + "GitHub Copilot is IDE-only; CLI resume is not supported." + ) + elif self.platform == "droid": + raise ValueError( + "Factory Droid CLI resume is not yet supported." + ) + elif self.platform == "pi": + return ["pi", "-c", session_id] + else: + return ["claude", "--resume", session_id] + + def get_resume_command_str(self, session_id: str, cwd: str | None = None) -> str: + """Get human-readable resume command string. + + Args: + session_id: Session ID to resume + cwd: Optional working directory to cd into + + Returns: + Command string for display + """ + cmd = self.build_resume_command(session_id) + cmd_str = " ".join(cmd) + + if cwd: + return f"cd {cwd} && {cmd_str}" + return cmd_str + + # ========================================================================= + # Platform Detection Helpers + # ========================================================================= + + @property + def is_opencode(self) -> bool: + """Check if platform is OpenCode.""" + return self.platform == "opencode" + + @property + def is_claude(self) -> bool: + """Check if platform is Claude Code.""" + return self.platform == "claude" + + @property + def is_cursor(self) -> bool: + """Check if platform is Cursor.""" + return self.platform == "cursor" + + @property + def is_iflow(self) -> bool: + """Check if platform is iFlow CLI.""" + return self.platform == "iflow" + + @property + def cli_name(self) -> str: + """Get CLI executable name. + + Note: Cursor doesn't have a CLI tool, returns None-like value. + """ + if self.is_opencode: + return "opencode" + elif self.is_cursor: + return "cursor" # Note: Cursor is IDE-only, no CLI + elif self.platform == "iflow": + return "iflow" + elif self.platform == "kiro": + return "kiro" + elif self.platform == "gemini": + return "gemini" + elif self.platform == "antigravity": + return "agy" + elif self.platform == "windsurf": + return "windsurf" + elif self.platform == "qoder": + return "qodercli" + elif self.platform == "codebuddy": + return "codebuddy" + elif self.platform == "copilot": + return "copilot" + elif self.platform == "droid": + return "droid" + elif self.platform == "pi": + return "pi" + else: + return "claude" + + @property + def supports_cli_agents(self) -> bool: + """Check if platform supports running agents via CLI. + + Claude Code, OpenCode, iFlow, and Codex support CLI agent execution. + Cursor is IDE-only and doesn't support CLI agents. + """ + return self.platform in ("claude", "opencode", "iflow", "codex", "pi") + + @property + def requires_agent_definition_file(self) -> bool: + """Check if platform requires an agent definition file (.md/.toml) to run. + + Claude Code, OpenCode, iFlow: require agent .md files (--agent flag). + Codex: auto-discovers agents from .codex/agents/*.toml, no --agent flag. + """ + return self.platform in ("claude", "opencode", "iflow") + + # ========================================================================= + # Session ID Handling + # ========================================================================= + + @property + def supports_session_id_on_create(self) -> bool: + """Check if platform supports specifying session ID on creation. + + Claude Code: Yes (--session-id) + OpenCode: No (auto-generated, extract from logs) + iFlow: No (no session ID support) + """ + return self.platform == "claude" + + def extract_session_id_from_log(self, log_content: str) -> str | None: + """Extract session ID from log output (OpenCode only). + + OpenCode generates session IDs in format: ses_xxx + + Args: + log_content: Log file content + + Returns: + Session ID if found, None otherwise + """ + import re + + # OpenCode session ID pattern + match = re.search(r"ses_[a-zA-Z0-9]+", log_content) + if match: + return match.group(0) + return None + + +# ============================================================================= +# Factory Function +# ============================================================================= + + +def get_cli_adapter(platform: str = "claude") -> CLIAdapter: + """Get CLI adapter for the specified platform. + + Args: + platform: Platform name ('claude', 'opencode', 'cursor', 'iflow', 'codex', 'kilo', 'kiro', 'gemini', 'antigravity', 'windsurf', 'qoder', 'codebuddy', 'copilot', 'droid', or 'pi') + + Returns: + CLIAdapter instance + + Raises: + ValueError: If platform is not supported + """ + if platform not in ( + "claude", + "opencode", + "cursor", + "iflow", + "codex", + "kilo", + "kiro", + "gemini", + "antigravity", + "windsurf", + "qoder", + "codebuddy", + "copilot", + "droid", + "pi", + ): + raise ValueError( + f"Unsupported platform: {platform} (must be 'claude', 'opencode', 'cursor', 'iflow', 'codex', 'kilo', 'kiro', 'gemini', 'antigravity', 'windsurf', 'qoder', 'codebuddy', 'copilot', 'droid', or 'pi')" + ) + + return CLIAdapter(platform=platform) # type: ignore + + +_ALL_PLATFORM_CONFIG_DIRS = ( + ".claude", + ".cursor", + ".iflow", + ".opencode", + ".codex", + ".kilocode", + ".kiro", + ".gemini", + ".agent", + ".windsurf", + ".qoder", + ".codebuddy", + ".github/copilot", + ".factory", + ".pi", +) +"""Platform-specific config directory names used by detect_platform exclusion +checks. `.agents/skills/` is NOT listed here: it is a shared cross-platform +layer (written by Codex, also consumed by Amp/Cline/Warp/etc. via the +agentskills.io standard), not a single-platform signal. Its presence must not +block detection of Kiro, Antigravity, Windsurf, or other platforms.""" + + +def _has_other_platform_dir(project_root: Path, exclude: set[str]) -> bool: + """Check if any platform config dir exists besides those in *exclude*.""" + return any( + (project_root / d).is_dir() + for d in _ALL_PLATFORM_CONFIG_DIRS + if d not in exclude + ) + + +def detect_platform(project_root: Path) -> Platform: + """Auto-detect platform based on existing config directories. + + Detection order: + 1. TRELLIS_PLATFORM environment variable (if set) + 2. .opencode directory exists → opencode + 3. .iflow directory exists → iflow + 4. .cursor directory exists (without .claude) → cursor + 5. .codex exists and no other platform dirs → codex + 6. .kilocode directory exists → kilo + 7. .kiro/skills exists and no other platform dirs → kiro + 8. .gemini directory exists → gemini + 9. .agent/workflows exists and no other platform dirs → antigravity + 10. .windsurf/workflows exists and no other platform dirs → windsurf + 11. .codebuddy directory exists → codebuddy + 12. .qoder directory exists → qoder + 13. .pi directory exists → pi + 14. Default → claude + + Args: + project_root: Project root directory + + Returns: + Detected platform ('claude', 'opencode', 'cursor', 'iflow', 'codex', 'kilo', 'kiro', 'gemini', 'antigravity', 'windsurf', 'qoder', 'codebuddy', 'copilot', 'droid', 'pi', or default 'claude') + """ + import os + + # Check environment variable first + env_platform = os.environ.get("TRELLIS_PLATFORM", "").lower() + if env_platform in ( + "claude", + "opencode", + "cursor", + "iflow", + "codex", + "kilo", + "kiro", + "gemini", + "antigravity", + "windsurf", + "qoder", + "codebuddy", + "copilot", + "droid", + "pi", + ): + return env_platform # type: ignore + + # Check for .opencode directory (OpenCode-specific) + if (project_root / ".opencode").is_dir(): + return "opencode" + + # Check for .iflow directory (iFlow-specific) + if (project_root / ".iflow").is_dir(): + return "iflow" + + # Check for .cursor directory (Cursor-specific) + # Only detect as cursor if .claude doesn't exist (to avoid confusion) + if (project_root / ".cursor").is_dir() and not (project_root / ".claude").is_dir(): + return "cursor" + + # Check for .gemini directory (Gemini CLI-specific) + if (project_root / ".gemini").is_dir(): + return "gemini" + + # Check for .codex directory (Codex-specific) + # .agents/skills/ alone does NOT trigger codex detection (it's a shared standard) + if (project_root / ".codex").is_dir() and not _has_other_platform_dir( + project_root, {".codex", ".agents"} + ): + return "codex" + + # Check for .kilocode directory (Kilo-specific) + if (project_root / ".kilocode").is_dir(): + return "kilo" + + # Check for Kiro skills directory only when no other platform config exists + if (project_root / ".kiro" / "skills").is_dir() and not _has_other_platform_dir( + project_root, {".kiro"} + ): + return "kiro" + + # Check for Antigravity workflow directory only when no other platform config exists + if ( + project_root / ".agent" / "workflows" + ).is_dir() and not _has_other_platform_dir( + project_root, {".agent", ".gemini"} + ): + return "antigravity" + + # Check for Windsurf workflow directory only when no other platform config exists + if ( + project_root / ".windsurf" / "workflows" + ).is_dir() and not _has_other_platform_dir( + project_root, {".windsurf"} + ): + return "windsurf" + + # Check for .codebuddy directory (CodeBuddy-specific) + if (project_root / ".codebuddy").is_dir(): + return "codebuddy" + + # Check for .qoder directory (Qoder-specific) + if (project_root / ".qoder").is_dir(): + return "qoder" + + # Check for .github/copilot directory (GitHub Copilot-specific) + if (project_root / ".github" / "copilot").is_dir(): + return "copilot" + + # Check for .factory directory (Factory Droid-specific) + if (project_root / ".factory").is_dir(): + return "droid" + + # Check for .pi directory (Pi Agent-specific) + if (project_root / ".pi").is_dir(): + return "pi" + + # Fallback: checkout only has the Codex shared-skills layer + # (.agents/skills/trellis-* dirs) and no explicit platform config dir. + # Happens on fresh clones where .codex/ is gitignored/absent but the + # shared skills were committed to git. Must guard against the case + # where .claude/ or any other platform dir also exists — .agents/skills/ + # can legitimately coexist with any platform as a shared consumption + # layer for Amp/Cline/Warp/etc. + agents_skills = project_root / ".agents" / "skills" + if agents_skills.is_dir() and not _has_other_platform_dir( + project_root, set() + ): + try: + for entry in agents_skills.iterdir(): + if entry.is_dir() and entry.name.startswith("trellis-"): + return "codex" + except OSError: + pass + + return "claude" + + +def get_cli_adapter_auto(project_root: Path) -> CLIAdapter: + """Get CLI adapter with auto-detected platform. + + Args: + project_root: Project root directory + + Returns: + CLIAdapter instance for detected platform + """ + platform = detect_platform(project_root) + return CLIAdapter(platform=platform) diff --git a/.trellis/scripts/common/config.py b/.trellis/scripts/common/config.py new file mode 100644 index 0000000..93df643 --- /dev/null +++ b/.trellis/scripts/common/config.py @@ -0,0 +1,445 @@ +#!/usr/bin/env python3 +""" +Trellis configuration reader. + +Reads settings from .trellis/config.yaml with sensible defaults. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +from .paths import DIR_WORKFLOW, get_repo_root + + +# ============================================================================= +# YAML Simple Parser (no dependencies) +# ============================================================================= + + +def _unquote(s: str) -> str: + """Remove exactly one layer of matching surrounding quotes. + + Unlike str.strip('"'), this only removes the outermost pair, + preserving any nested quotes inside the value. + + Examples: + _unquote('"hello"') -> 'hello' + _unquote("'hello'") -> 'hello' + _unquote('"echo \\'hi\\'"') -> "echo 'hi'" + _unquote('hello') -> 'hello' + _unquote('"hello\\'') -> '"hello\\'' (mismatched, unchanged) + """ + if len(s) >= 2 and s[0] == s[-1] and s[0] in ('"', "'"): + return s[1:-1] + return s + + +def _strip_inline_comment(value: str) -> str: + """Strip ` # …` inline comments while preserving `#` inside quoted strings. + + YAML treats ` #` (space-hash) as a comment opener; bare `#` inside a token + is part of the value. Quoted strings are immune. + + Mirrors :func:`common.trellis_config._strip_inline_comment` so both + parsers handle ``key: value # comment`` identically. + """ + in_quote: str | None = None + for idx, ch in enumerate(value): + if in_quote: + if ch == in_quote: + in_quote = None + continue + if ch in ('"', "'"): + in_quote = ch + continue + if ch == "#" and (idx == 0 or value[idx - 1].isspace()): + return value[:idx] + return value + + +def parse_simple_yaml(content: str) -> dict: + """Parse simple YAML with nested dict support (no dependencies). + + Supports: + - key: value (string) + - key: (followed by list items) + - item1 + - item2 + - key: (followed by nested dict) + nested_key: value + nested_key2: + - item + + Uses indentation to detect nesting (2+ spaces deeper = child). + + Args: + content: YAML content string. + + Returns: + Parsed dict (values can be str, list[str], or dict). + """ + lines = content.splitlines() + result: dict = {} + _parse_yaml_block(lines, 0, 0, result) + return result + + +def _parse_yaml_block( + lines: list[str], start: int, min_indent: int, target: dict +) -> int: + """Parse a YAML block into target dict, returning next line index.""" + i = start + current_list: list | None = None + + while i < len(lines): + line = lines[i] + stripped = line.strip() + + # Skip empty lines and comments + if not stripped or stripped.startswith("#"): + i += 1 + continue + + # Calculate indentation + indent = len(line) - len(line.lstrip()) + + # If dedented past our block, we're done + if indent < min_indent: + break + + if stripped.startswith("- "): + if current_list is not None: + current_list.append(_unquote(stripped[2:].strip())) + i += 1 + elif ":" in stripped: + key, _, value = stripped.partition(":") + key = key.strip() + value = _strip_inline_comment(value).strip() + value = _unquote(value) + current_list = None + + if value: + # key: value + target[key] = value + i += 1 + else: + # key: (no value) — peek ahead to determine list vs nested dict + next_i, next_line = _next_content_line(lines, i + 1) + if next_i >= len(lines): + target[key] = {} + i = next_i + elif next_line.strip().startswith("- "): + # It's a list + current_list = [] + target[key] = current_list + i += 1 + else: + next_indent = len(next_line) - len(next_line.lstrip()) + if next_indent > indent: + # It's a nested dict + nested: dict = {} + target[key] = nested + i = _parse_yaml_block(lines, i + 1, next_indent, nested) + else: + # Empty value, same or less indent follows + target[key] = {} + i += 1 + else: + i += 1 + + return i + + +def _next_content_line(lines: list[str], start: int) -> tuple[int, str]: + """Find the next non-empty, non-comment line.""" + i = start + while i < len(lines): + stripped = lines[i].strip() + if stripped and not stripped.startswith("#"): + return i, lines[i] + i += 1 + return i, "" + + +# Defaults +DEFAULT_SESSION_COMMIT_MESSAGE = "chore: record journal" +DEFAULT_MAX_JOURNAL_LINES = 2000 +DEFAULT_SESSION_AUTO_COMMIT = True + +CONFIG_FILE = "config.yaml" + + +def _is_true_config_value(value: object) -> bool: + """Return True when a config value represents an enabled flag.""" + if isinstance(value, bool): + return value + if isinstance(value, str): + return value.strip().lower() == "true" + return False + + +def _get_config_path(repo_root: Path | None = None) -> Path: + """Get path to config.yaml.""" + root = repo_root or get_repo_root() + return root / DIR_WORKFLOW / CONFIG_FILE + + +def _load_config(repo_root: Path | None = None) -> dict: + """Load and parse config.yaml. Returns empty dict on any error.""" + config_file = _get_config_path(repo_root) + try: + content = config_file.read_text(encoding="utf-8") + return parse_simple_yaml(content) + except (OSError, IOError): + return {} + + +def get_session_commit_message(repo_root: Path | None = None) -> str: + """Get the commit message for auto-committing session records.""" + config = _load_config(repo_root) + return config.get("session_commit_message", DEFAULT_SESSION_COMMIT_MESSAGE) + + +def get_max_journal_lines(repo_root: Path | None = None) -> int: + """Get the maximum lines per journal file.""" + config = _load_config(repo_root) + value = config.get("max_journal_lines", DEFAULT_MAX_JOURNAL_LINES) + try: + return int(value) + except (ValueError, TypeError): + return DEFAULT_MAX_JOURNAL_LINES + + +def get_session_auto_commit(repo_root: Path | None = None) -> bool: + """Whether scripts should auto-stage + auto-commit session/task changes. + + Governs both ``add_session.py:_auto_commit_workspace`` and + ``task_store.py:_auto_commit_archive``. + + Default: ``True`` (existing behavior — auto-stage + auto-commit). + Set ``session_auto_commit: false`` in ``.trellis/config.yaml`` to skip + auto-staging entirely; the journal/archive files are still written to + disk, but the user manages ``git add`` / ``git commit`` themselves. + + Accepts native YAML booleans (``true`` / ``false``) and the string + aliases ``true / false / yes / no / 1 / 0 / on / off`` (case-insensitive). + Invalid values fall back to ``True`` with a stderr warning. + """ + config = _load_config(repo_root) + raw = config.get("session_auto_commit", DEFAULT_SESSION_AUTO_COMMIT) + if isinstance(raw, bool): + return raw + s = str(raw).strip().lower() + if s in ("true", "yes", "1", "on"): + return True + if s in ("false", "no", "0", "off"): + return False + print( + f"[WARN] invalid session_auto_commit value: {raw!r}; using true (default)", + file=sys.stderr, + ) + return DEFAULT_SESSION_AUTO_COMMIT + + +def get_hooks(event: str, repo_root: Path | None = None) -> list[str]: + """Get hook commands for a lifecycle event. + + Args: + event: Event name (e.g. "after_create", "after_archive"). + repo_root: Repository root path. + + Returns: + List of shell commands to execute, empty if none configured. + """ + config = _load_config(repo_root) + hooks = config.get("hooks") + if not isinstance(hooks, dict): + return [] + commands = hooks.get(event) + if isinstance(commands, list): + return [str(c) for c in commands] + return [] + + +# ============================================================================= +# Monorepo / Packages +# ============================================================================= + + +def get_packages(repo_root: Path | None = None) -> dict[str, dict] | None: + """Get monorepo package declarations. + + Returns: + Dict mapping package name to its config (path, type, etc.), + or None if not configured (single-repo mode). + + Example return: + {"cli": {"path": "packages/cli"}, "docs-site": {"path": "docs-site", "type": "submodule"}} + """ + config = _load_config(repo_root) + packages = config.get("packages") + if not isinstance(packages, dict): + return None + # Ensure each value is a dict (filter out scalar entries) + filtered = {k: v for k, v in packages.items() if isinstance(v, dict)} + if not filtered: + return None + return filtered + + +def get_default_package(repo_root: Path | None = None) -> str | None: + """Get the default package name from config. + + Returns: + Package name string, or None if not configured. + """ + config = _load_config(repo_root) + value = config.get("default_package") + return str(value) if value else None + + +def get_submodule_packages(repo_root: Path | None = None) -> dict[str, str]: + """Get packages that are git submodules. + + Returns: + Dict mapping package name to its path for submodule-type packages. + Empty dict if none configured. + + Example return: + {"docs-site": "docs-site"} + """ + packages = get_packages(repo_root) + if packages is None: + return {} + return { + name: cfg.get("path", name) + for name, cfg in packages.items() + if cfg.get("type") == "submodule" + } + + +def get_git_packages(repo_root: Path | None = None) -> dict[str, str]: + """Get packages that have their own independent git repository. + + These are sub-directories with their own .git (not submodules), + marked with ``git: true`` in config.yaml. + + Returns: + Dict mapping package name to its path for git-repo packages. + Empty dict if none configured. + + Example config:: + + packages: + backend: + path: iqs + git: true + + Example return:: + + {"backend": "iqs"} + """ + packages = get_packages(repo_root) + if packages is None: + return {} + return { + name: cfg.get("path", name) + for name, cfg in packages.items() + if _is_true_config_value(cfg.get("git")) + } + + +def is_monorepo(repo_root: Path | None = None) -> bool: + """Check if the project is configured as a monorepo (has packages in config).""" + return get_packages(repo_root) is not None + + +def get_spec_base(package: str | None = None, repo_root: Path | None = None) -> str: + """Get the spec directory base path relative to .trellis/. + + Single-repo: returns "spec" + Monorepo with package: returns "spec/<package>" + Monorepo without package: returns "spec" (caller should specify package) + """ + if package and is_monorepo(repo_root): + return f"spec/{package}" + return "spec" + + +def validate_package(package: str, repo_root: Path | None = None) -> bool: + """Check if a package name is valid in this project. + + Single-repo (no packages configured): always returns True. + Monorepo: returns True only if package exists in config.yaml packages. + """ + packages = get_packages(repo_root) + if packages is None: + return True # Single-repo, no validation needed + return package in packages + + +def resolve_package( + task_package: str | None = None, + repo_root: Path | None = None, +) -> str | None: + """Resolve package from inferred sources with validation. + + Checks in order: task_package → default_package. + Invalid inferred values print a warning to stderr and are skipped. + + Returns: + Resolved package name, or None if no valid package found. + + Note: + CLI --package should be validated separately by the caller + (fail-fast with available packages list on error). + """ + packages = get_packages(repo_root) + if packages is None: + return None # Single-repo, no package needed + + # Try task_package (guard against non-string values from malformed JSON) + if task_package and isinstance(task_package, str): + if task_package in packages: + return task_package + print( + f"Warning: task.json package '{task_package}' not found in config, skipping", + file=sys.stderr, + ) + + # Try default_package + default = get_default_package(repo_root) + if default: + if default in packages: + return default + print( + f"Warning: default_package '{default}' not found in config, skipping", + file=sys.stderr, + ) + + return None + + +def get_spec_scope(repo_root: Path | None = None) -> list[str] | str | None: + """Get session.spec_scope configuration. + + Returns: + list[str]: Package names to include in spec scanning. + str: "active_task" to use current task's package. + None: No scope configured (scan all packages). + """ + config = _load_config(repo_root) + session = config.get("session") + if not isinstance(session, dict): + return None + + scope = session.get("spec_scope") + if scope is None: + return None + if isinstance(scope, str): + return scope # e.g. "active_task" + if isinstance(scope, list): + return [str(s) for s in scope] + return None diff --git a/.trellis/scripts/common/developer.py b/.trellis/scripts/common/developer.py new file mode 100644 index 0000000..f422778 --- /dev/null +++ b/.trellis/scripts/common/developer.py @@ -0,0 +1,190 @@ +#!/usr/bin/env python3 +""" +Developer management utilities. + +Provides: + init_developer - Initialize developer + ensure_developer - Ensure developer is initialized (exit if not) + show_developer_info - Show developer information +""" + +from __future__ import annotations + +import sys +from datetime import datetime +from pathlib import Path + +from .paths import ( + DIR_WORKFLOW, + DIR_WORKSPACE, + DIR_TASKS, + FILE_DEVELOPER, + FILE_JOURNAL_PREFIX, + get_repo_root, + get_developer, + check_developer, +) + + +# ============================================================================= +# Developer Initialization +# ============================================================================= + +def init_developer(name: str, repo_root: Path | None = None) -> bool: + """Initialize developer. + + Creates: + - .trellis/.developer file with developer info + - .trellis/workspace/<name>/ directory structure + - Initial journal file and index.md + + Args: + name: Developer name. + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + True on success, False on error. + """ + if not name: + print("Error: developer name is required", file=sys.stderr) + return False + + if repo_root is None: + repo_root = get_repo_root() + + dev_file = repo_root / DIR_WORKFLOW / FILE_DEVELOPER + workspace_dir = repo_root / DIR_WORKFLOW / DIR_WORKSPACE / name + + # Create .developer file + initialized_at = datetime.now().isoformat() + try: + dev_file.write_text( + f"name={name}\ninitialized_at={initialized_at}\n", + encoding="utf-8" + ) + except (OSError, IOError) as e: + print(f"Error: Failed to create .developer file: {e}", file=sys.stderr) + return False + + # Create workspace directory structure + try: + workspace_dir.mkdir(parents=True, exist_ok=True) + except (OSError, IOError) as e: + print(f"Error: Failed to create workspace directory: {e}", file=sys.stderr) + return False + + # Create initial journal file + journal_file = workspace_dir / f"{FILE_JOURNAL_PREFIX}1.md" + if not journal_file.exists(): + today = datetime.now().strftime("%Y-%m-%d") + journal_content = f"""# Journal - {name} (Part 1) + +> AI development session journal +> Started: {today} + +--- + +""" + try: + journal_file.write_text(journal_content, encoding="utf-8") + except (OSError, IOError) as e: + print(f"Error: Failed to create journal file: {e}", file=sys.stderr) + return False + + # Create index.md with markers for auto-update + index_file = workspace_dir / "index.md" + if not index_file.exists(): + index_content = f"""# Workspace Index - {name} + +> Journal tracking for AI development sessions. + +--- + +## Current Status + +<!-- @@@auto:current-status --> +- **Active File**: `journal-1.md` +- **Total Sessions**: 0 +- **Last Active**: - +<!-- @@@/auto:current-status --> + +--- + +## Active Documents + +<!-- @@@auto:active-documents --> +| File | Lines | Status | +|------|-------|--------| +| `journal-1.md` | ~0 | Active | +<!-- @@@/auto:active-documents --> + +--- + +## Session History + +<!-- @@@auto:session-history --> +| # | Date | Title | Commits | Branch | +|---|------|-------|---------|--------| +<!-- @@@/auto:session-history --> + +--- + +## Notes + +- Sessions are appended to journal files +- New journal file created when current exceeds 2000 lines +- Use `add_session.py` to record sessions +""" + try: + index_file.write_text(index_content, encoding="utf-8") + except (OSError, IOError) as e: + print(f"Error: Failed to create index.md: {e}", file=sys.stderr) + return False + + print(f"Developer initialized: {name}") + print(f" .developer file: {dev_file}") + print(f" Workspace dir: {workspace_dir}") + + return True + + +def ensure_developer(repo_root: Path | None = None) -> None: + """Ensure developer is initialized, exit if not. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + """ + if repo_root is None: + repo_root = get_repo_root() + + if not check_developer(repo_root): + print("Error: Developer not initialized.", file=sys.stderr) + print(f"Run: python ./{DIR_WORKFLOW}/scripts/init_developer.py <your-name>", file=sys.stderr) + sys.exit(1) + + +def show_developer_info(repo_root: Path | None = None) -> None: + """Show developer information. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + """ + if repo_root is None: + repo_root = get_repo_root() + + developer = get_developer(repo_root) + + if not developer: + print("Developer: (not initialized)") + else: + print(f"Developer: {developer}") + print(f"Workspace: {DIR_WORKFLOW}/{DIR_WORKSPACE}/{developer}/") + print(f"Tasks: {DIR_WORKFLOW}/{DIR_TASKS}/") + + +# ============================================================================= +# Main Entry (for testing) +# ============================================================================= + +if __name__ == "__main__": + show_developer_info() diff --git a/.trellis/scripts/common/git.py b/.trellis/scripts/common/git.py new file mode 100644 index 0000000..c4bf29f --- /dev/null +++ b/.trellis/scripts/common/git.py @@ -0,0 +1,31 @@ +""" +Git command execution utility. + +Single source of truth for running git commands across all Trellis scripts. +""" + +from __future__ import annotations + +import subprocess +from pathlib import Path + + +def run_git(args: list[str], cwd: Path | None = None) -> tuple[int, str, str]: + """Run a git command and return (returncode, stdout, stderr). + + Uses UTF-8 encoding with -c i18n.logOutputEncoding=UTF-8 to ensure + consistent output across all platforms (Windows, macOS, Linux). + """ + try: + git_args = ["git", "-c", "i18n.logOutputEncoding=UTF-8"] + args + result = subprocess.run( + git_args, + cwd=cwd, + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + return result.returncode, result.stdout, result.stderr + except Exception as e: + return 1, "", str(e) diff --git a/.trellis/scripts/common/git_context.py b/.trellis/scripts/common/git_context.py new file mode 100644 index 0000000..23fc6ec --- /dev/null +++ b/.trellis/scripts/common/git_context.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Git and Session Context utilities. + +Entry shim — delegates to session_context and packages_context. + +Provides: + output_json - Output context in JSON format + output_text - Output context in text format +""" + +from __future__ import annotations + +import json + +from .git import run_git +from .session_context import ( + get_context_json, + get_context_text, + get_context_record_json, + get_context_text_record, + output_json, + output_text, +) +from .packages_context import ( + get_context_packages_text, + get_context_packages_json, +) +from .trellis_config import read_trellis_config +from .workflow_phase import ( + filter_platform, + get_phase_index, + get_step, + resolve_effective_platform, +) + +# Backward-compatible alias — external modules import this name +_run_git_command = run_git + + +# ============================================================================= +# Main Entry +# ============================================================================= + +def main() -> None: + """CLI entry point.""" + import argparse + + parser = argparse.ArgumentParser(description="Get Session Context for AI Agent") + parser.add_argument( + "--json", + "-j", + action="store_true", + help="Output in JSON format (works with any --mode)", + ) + parser.add_argument( + "--mode", + "-m", + choices=["default", "record", "packages", "phase"], + default="default", + help="Output mode: default (full context), record (for record-session), packages (package info only), phase (workflow step extraction)", + ) + parser.add_argument( + "--step", + help="Step id for --mode phase, e.g. 1.1, 2.2. Omit to get the Phase Index.", + ) + parser.add_argument( + "--platform", + help="Platform name for --mode phase, e.g. cursor, claude-code. Filters platform-tagged blocks.", + ) + + args = parser.parse_args() + + if args.mode == "record": + if args.json: + print(json.dumps(get_context_record_json(), indent=2, ensure_ascii=False)) + else: + print(get_context_text_record()) + elif args.mode == "packages": + if args.json: + print(json.dumps(get_context_packages_json(), indent=2, ensure_ascii=False)) + else: + print(get_context_packages_text()) + elif args.mode == "phase": + content = get_step(args.step) if args.step else get_phase_index() + if not content.strip(): + if args.step: + parser.exit(2, f"Step not found: {args.step}\n") + else: + parser.exit(2, "Phase Index section not found in workflow.md\n") + if args.platform: + effective = resolve_effective_platform( + args.platform, read_trellis_config() + ) + content = filter_platform(content, effective) + print(content, end="") + else: + if args.json: + output_json() + else: + output_text() + + +if __name__ == "__main__": + main() diff --git a/.trellis/scripts/common/io.py b/.trellis/scripts/common/io.py new file mode 100644 index 0000000..44288f4 --- /dev/null +++ b/.trellis/scripts/common/io.py @@ -0,0 +1,37 @@ +""" +JSON file I/O utilities. + +Provides read_json and write_json as the single source of truth +for JSON file operations across all Trellis scripts. +""" + +from __future__ import annotations + +import json +from pathlib import Path + + +def read_json(path: Path) -> dict | None: + """Read and parse a JSON file. + + Returns None if the file doesn't exist, is invalid JSON, or can't be read. + """ + try: + return json.loads(path.read_text(encoding="utf-8")) + except (FileNotFoundError, json.JSONDecodeError, OSError): + return None + + +def write_json(path: Path, data: dict) -> bool: + """Write dict to JSON file with pretty formatting. + + Returns True on success, False on error. + """ + try: + path.write_text( + json.dumps(data, indent=2, ensure_ascii=False), + encoding="utf-8", + ) + return True + except (OSError, IOError): + return False diff --git a/.trellis/scripts/common/log.py b/.trellis/scripts/common/log.py new file mode 100644 index 0000000..839c643 --- /dev/null +++ b/.trellis/scripts/common/log.py @@ -0,0 +1,45 @@ +""" +Terminal output utilities: colors and structured logging. + +Single source of truth for Colors and log_* functions +used across all Trellis scripts. +""" + +from __future__ import annotations + + +class Colors: + """ANSI color codes for terminal output.""" + + RED = "\033[0;31m" + GREEN = "\033[0;32m" + YELLOW = "\033[1;33m" + BLUE = "\033[0;34m" + CYAN = "\033[0;36m" + DIM = "\033[2m" + NC = "\033[0m" # No Color / Reset + + +def colored(text: str, color: str) -> str: + """Apply ANSI color to text.""" + return f"{color}{text}{Colors.NC}" + + +def log_info(msg: str) -> None: + """Print info-level message with [INFO] prefix.""" + print(f"{Colors.BLUE}[INFO]{Colors.NC} {msg}") + + +def log_success(msg: str) -> None: + """Print success message with [SUCCESS] prefix.""" + print(f"{Colors.GREEN}[SUCCESS]{Colors.NC} {msg}") + + +def log_warn(msg: str) -> None: + """Print warning message with [WARN] prefix.""" + print(f"{Colors.YELLOW}[WARN]{Colors.NC} {msg}") + + +def log_error(msg: str) -> None: + """Print error message with [ERROR] prefix.""" + print(f"{Colors.RED}[ERROR]{Colors.NC} {msg}") diff --git a/.trellis/scripts/common/packages_context.py b/.trellis/scripts/common/packages_context.py new file mode 100644 index 0000000..e7d4e8c --- /dev/null +++ b/.trellis/scripts/common/packages_context.py @@ -0,0 +1,238 @@ +#!/usr/bin/env python3 +""" +Package discovery and context output. + +Provides: + get_packages_info - Get structured package info + get_packages_section - Build PACKAGES text section + get_context_packages_text - Full packages text output (--mode packages) + get_context_packages_json - Full packages JSON output (--mode packages --json) +""" + +from __future__ import annotations + +from pathlib import Path + +from .config import _is_true_config_value, get_default_package, get_packages, get_spec_scope +from .paths import ( + DIR_SPEC, + DIR_WORKFLOW, + get_current_task, + get_repo_root, +) +from .tasks import load_task + + +# ============================================================================= +# Internal Helpers +# ============================================================================= + +def _scan_spec_layers(spec_dir: Path, package: str | None = None) -> list[str]: + """Scan spec directory for available layers (subdirectories). + + For monorepo: scans spec/<package>/ + For single-repo: scans spec/ + """ + target = spec_dir / package if package else spec_dir + if not target.is_dir(): + return [] + return sorted( + d.name for d in target.iterdir() if d.is_dir() and d.name != "guides" + ) + + +def _get_active_task_package(repo_root: Path) -> str | None: + """Get the package field from the active task's task.json.""" + current = get_current_task(repo_root) + if not current: + return None + ct = load_task(repo_root / current) + return ct.package if ct and ct.package else None + + +def _resolve_scope_set( + packages: dict, + spec_scope, + task_pkg: str | None, + default_pkg: str | None, +) -> set | None: + """Resolve spec_scope to a set of allowed package names, or None for full scan.""" + if not packages: + return None + + if spec_scope is None: + return None + + if isinstance(spec_scope, str) and spec_scope == "active_task": + if task_pkg and task_pkg in packages: + return {task_pkg} + if default_pkg and default_pkg in packages: + return {default_pkg} + return None + + if isinstance(spec_scope, list): + valid = {e for e in spec_scope if e in packages} + if valid: + return valid + # All invalid: fallback + if task_pkg and task_pkg in packages: + return {task_pkg} + if default_pkg and default_pkg in packages: + return {default_pkg} + return None + + return None + + +# ============================================================================= +# Public Functions +# ============================================================================= + +def get_packages_info(repo_root: Path) -> list[dict]: + """Get structured package info for monorepo projects. + + Returns list of dicts with keys: name, path, type, default, specLayers, + isSubmodule, isGitRepo. + Returns empty list for single-repo projects. + """ + packages = get_packages(repo_root) + if not packages: + return [] + + default_pkg = get_default_package(repo_root) + spec_dir = repo_root / DIR_WORKFLOW / DIR_SPEC + result = [] + + for pkg_name, pkg_config in packages.items(): + pkg_path = pkg_config.get("path", pkg_name) if isinstance(pkg_config, dict) else str(pkg_config) + pkg_type = pkg_config.get("type", "local") if isinstance(pkg_config, dict) else "local" + pkg_git = pkg_config.get("git", False) if isinstance(pkg_config, dict) else False + layers = _scan_spec_layers(spec_dir, pkg_name) + + result.append({ + "name": pkg_name, + "path": pkg_path, + "type": pkg_type, + "default": pkg_name == default_pkg, + "specLayers": layers, + "isSubmodule": pkg_type == "submodule", + "isGitRepo": _is_true_config_value(pkg_git), + }) + + return result + + +def get_packages_section(repo_root: Path) -> str: + """Build the PACKAGES section for text output.""" + spec_dir = repo_root / DIR_WORKFLOW / DIR_SPEC + pkg_info = get_packages_info(repo_root) + + lines: list[str] = [] + lines.append("## PACKAGES") + + if not pkg_info: + lines.append("(single-repo mode)") + layers = _scan_spec_layers(spec_dir) + if layers: + lines.append(f"Spec layers: {', '.join(layers)}") + return "\n".join(lines) + + default_pkg = get_default_package(repo_root) + + for pkg in pkg_info: + layers_str = f" [{', '.join(pkg['specLayers'])}]" if pkg["specLayers"] else "" + submodule_tag = " (submodule)" if pkg["isSubmodule"] else "" + git_repo_tag = " (git repo)" if pkg["isGitRepo"] else "" + default_tag = " *" if pkg["default"] else "" + lines.append( + f"- {pkg['name']:<16} {pkg['path']:<20}{layers_str}{submodule_tag}{git_repo_tag}{default_tag}" + ) + + if default_pkg: + lines.append(f"Default package: {default_pkg}") + + return "\n".join(lines) + + +def get_context_packages_text(repo_root: Path | None = None) -> str: + """Get packages context as formatted text (for --mode packages).""" + if repo_root is None: + repo_root = get_repo_root() + + pkg_info = get_packages_info(repo_root) + lines: list[str] = [] + + if not pkg_info: + spec_dir = repo_root / DIR_WORKFLOW / DIR_SPEC + lines.append("Single-repo project (no packages configured)") + lines.append("") + layers = _scan_spec_layers(spec_dir) + if layers: + lines.append(f"Spec layers: {', '.join(layers)}") + return "\n".join(lines) + + # Resolve scope for annotations + packages_dict = get_packages(repo_root) or {} + default_pkg = get_default_package(repo_root) + spec_scope = get_spec_scope(repo_root) + task_pkg = _get_active_task_package(repo_root) + scope_set = _resolve_scope_set(packages_dict, spec_scope, task_pkg, default_pkg) + + lines.append("## PACKAGES") + lines.append("") + for pkg in pkg_info: + default_tag = " (default)" if pkg["default"] else "" + type_tag = f" [{pkg['type']}]" if pkg["type"] != "local" else "" + git_tag = " [git repo]" if pkg["isGitRepo"] else "" + + # Scope annotation + scope_tag = "" + if scope_set is not None and pkg["name"] not in scope_set: + scope_tag = " (out of scope)" + + lines.append(f"### {pkg['name']}{default_tag}{type_tag}{git_tag}{scope_tag}") + lines.append(f"Path: {pkg['path']}") + if pkg["specLayers"]: + lines.append(f"Spec layers: {', '.join(pkg['specLayers'])}") + for layer in pkg["specLayers"]: + lines.append(f" - .trellis/spec/{pkg['name']}/{layer}/index.md") + else: + lines.append("Spec: not configured") + lines.append("") + + # Also show shared guides + guides_dir = repo_root / DIR_WORKFLOW / DIR_SPEC / "guides" + if guides_dir.is_dir(): + lines.append("### Shared Guides (always included)") + lines.append("Path: .trellis/spec/guides/index.md") + lines.append("") + + return "\n".join(lines) + + +def get_context_packages_json(repo_root: Path | None = None) -> dict: + """Get packages context as a dictionary (for --mode packages --json).""" + if repo_root is None: + repo_root = get_repo_root() + + pkg_info = get_packages_info(repo_root) + + if not pkg_info: + spec_dir = repo_root / DIR_WORKFLOW / DIR_SPEC + layers = _scan_spec_layers(spec_dir) + return { + "mode": "single-repo", + "specLayers": layers, + } + + default_pkg = get_default_package(repo_root) + spec_scope = get_spec_scope(repo_root) + task_pkg = _get_active_task_package(repo_root) + + return { + "mode": "monorepo", + "packages": pkg_info, + "defaultPackage": default_pkg, + "specScope": spec_scope, + "activeTaskPackage": task_pkg, + } diff --git a/.trellis/scripts/common/paths.py b/.trellis/scripts/common/paths.py new file mode 100644 index 0000000..1c5a58e --- /dev/null +++ b/.trellis/scripts/common/paths.py @@ -0,0 +1,447 @@ +#!/usr/bin/env python3 +""" +Common path utilities for Trellis workflow. + +Provides: + get_repo_root - Get repository root directory + get_developer - Get developer name + get_workspace_dir - Get developer workspace directory + get_tasks_dir - Get tasks directory + get_active_journal_file - Get current journal file +""" + +from __future__ import annotations + +import re +from datetime import datetime +from pathlib import Path + + +# ============================================================================= +# Path Constants (change here to rename directories) +# ============================================================================= + +# Directory names +DIR_WORKFLOW = ".trellis" +DIR_WORKSPACE = "workspace" +DIR_TASKS = "tasks" +DIR_ARCHIVE = "archive" +DIR_SPEC = "spec" +DIR_SCRIPTS = "scripts" + +# File names +FILE_DEVELOPER = ".developer" +FILE_CURRENT_TASK = ".current-task" +FILE_TASK_JSON = "task.json" +FILE_JOURNAL_PREFIX = "journal-" + + +# ============================================================================= +# Repository Root +# ============================================================================= + +def get_repo_root(start_path: Path | None = None) -> Path: + """Find the nearest directory containing .trellis/ folder. + + This handles nested git repos correctly (e.g., test project inside another repo). + + Args: + start_path: Starting directory to search from. Defaults to current directory. + + Returns: + Path to repository root, or current directory if no .trellis/ found. + """ + current = (start_path or Path.cwd()).resolve() + + while current != current.parent: + if (current / DIR_WORKFLOW).is_dir(): + return current + current = current.parent + + # Fallback to current directory if no .trellis/ found + return Path.cwd().resolve() + + +# ============================================================================= +# Developer +# ============================================================================= + +def get_developer(repo_root: Path | None = None) -> str | None: + """Get developer name from .developer file. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Developer name or None if not initialized. + """ + if repo_root is None: + repo_root = get_repo_root() + + dev_file = repo_root / DIR_WORKFLOW / FILE_DEVELOPER + + if not dev_file.is_file(): + return None + + try: + content = dev_file.read_text(encoding="utf-8") + for line in content.splitlines(): + if line.startswith("name="): + return line.split("=", 1)[1].strip() + except (OSError, IOError): + pass + + return None + + +def check_developer(repo_root: Path | None = None) -> bool: + """Check if developer is initialized. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + True if developer is initialized. + """ + return get_developer(repo_root) is not None + + +# ============================================================================= +# Tasks Directory +# ============================================================================= + +def get_tasks_dir(repo_root: Path | None = None) -> Path: + """Get tasks directory path. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Path to tasks directory. + """ + if repo_root is None: + repo_root = get_repo_root() + return repo_root / DIR_WORKFLOW / DIR_TASKS + + +# ============================================================================= +# Workspace Directory +# ============================================================================= + +def get_workspace_dir(repo_root: Path | None = None) -> Path | None: + """Get developer workspace directory. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Path to workspace directory or None if developer not set. + """ + if repo_root is None: + repo_root = get_repo_root() + + developer = get_developer(repo_root) + if developer: + return repo_root / DIR_WORKFLOW / DIR_WORKSPACE / developer + return None + + +# ============================================================================= +# Journal File +# ============================================================================= + +def get_active_journal_file(repo_root: Path | None = None) -> Path | None: + """Get the current active journal file. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Path to active journal file or None if not found. + """ + if repo_root is None: + repo_root = get_repo_root() + + workspace_dir = get_workspace_dir(repo_root) + if workspace_dir is None or not workspace_dir.is_dir(): + return None + + latest: Path | None = None + highest = 0 + + for f in workspace_dir.glob(f"{FILE_JOURNAL_PREFIX}*.md"): + if not f.is_file(): + continue + + # Extract number from filename + name = f.stem # e.g., "journal-1" + match = re.search(r"(\d+)$", name) + if match: + num = int(match.group(1)) + if num > highest: + highest = num + latest = f + + return latest + + +def count_lines(file_path: Path) -> int: + """Count lines in a file. + + Args: + file_path: Path to file. + + Returns: + Number of lines, or 0 if file doesn't exist. + """ + if not file_path.is_file(): + return 0 + + try: + return len(file_path.read_text(encoding="utf-8").splitlines()) + except (OSError, IOError): + return 0 + + +# ============================================================================= +# Current Task Management +# ============================================================================= + +def normalize_task_ref(task_ref: str) -> str: + """Normalize a task ref for stable runtime storage. + + Stored refs should prefer repo-relative POSIX paths like + `.trellis/tasks/03-27-my-task`, even on Windows. Absolute paths are preserved + unless they can later be converted back to repo-relative form by callers. + """ + normalized = task_ref.strip() + if not normalized: + return "" + + path_obj = Path(normalized) + if path_obj.is_absolute(): + return str(path_obj) + + normalized = normalized.replace("\\", "/") + while normalized.startswith("./"): + normalized = normalized[2:] + + if normalized.startswith(f"{DIR_TASKS}/"): + return f"{DIR_WORKFLOW}/{normalized}" + + return normalized + + +def resolve_task_ref(task_ref: str, repo_root: Path | None = None) -> Path | None: + """Resolve a task ref to an absolute task directory path.""" + if repo_root is None: + repo_root = get_repo_root() + + normalized = normalize_task_ref(task_ref) + if not normalized: + return None + + path_obj = Path(normalized) + if path_obj.is_absolute(): + return path_obj + + if normalized.startswith(f"{DIR_WORKFLOW}/"): + return repo_root / path_obj + + return repo_root / DIR_WORKFLOW / DIR_TASKS / path_obj + + +def get_current_task( + repo_root: Path | None = None, + platform_input: dict | None = None, + platform: str | None = None, +) -> str | None: + """Get current task directory path (relative to repo_root). + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Relative path to current task directory or None. + """ + if repo_root is None: + repo_root = get_repo_root() + + from .active_task import resolve_active_task + + return resolve_active_task(repo_root, platform_input, platform).task_path + + +def get_current_task_abs( + repo_root: Path | None = None, + platform_input: dict | None = None, + platform: str | None = None, +) -> Path | None: + """Get current task directory absolute path. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Absolute path to current task directory or None. + """ + if repo_root is None: + repo_root = get_repo_root() + + relative = get_current_task(repo_root, platform_input, platform) + if relative: + return resolve_task_ref(relative, repo_root) + return None + + +def get_current_task_source( + repo_root: Path | None = None, + platform_input: dict | None = None, + platform: str | None = None, +) -> tuple[str, str | None, str | None]: + """Get active task source as (`source`, `context_key`, `task_path`).""" + if repo_root is None: + repo_root = get_repo_root() + + from .active_task import get_current_task_source as _get_source + + return _get_source(repo_root, platform_input, platform) + + +def set_current_task( + task_path: str, + repo_root: Path | None = None, + platform_input: dict | None = None, + platform: str | None = None, +) -> bool: + """Set current task in session scope. + + Args: + task_path: Task directory path (relative to repo_root). + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + True on success, False on error. + """ + if repo_root is None: + repo_root = get_repo_root() + + from .active_task import set_active_task + + return set_active_task( + task_path, + repo_root, + platform_input=platform_input, + platform=platform, + ) is not None + + +def clear_current_task( + repo_root: Path | None = None, + platform_input: dict | None = None, + platform: str | None = None, +) -> bool: + """Clear current task in session scope. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + True on success. + """ + if repo_root is None: + repo_root = get_repo_root() + + from .active_task import clear_active_task + + clear_active_task( + repo_root, + platform_input=platform_input, + platform=platform, + ) + return True + + +def has_current_task(repo_root: Path | None = None) -> bool: + """Check if has current task. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + True if current task is set. + """ + return get_current_task(repo_root) is not None + + +# ============================================================================= +# Task ID Generation +# ============================================================================= + +def generate_task_date_prefix() -> str: + """Generate task ID based on date (MM-DD format). + + Returns: + Date prefix string (e.g., "01-21"). + """ + return datetime.now().strftime("%m-%d") + + +# ============================================================================= +# Monorepo / Package Paths +# ============================================================================= + + +def get_spec_dir(package: str | None = None, repo_root: Path | None = None) -> Path: + """Get the spec directory path. + + Single-repo: .trellis/spec + Monorepo with package: .trellis/spec/<package> + + Uses lazy import to avoid circular dependency with config.py. + """ + if repo_root is None: + repo_root = get_repo_root() + + from .config import get_spec_base + + base = get_spec_base(package, repo_root) + return repo_root / DIR_WORKFLOW / base + + +def get_package_path(package: str, repo_root: Path | None = None) -> Path | None: + """Get a package's source directory absolute path from config. + + Returns: + Absolute path to the package directory, or None if not found. + """ + if repo_root is None: + repo_root = get_repo_root() + + from .config import get_packages + + packages = get_packages(repo_root) + if not packages or package not in packages: + return None + + info = packages[package] + if isinstance(info, dict): + rel_path = info.get("path", package) + else: + rel_path = str(info) + + return repo_root / rel_path + + +# ============================================================================= +# Main Entry (for testing) +# ============================================================================= + +if __name__ == "__main__": + repo = get_repo_root() + print(f"Repository root: {repo}") + print(f"Developer: {get_developer(repo)}") + print(f"Tasks dir: {get_tasks_dir(repo)}") + print(f"Workspace dir: {get_workspace_dir(repo)}") + print(f"Journal file: {get_active_journal_file(repo)}") + print(f"Current task: {get_current_task(repo)}") diff --git a/.trellis/scripts/common/safe_commit.py b/.trellis/scripts/common/safe_commit.py new file mode 100644 index 0000000..4174191 --- /dev/null +++ b/.trellis/scripts/common/safe_commit.py @@ -0,0 +1,285 @@ +""" +Safe git-add helpers for Trellis-owned paths. + +Why this module exists +---------------------- +A real user incident: a project's `.gitignore` listed `.trellis/` (company-wide +template / personal habit). When `add_session.py` and `task.py archive` ran +their auto-commit and `git add` failed with `ignored by .gitignore`, the AI +agent driving the workflow "fixed" it by retrying with +`git add -f .trellis/` — which fan-out-included every ignored subtree +(`.trellis/.backup-*/`, `.trellis/worktrees/`, `.trellis/.template-hashes.json`, +`.trellis/.runtime/`), committing 548 files / 83474 lines of caches/backups. + +Design +------ +- Scripts only stage SPECIFIC product paths (journal files, index.md, the + current task dir, the archive dir). Never the whole `.trellis/` tree. +- If plain `git add <specific>` fails with "ignored by", DO NOT retry with + ``-f``. The presence of `.trellis/` in `.gitignore` is treated as user + intent ("keep .trellis/ local-only"). The script warns and skips the + auto-commit; users who want auto-staging can either fix their `.gitignore` + or set ``session_auto_commit: false`` and manage git themselves. +- The warning includes a negative example: ``Do NOT use `git add -f .trellis/` ...`` + so any AI rereading the log doesn't reinvent the bug. + +History note: 0.5.10 introduced an automatic ``git add -f`` retry on the +specific paths. That was reverted in 0.5.11 — auto-forcing into a tree the +user had gitignored violates user intent even when the path list is narrow. +The wider-grain forbidden command stays forbidden, and the narrow-grain auto +``-f`` is gone too. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +from .git import run_git +from .paths import ( + DIR_ARCHIVE, + DIR_TASKS, + DIR_WORKFLOW, + DIR_WORKSPACE, + FILE_JOURNAL_PREFIX, + get_developer, +) + + +# Paths under .trellis/ that must NEVER be auto-staged. Listed here so the +# warning to the user can show concrete subpaths to ignore individually +# instead of ignoring the whole `.trellis/` tree. +TRELLIS_IGNORED_SUBPATHS = ( + ".trellis/.backup-*", + ".trellis/worktrees/", + ".trellis/.template-hashes.json", + ".trellis/.runtime/", + ".trellis/.cache/", +) + + +def safe_trellis_paths_to_add(repo_root: Path) -> list[str]: + """Return the list of repo-relative paths the auto-commit should stage. + + Only includes paths that exist on disk so callers don't pass non-existent + arguments to git. The caller is responsible for `git diff --cached` + checking afterwards. + + Included: + - .trellis/workspace/<developer>/journal-*.md + - .trellis/workspace/<developer>/index.md + - .trellis/tasks/<task-dir>/ (every active task directory) + - .trellis/tasks/archive/ (whole archive subtree, if present) + + Excluded (intentionally — these must not be staged): + - .trellis/.backup-*, .trellis/worktrees/, + .trellis/.template-hashes.json, .trellis/.runtime/, .trellis/.cache/ + """ + paths: list[str] = [] + + # Workspace journal files + index.md + developer = get_developer(repo_root) + if developer: + ws = repo_root / DIR_WORKFLOW / DIR_WORKSPACE / developer + if ws.is_dir(): + for f in sorted(ws.glob(f"{FILE_JOURNAL_PREFIX}*.md")): + if f.is_file(): + paths.append( + f"{DIR_WORKFLOW}/{DIR_WORKSPACE}/{developer}/{f.name}" + ) + index_md = ws / "index.md" + if index_md.is_file(): + paths.append( + f"{DIR_WORKFLOW}/{DIR_WORKSPACE}/{developer}/index.md" + ) + + # Active tasks: each direct child of tasks/ that is a directory and not + # the archive root. The archive subtree is added as a single path below. + tasks_dir = repo_root / DIR_WORKFLOW / DIR_TASKS + if tasks_dir.is_dir(): + for child in sorted(tasks_dir.iterdir()): + if not child.is_dir(): + continue + if child.name == DIR_ARCHIVE: + continue + paths.append(f"{DIR_WORKFLOW}/{DIR_TASKS}/{child.name}") + + archive_dir = tasks_dir / DIR_ARCHIVE + if archive_dir.is_dir(): + paths.append(f"{DIR_WORKFLOW}/{DIR_TASKS}/{DIR_ARCHIVE}") + + return paths + + +def safe_archive_paths_to_add( + repo_root: Path, + task_name: str | None = None, + modified_children: list[str] | None = None, +) -> list[str]: + """Return paths to stage after `task.py archive`. + + Scoped to ONLY the paths the archive operation actually touched: + + - the archive subtree (where the freshly-moved task lives) + - the source task directory (for source-side deletes; caller pairs + this with `git rm --cached` since `git add` won't stage deletes + for a path that no longer exists in the working tree) + - any child task directories whose `task.json` was edited to drop + the archived parent (parent-children relationship update) + + This narrow scope avoids "scope creep" — dirty changes in OTHER + active task dirs (parallel-window edits) are NOT bundled into the + archive commit. Callers handle each kind of change in its own + commit boundary. + + Backwards-compat: with no arguments, the function walks the whole + `.trellis/tasks/` subtree the old way (active tasks + archive). New + callers should always pass `task_name`. + """ + paths: list[str] = [] + tasks_dir = repo_root / DIR_WORKFLOW / DIR_TASKS + if not tasks_dir.is_dir(): + return paths + + archive_dir = tasks_dir / DIR_ARCHIVE + + if task_name is not None: + # Narrow scope — only paths that still exist on disk (so + # `git add` doesn't choke on the moved-away source). The caller + # handles the source-side deletes via `git rm --cached` + # explicitly. + if archive_dir.is_dir(): + paths.append( + f"{DIR_WORKFLOW}/{DIR_TASKS}/{DIR_ARCHIVE}" + ) + for child_name in modified_children or []: + paths.append(f"{DIR_WORKFLOW}/{DIR_TASKS}/{child_name}") + return paths + + # Legacy wide scope (no task_name): preserve old behavior so callers + # that have not been updated keep working. + if archive_dir.is_dir(): + paths.append(f"{DIR_WORKFLOW}/{DIR_TASKS}/{DIR_ARCHIVE}") + for child in sorted(tasks_dir.iterdir()): + if not child.is_dir(): + continue + if child.name == DIR_ARCHIVE: + continue + paths.append(f"{DIR_WORKFLOW}/{DIR_TASKS}/{child.name}") + return paths + + +def _stderr_indicates_ignored(stderr: str) -> bool: + """git add error indicates the path is excluded by .gitignore.""" + if not stderr: + return False + lowered = stderr.lower() + return "ignored by" in lowered + + +def safe_git_add( + paths: list[str], repo_root: Path +) -> tuple[bool, bool, str]: + """Run `git add` on specific paths; never retry with -f. + + Returns ``(success, used_force, stderr)``. The ``used_force`` field is + kept for signature compatibility with the 0.5.10 implementation but is + always ``False`` — we never auto-force. + + Behavior: + - No paths passed → success, no force, empty stderr. + - Plain ``git add -- <paths>`` succeeds → return success. + - Plain fails (any reason — ignored or otherwise) → return failure with + the stderr. Callers should inspect the stderr (see + :func:`print_gitignore_warning`) and skip the auto-commit. + """ + if not paths: + return True, False, "" + + rc, _, err = run_git(["add", "--", *paths], cwd=repo_root) + if rc == 0: + return True, False, "" + return False, False, err + + +def print_gitignore_warning(paths: list[str]) -> None: + """Explain to the user (and any AI reading the log) what to do. + + CRITICAL: includes the negative example + ``Do NOT use `git add -f .trellis/``` — agents reading the warning are + known to invent that command, which fans out to ignored caches/backups. + """ + print( + "[WARN] git add failed because .trellis/ paths are ignored by your .gitignore.", + file=sys.stderr, + ) + print( + "[WARN] Skipping auto-commit. The journal/task files were still written to disk;", + file=sys.stderr, + ) + print( + "[WARN] git was not touched.", + file=sys.stderr, + ) + print("[WARN]", file=sys.stderr) + print( + "[WARN] Trellis manages these specific paths and they should be tracked:", + file=sys.stderr, + ) + if paths: + for p in paths: + print(f"[WARN] {p}", file=sys.stderr) + else: + print( + "[WARN] .trellis/workspace/<developer>/{journal-*.md,index.md}", + file=sys.stderr, + ) + print( + "[WARN] .trellis/tasks/<task-dir>/", + file=sys.stderr, + ) + print( + "[WARN] .trellis/tasks/archive/", + file=sys.stderr, + ) + print("[WARN]", file=sys.stderr) + print( + "[WARN] Recommended: change your .gitignore from `.trellis/` to specific", + file=sys.stderr, + ) + print( + "[WARN] subpaths that should remain ignored, e.g.:", + file=sys.stderr, + ) + for sub in TRELLIS_IGNORED_SUBPATHS: + print(f"[WARN] {sub}", file=sys.stderr) + print("[WARN]", file=sys.stderr) + print( + "[WARN] Or, if you intentionally keep .trellis/ local-only, set in", + file=sys.stderr, + ) + print( + "[WARN] .trellis/config.yaml:", + file=sys.stderr, + ) + print( + "[WARN] session_auto_commit: false", + file=sys.stderr, + ) + print( + "[WARN] so the scripts skip git entirely and you can review / commit", + file=sys.stderr, + ) + print( + "[WARN] manually with `git status` / `git add` / `git commit`.", + file=sys.stderr, + ) + print("[WARN]", file=sys.stderr) + print( + "[WARN] Do NOT use `git add -f .trellis/` — it pulls in backups, worktrees,", + file=sys.stderr, + ) + print( + "[WARN] and runtime caches that should never be committed.", + file=sys.stderr, + ) diff --git a/.trellis/scripts/common/session_context.py b/.trellis/scripts/common/session_context.py new file mode 100644 index 0000000..b4de49a --- /dev/null +++ b/.trellis/scripts/common/session_context.py @@ -0,0 +1,821 @@ +#!/usr/bin/env python3 +""" +Session context generation (default + record modes). + +Provides: + get_context_json - JSON output for default mode + get_context_text - Text output for default mode + get_context_record_json - JSON for record mode + get_context_text_record - Text for record mode + output_json - Print JSON + output_text - Print text +""" + +from __future__ import annotations + +import json +import os +import re +import subprocess +from pathlib import Path + +from .active_task import resolve_context_key +from .config import get_git_packages +from .git import run_git +from .packages_context import get_packages_section +from .tasks import iter_active_tasks, load_task, get_all_statuses, children_progress +from .paths import ( + DIR_SCRIPTS, + DIR_SPEC, + DIR_TASKS, + DIR_WORKFLOW, + DIR_WORKSPACE, + count_lines, + get_active_journal_file, + get_current_task, + get_current_task_source, + get_developer, + get_repo_root, + get_tasks_dir, +) + + +# ============================================================================= +# Helpers +# ============================================================================= + +_PACKAGE_NAME = "@mindfoldhq/trellis" +_UPDATE_CHECK_TIMEOUT_SECONDS = 1.0 +_VERSION_RE = re.compile( + r"^\s*(\d+)(?:\.(\d+))?(?:\.(\d+))?(?:-([0-9A-Za-z.-]+))?\s*$" +) +_VERSION_TOKEN_RE = re.compile(r"\b\d+(?:\.\d+){1,2}(?:-[0-9A-Za-z.-]+)?\b") +_POLYREPO_IGNORED_DIRS = { + "node_modules", + "target", + "dist", + "build", + "out", + "bin", + "obj", + "vendor", + "coverage", + "tmp", + "__pycache__", +} +_POLYREPO_SCAN_MAX_DEPTH = 2 + + +def _is_git_worktree(path: Path) -> bool: + """Return True when path is inside a Git worktree.""" + rc, out, _ = run_git(["rev-parse", "--is-inside-work-tree"], cwd=path) + return rc == 0 and out.strip().lower() == "true" + + +def _parse_recent_commits(log_output: str) -> list[dict]: + """Parse `git log --oneline` output into structured commit entries.""" + commits = [] + for line in log_output.splitlines(): + if not line.strip(): + continue + parts = line.split(" ", 1) + if len(parts) >= 2: + commits.append({"hash": parts[0], "message": parts[1]}) + elif len(parts) == 1: + commits.append({"hash": parts[0], "message": ""}) + return commits + + +def _collect_git_repo_info(name: str, rel_path: str, repo_dir: Path) -> dict | None: + """Collect Git status for one known repository directory.""" + if not (repo_dir / ".git").exists(): + return None + + _, branch_out, _ = run_git(["branch", "--show-current"], cwd=repo_dir) + branch = branch_out.strip() or "unknown" + + _, status_out, _ = run_git(["status", "--porcelain"], cwd=repo_dir) + changes = len([l for l in status_out.splitlines() if l.strip()]) + + _, log_out, _ = run_git(["log", "--oneline", "-5"], cwd=repo_dir) + + return { + "name": name, + "path": rel_path, + "branch": branch, + "isClean": changes == 0, + "uncommittedChanges": changes, + "recentCommits": _parse_recent_commits(log_out), + } + + +def _collect_root_git_info(repo_root: Path) -> dict: + """Collect root Git info without pretending a non-Git root is clean.""" + if not _is_git_worktree(repo_root): + return { + "isRepo": False, + "branch": "", + "isClean": False, + "uncommittedChanges": 0, + "recentCommits": [], + } + + _, branch_out, _ = run_git(["branch", "--show-current"], cwd=repo_root) + branch = branch_out.strip() or "unknown" + + _, status_out, _ = run_git(["status", "--porcelain"], cwd=repo_root) + status_lines = [line for line in status_out.splitlines() if line.strip()] + + _, short_out, _ = run_git(["status", "--short"], cwd=repo_root) + + _, log_out, _ = run_git(["log", "--oneline", "-5"], cwd=repo_root) + + return { + "isRepo": True, + "branch": branch, + "isClean": len(status_lines) == 0, + "uncommittedChanges": len(status_lines), + "statusShort": short_out.splitlines(), + "recentCommits": _parse_recent_commits(log_out), + } + + +def _discover_child_git_repos(repo_root: Path) -> list[tuple[str, str]]: + """Discover child Git repositories using the init-time polyrepo heuristic.""" + found: list[str] = [] + + def is_candidate_dir(path: Path) -> bool: + name = path.name + return not name.startswith(".") and name not in _POLYREPO_IGNORED_DIRS + + def scan(rel_dir: Path, depth: int) -> None: + if depth >= _POLYREPO_SCAN_MAX_DEPTH: + return + abs_dir = repo_root / rel_dir + try: + children = sorted(abs_dir.iterdir(), key=lambda p: p.name) + except OSError: + return + + for child in children: + if not child.is_dir() or not is_candidate_dir(child): + continue + + child_rel = ( + rel_dir / child.name if rel_dir != Path(".") else Path(child.name) + ) + if (child / ".git").exists(): + found.append(child_rel.as_posix()) + continue + scan(child_rel, depth + 1) + + scan(Path("."), 0) + if len(found) < 2: + return [] + return [(path.replace("/", "_"), path) for path in sorted(found)] + + +def _collect_package_git_info( + repo_root: Path, + discover_unconfigured: bool = False, +) -> list[dict]: + """Collect Git status for independent package repositories. + + Packages marked with ``git: true`` in config.yaml are authoritative. + When the Trellis root is not a Git repo and no configured package repos are + available, optionally fall back to the bounded polyrepo child scan. + + Returns: + List of dicts with keys: name, path, branch, isClean, + uncommittedChanges, recentCommits. + Empty list if no git-repo packages are configured. + """ + git_pkgs = get_git_packages(repo_root) + result = [] + for pkg_name, pkg_path in git_pkgs.items(): + pkg_dir = repo_root / pkg_path + info = _collect_git_repo_info(pkg_name, pkg_path, pkg_dir) + if info is not None: + result.append(info) + + if result or not discover_unconfigured: + return result + + discovered = [] + for pkg_name, pkg_path in _discover_child_git_repos(repo_root): + info = _collect_git_repo_info(pkg_name, pkg_path, repo_root / pkg_path) + if info is not None: + discovered.append(info) + return discovered + + +def _append_root_git_context(lines: list[str], root_git_info: dict) -> None: + """Append root Git status without misleading non-Git roots.""" + lines.append("## GIT STATUS") + if not root_git_info["isRepo"]: + lines.append("Root is not a Git repository.") + lines.append("Run Git commands from the package repository paths listed below.") + else: + lines.append(f"Branch: {root_git_info['branch']}") + if root_git_info["isClean"]: + lines.append("Working directory: Clean") + else: + lines.append( + f"Working directory: {root_git_info['uncommittedChanges']} " + "uncommitted change(s)" + ) + lines.append("") + lines.append("Changes:") + for line in root_git_info.get("statusShort", [])[:10]: + lines.append(line) + lines.append("") + + lines.append("## RECENT COMMITS") + if not root_git_info["isRepo"]: + lines.append( + "Root has no Git commit history because it is not a Git repository." + ) + elif root_git_info["recentCommits"]: + for commit in root_git_info["recentCommits"]: + lines.append(f"{commit['hash']} {commit['message']}") + else: + lines.append("(no commits)") + lines.append("") + + +def _append_package_git_context(lines: list[str], package_git_info: list[dict]) -> None: + """Append Git status and recent commits for package repositories.""" + for pkg in package_git_info: + lines.append(f"## GIT STATUS ({pkg['name']}: {pkg['path']})") + lines.append(f"Branch: {pkg['branch']}") + if pkg["isClean"]: + lines.append("Working directory: Clean") + else: + lines.append( + f"Working directory: {pkg['uncommittedChanges']} uncommitted change(s)" + ) + lines.append("") + lines.append(f"## RECENT COMMITS ({pkg['name']}: {pkg['path']})") + if pkg["recentCommits"]: + for commit in pkg["recentCommits"]: + lines.append(f"{commit['hash']} {commit['message']}") + else: + lines.append("(no commits)") + lines.append("") + + +def _read_project_version(repo_root: Path) -> str | None: + try: + version = (repo_root / DIR_WORKFLOW / ".version").read_text( + encoding="utf-8" + ).strip() + except OSError: + return None + return version or None + + +def _fetch_trellis_version_output() -> str | None: + try: + result = subprocess.run( + ["trellis", "--version"], + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=_UPDATE_CHECK_TIMEOUT_SECONDS, + ) + except (OSError, subprocess.SubprocessError, TimeoutError): + return None + + if result.returncode != 0: + return None + output = f"{result.stdout}\n{result.stderr}".strip() + return output or None + + +def _extract_available_update_version(output: str) -> str | None: + update_match = re.search( + r"Trellis update available:\s*" + r"(?P<current>\S+)\s*(?:→|->)\s*(?P<latest>\S+)", + output, + ) + if update_match: + return update_match.group("latest").strip() + candidates = _VERSION_TOKEN_RE.findall(output) + return candidates[-1] if candidates else None + + +def _resolve_available_update_version() -> str | None: + output = _fetch_trellis_version_output() + if not output: + return None + return _extract_available_update_version(output) + + +def _parse_version(version: str) -> tuple[tuple[int, int, int], tuple[str, ...] | None] | None: + match = _VERSION_RE.match(version) + if not match: + return None + major, minor, patch, prerelease = match.groups() + numbers = (int(major), int(minor or "0"), int(patch or "0")) + prerelease_parts = tuple(prerelease.split(".")) if prerelease else None + return numbers, prerelease_parts + + +def _compare_prerelease( + left: tuple[str, ...] | None, + right: tuple[str, ...] | None, +) -> int: + if left is None and right is None: + return 0 + if left is None: + return 1 + if right is None: + return -1 + + for left_part, right_part in zip(left, right): + if left_part == right_part: + continue + left_numeric = left_part.isdigit() + right_numeric = right_part.isdigit() + if left_numeric and right_numeric: + left_int = int(left_part) + right_int = int(right_part) + return (left_int > right_int) - (left_int < right_int) + if left_numeric: + return -1 + if right_numeric: + return 1 + return (left_part > right_part) - (left_part < right_part) + + return (len(left) > len(right)) - (len(left) < len(right)) + + +def _compare_versions(left: str, right: str) -> int | None: + parsed_left = _parse_version(left) + parsed_right = _parse_version(right) + if parsed_left is None or parsed_right is None: + return None + + left_numbers, left_prerelease = parsed_left + right_numbers, right_prerelease = parsed_right + if left_numbers != right_numbers: + return (left_numbers > right_numbers) - (left_numbers < right_numbers) + return _compare_prerelease(left_prerelease, right_prerelease) + + +def _update_marker_path(repo_root: Path) -> Path: + context_key = resolve_context_key() + if not context_key: + terminal_key = os.environ.get("TERM_SESSION_ID", "").strip() + context_key = terminal_key or f"ppid-{os.getppid()}" + safe_key = re.sub(r"[^A-Za-z0-9._-]+", "_", context_key).strip("._-") + if not safe_key: + safe_key = "session" + return ( + repo_root + / DIR_WORKFLOW + / ".runtime" + / f"update-check-{safe_key[:160]}.marker" + ) + + +def _mark_update_check_attempted(repo_root: Path) -> bool: + marker_path = _update_marker_path(repo_root) + if marker_path.exists(): + return False + try: + marker_path.parent.mkdir(parents=True, exist_ok=True) + marker_path.write_text("checked\n", encoding="utf-8") + except OSError: + pass + return True + + +def _get_update_hint(repo_root: Path) -> str | None: + marker_path = _update_marker_path(repo_root) + if marker_path.exists(): + return None + + current_version = _read_project_version(repo_root) + if not current_version: + return None + + latest_version = _resolve_available_update_version() + if not latest_version: + return None + + _mark_update_check_attempted(repo_root) + comparison = _compare_versions(current_version, latest_version) + if comparison is None or comparison >= 0: + return None + + return ( + f"Trellis update available: {current_version} -> {latest_version}, " + "run trellis upgrade" + ) + + +# ============================================================================= +# JSON Output +# ============================================================================= + +def get_context_json(repo_root: Path | None = None) -> dict: + """Get context as a dictionary. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Context dictionary. + """ + if repo_root is None: + repo_root = get_repo_root() + + developer = get_developer(repo_root) + tasks_dir = get_tasks_dir(repo_root) + journal_file = get_active_journal_file(repo_root) + + journal_lines = 0 + journal_relative = "" + if journal_file and developer: + journal_lines = count_lines(journal_file) + journal_relative = ( + f"{DIR_WORKFLOW}/{DIR_WORKSPACE}/{developer}/{journal_file.name}" + ) + + root_git_info = _collect_root_git_info(repo_root) + + # Tasks + tasks = [ + { + "dir": t.dir_name, + "name": t.name, + "status": t.status, + "children": list(t.children), + "parent": t.parent, + } + for t in iter_active_tasks(tasks_dir) + ] + + # Package git repos (independent sub-repositories) + pkg_git_info = _collect_package_git_info( + repo_root, + discover_unconfigured=not root_git_info["isRepo"], + ) + + result = { + "developer": developer or "", + "git": { + "isRepo": root_git_info["isRepo"], + "branch": root_git_info["branch"], + "isClean": root_git_info["isClean"], + "uncommittedChanges": root_git_info["uncommittedChanges"], + "recentCommits": root_git_info["recentCommits"], + }, + "tasks": { + "active": tasks, + "directory": f"{DIR_WORKFLOW}/{DIR_TASKS}", + }, + "journal": { + "file": journal_relative, + "lines": journal_lines, + "nearLimit": journal_lines > 1800, + }, + } + + if pkg_git_info: + result["packageGit"] = pkg_git_info + + return result + + +def output_json(repo_root: Path | None = None) -> None: + """Output context in JSON format. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + """ + context = get_context_json(repo_root) + print(json.dumps(context, indent=2, ensure_ascii=False)) + + +# ============================================================================= +# Text Output +# ============================================================================= + +def get_context_text(repo_root: Path | None = None) -> str: + """Get context as formatted text. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Formatted text output. + """ + if repo_root is None: + repo_root = get_repo_root() + + lines = [] + lines.append("========================================") + lines.append("SESSION CONTEXT") + lines.append("========================================") + lines.append("") + + developer = get_developer(repo_root) + + # Developer section + lines.append("## DEVELOPER") + if not developer: + lines.append( + f"ERROR: Not initialized. Run: python ./{DIR_WORKFLOW}/{DIR_SCRIPTS}/init_developer.py <name>" + ) + return "\n".join(lines) + + lines.append(f"Name: {developer}") + lines.append("") + + root_git_info = _collect_root_git_info(repo_root) + _append_root_git_context(lines, root_git_info) + + # Package git repos — independent sub-repositories + _append_package_git_context( + lines, + _collect_package_git_info( + repo_root, + discover_unconfigured=not root_git_info["isRepo"], + ), + ) + + # Current task + lines.append("## CURRENT TASK") + current_task = get_current_task(repo_root) + if current_task: + current_task_dir = repo_root / current_task + source_type, context_key, _ = get_current_task_source(repo_root) + lines.append(f"Path: {current_task}") + lines.append( + f"Source: {source_type}" + (f":{context_key}" if context_key else "") + ) + + ct = load_task(current_task_dir) + if ct: + lines.append(f"Name: {ct.name}") + lines.append(f"Status: {ct.status}") + lines.append(f"Created: {ct.raw.get('createdAt', 'unknown')}") + if ct.description: + lines.append(f"Description: {ct.description}") + + # Check for prd.md + prd_file = current_task_dir / "prd.md" + if prd_file.is_file(): + lines.append("") + lines.append("[!] This task has prd.md - read it for task details") + else: + lines.append("(none)") + lines.append("") + + # Active tasks + lines.append("## ACTIVE TASKS") + tasks_dir = get_tasks_dir(repo_root) + task_count = 0 + + # Collect all task data for hierarchy display + all_tasks = {t.dir_name: t for t in iter_active_tasks(tasks_dir)} + all_statuses = {name: t.status for name, t in all_tasks.items()} + + def _print_task_tree(name: str, indent: int = 0) -> None: + nonlocal task_count + t = all_tasks[name] + progress = children_progress(t.children, all_statuses) + prefix = " " * indent + lines.append(f"{prefix}- {name}/ ({t.status}){progress} @{t.assignee or '-'}") + task_count += 1 + for child in t.children: + if child in all_tasks: + _print_task_tree(child, indent + 1) + + for dir_name in sorted(all_tasks.keys()): + if not all_tasks[dir_name].parent: + _print_task_tree(dir_name) + + if task_count == 0: + lines.append("(no active tasks)") + lines.append(f"Total: {task_count} active task(s)") + lines.append("") + + # My tasks + lines.append("## MY TASKS (Assigned to me)") + my_task_count = 0 + + for t in all_tasks.values(): + if t.assignee == developer and t.status != "done": + progress = children_progress(t.children, all_statuses) + lines.append(f"- [{t.priority}] {t.title} ({t.status}){progress}") + my_task_count += 1 + + if my_task_count == 0: + lines.append("(no tasks assigned to you)") + lines.append("") + + # Journal file + lines.append("## JOURNAL FILE") + journal_file = get_active_journal_file(repo_root) + if journal_file: + journal_lines = count_lines(journal_file) + relative = f"{DIR_WORKFLOW}/{DIR_WORKSPACE}/{developer}/{journal_file.name}" + lines.append(f"Active file: {relative}") + lines.append(f"Line count: {journal_lines} / 2000") + if journal_lines > 1800: + lines.append("[!] WARNING: Approaching 2000 line limit!") + else: + lines.append("No journal file found") + lines.append("") + + # Packages + packages_text = get_packages_section(repo_root) + if packages_text: + lines.append(packages_text) + lines.append("") + + # Paths + lines.append("## PATHS") + lines.append(f"Workspace: {DIR_WORKFLOW}/{DIR_WORKSPACE}/{developer}/") + lines.append(f"Tasks: {DIR_WORKFLOW}/{DIR_TASKS}/") + lines.append(f"Spec: {DIR_WORKFLOW}/{DIR_SPEC}/") + lines.append("") + + lines.append("========================================") + + return "\n".join(lines) + + +# ============================================================================= +# Record Mode +# ============================================================================= + +def get_context_record_json(repo_root: Path | None = None) -> dict: + """Get record-mode context as a dictionary. + + Focused on: my active tasks, git status, current task. + """ + if repo_root is None: + repo_root = get_repo_root() + + developer = get_developer(repo_root) + tasks_dir = get_tasks_dir(repo_root) + + root_git_info = _collect_root_git_info(repo_root) + + # My tasks (single pass — collect statuses and filter by assignee) + all_tasks_list = list(iter_active_tasks(tasks_dir)) + all_statuses = {t.dir_name: t.status for t in all_tasks_list} + + my_tasks = [] + for t in all_tasks_list: + if t.assignee == developer: + done = sum( + 1 for c in t.children + if all_statuses.get(c) in ("completed", "done") + ) + my_tasks.append({ + "dir": t.dir_name, + "title": t.title, + "status": t.status, + "priority": t.priority, + "children": list(t.children), + "childrenDone": done, + "parent": t.parent, + "meta": t.meta, + }) + + # Current task + current_task_info = None + current_task = get_current_task(repo_root) + if current_task: + source_type, context_key, _ = get_current_task_source(repo_root) + ct = load_task(repo_root / current_task) + if ct: + current_task_info = { + "path": current_task, + "name": ct.name, + "status": ct.status, + "source": source_type, + "contextKey": context_key, + } + + # Package git repos + pkg_git_info = _collect_package_git_info( + repo_root, + discover_unconfigured=not root_git_info["isRepo"], + ) + + result = { + "developer": developer or "", + "git": { + "isRepo": root_git_info["isRepo"], + "branch": root_git_info["branch"], + "isClean": root_git_info["isClean"], + "uncommittedChanges": root_git_info["uncommittedChanges"], + "recentCommits": root_git_info["recentCommits"], + }, + "myTasks": my_tasks, + "currentTask": current_task_info, + } + + if pkg_git_info: + result["packageGit"] = pkg_git_info + + return result + + +def get_context_text_record(repo_root: Path | None = None) -> str: + """Get context as formatted text for record-session mode. + + Focused output: MY ACTIVE TASKS first (with [!!!] emphasis), + then GIT STATUS, RECENT COMMITS, CURRENT TASK. + """ + if repo_root is None: + repo_root = get_repo_root() + + lines: list[str] = [] + lines.append("========================================") + lines.append("SESSION CONTEXT (RECORD MODE)") + lines.append("========================================") + lines.append("") + + developer = get_developer(repo_root) + if not developer: + lines.append( + f"ERROR: Not initialized. Run: python ./{DIR_WORKFLOW}/{DIR_SCRIPTS}/init_developer.py <name>" + ) + return "\n".join(lines) + + # MY ACTIVE TASKS — first and prominent + lines.append(f"## [!!!] MY ACTIVE TASKS (Assigned to {developer})") + lines.append("[!] Review whether any should be archived before recording this session.") + lines.append("") + + tasks_dir = get_tasks_dir(repo_root) + my_task_count = 0 + + # Single pass — collect all tasks and filter by assignee + all_statuses = get_all_statuses(tasks_dir) + + for t in iter_active_tasks(tasks_dir): + if t.assignee == developer: + progress = children_progress(t.children, all_statuses) + lines.append(f"- [{t.priority}] {t.title} ({t.status}){progress} — {t.dir_name}") + my_task_count += 1 + + if my_task_count == 0: + lines.append("(no active tasks assigned to you)") + lines.append("") + + root_git_info = _collect_root_git_info(repo_root) + _append_root_git_context(lines, root_git_info) + + # Package git repos — independent sub-repositories + _append_package_git_context( + lines, + _collect_package_git_info( + repo_root, + discover_unconfigured=not root_git_info["isRepo"], + ), + ) + + # CURRENT TASK + lines.append("## CURRENT TASK") + current_task = get_current_task(repo_root) + if current_task: + source_type, context_key, _ = get_current_task_source(repo_root) + lines.append(f"Path: {current_task}") + lines.append( + f"Source: {source_type}" + (f":{context_key}" if context_key else "") + ) + ct = load_task(repo_root / current_task) + if ct: + lines.append(f"Name: {ct.name}") + lines.append(f"Status: {ct.status}") + else: + lines.append("(none)") + lines.append("") + + lines.append("========================================") + + return "\n".join(lines) + + +def output_text(repo_root: Path | None = None) -> None: + """Output context in text format. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + """ + if repo_root is None: + repo_root = get_repo_root() + update_hint = _get_update_hint(repo_root) + if update_hint: + print(update_hint) + print("") + print(get_context_text(repo_root)) diff --git a/.trellis/scripts/common/task_context.py b/.trellis/scripts/common/task_context.py new file mode 100644 index 0000000..7ffc9f5 --- /dev/null +++ b/.trellis/scripts/common/task_context.py @@ -0,0 +1,223 @@ +#!/usr/bin/env python3 +""" +Task JSONL context management. + +Provides: + cmd_add_context - Add entry to JSONL context file + cmd_validate - Validate JSONL context files + cmd_list_context - List JSONL context entries + +Note: + ``cmd_init_context`` was removed in v0.5.0-beta.12. JSONL context files + are now seeded at ``task.py create`` time with a self-describing + ``_example`` line; the AI agent curates real entries during planning when + the task needs sub-agent/spec context. See ``.trellis/workflow.md`` for the + current planning artifact contract. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +from .log import Colors, colored +from .paths import get_repo_root +from .task_utils import resolve_task_dir + + +# ============================================================================= +# Command: add-context +# ============================================================================= + +def cmd_add_context(args: argparse.Namespace) -> int: + """Add entry to JSONL context file.""" + repo_root = get_repo_root() + target_dir = resolve_task_dir(args.dir, repo_root) + + jsonl_name = args.file + path = args.path + reason = args.reason or "Added manually" + + if not target_dir.is_dir(): + print(colored(f"Error: Directory not found: {target_dir}", Colors.RED)) + return 1 + + # Support shorthand + if not jsonl_name.endswith(".jsonl"): + jsonl_name = f"{jsonl_name}.jsonl" + + jsonl_file = target_dir / jsonl_name + full_path = repo_root / path + + entry_type = "file" + if full_path.is_dir(): + entry_type = "directory" + if not path.endswith("/"): + path = f"{path}/" + elif not full_path.is_file(): + print(colored(f"Error: Path not found: {path}", Colors.RED)) + return 1 + + # Check if already exists + if jsonl_file.is_file(): + content = jsonl_file.read_text(encoding="utf-8") + if f'"{path}"' in content: + print(colored(f"Warning: Entry already exists for {path}", Colors.YELLOW)) + return 0 + + # Add entry + entry: dict + if entry_type == "directory": + entry = {"file": path, "type": "directory", "reason": reason} + else: + entry = {"file": path, "reason": reason} + + with jsonl_file.open("a", encoding="utf-8") as f: + f.write(json.dumps(entry, ensure_ascii=False) + "\n") + + print(colored(f"Added {entry_type}: {path}", Colors.GREEN)) + return 0 + + +# ============================================================================= +# Command: validate +# ============================================================================= + +def cmd_validate(args: argparse.Namespace) -> int: + """Validate JSONL context files.""" + repo_root = get_repo_root() + target_dir = resolve_task_dir(args.dir, repo_root) + + if not target_dir.is_dir(): + print(colored("Error: task directory required", Colors.RED)) + return 1 + + print(colored("=== Validating Context Files ===", Colors.BLUE)) + print(f"Target dir: {target_dir}") + print() + + total_errors = 0 + for jsonl_name in ["implement.jsonl", "check.jsonl"]: + jsonl_file = target_dir / jsonl_name + errors = _validate_jsonl(jsonl_file, repo_root) + total_errors += errors + + print() + if total_errors == 0: + print(colored("✓ All validations passed", Colors.GREEN)) + return 0 + else: + print(colored(f"✗ Validation failed ({total_errors} errors)", Colors.RED)) + return 1 + + +def _validate_jsonl(jsonl_file: Path, repo_root: Path) -> int: + """Validate a single JSONL file. + + Seed rows (no ``file`` field — typically ``{"_example": "..."}``) are + skipped silently; they are self-describing comments, not real entries. + """ + file_name = jsonl_file.name + errors = 0 + + if not jsonl_file.is_file(): + print(f" {colored(f'{file_name}: not found (skipped)', Colors.YELLOW)}") + return 0 + + line_num = 0 + real_entries = 0 + for line in jsonl_file.read_text(encoding="utf-8").splitlines(): + line_num += 1 + if not line.strip(): + continue + + try: + data = json.loads(line) + except json.JSONDecodeError: + print(f" {colored(f'{file_name}:{line_num}: Invalid JSON', Colors.RED)}") + errors += 1 + continue + + file_path = data.get("file") + entry_type = data.get("type", "file") + + if not file_path: + # Seed / comment row — skip silently + continue + + real_entries += 1 + full_path = repo_root / file_path + if entry_type == "directory": + if not full_path.is_dir(): + print(f" {colored(f'{file_name}:{line_num}: Directory not found: {file_path}', Colors.RED)}") + errors += 1 + else: + if not full_path.is_file(): + print(f" {colored(f'{file_name}:{line_num}: File not found: {file_path}', Colors.RED)}") + errors += 1 + + if errors == 0: + print(f" {colored(f'{file_name}: ✓ ({real_entries} entries)', Colors.GREEN)}") + else: + print(f" {colored(f'{file_name}: ✗ ({errors} errors)', Colors.RED)}") + + return errors + + +# ============================================================================= +# Command: list-context +# ============================================================================= + +def cmd_list_context(args: argparse.Namespace) -> int: + """List JSONL context entries.""" + repo_root = get_repo_root() + target_dir = resolve_task_dir(args.dir, repo_root) + + if not target_dir.is_dir(): + print(colored("Error: task directory required", Colors.RED)) + return 1 + + print(colored("=== Context Files ===", Colors.BLUE)) + print() + + for jsonl_name in ["implement.jsonl", "check.jsonl"]: + jsonl_file = target_dir / jsonl_name + if not jsonl_file.is_file(): + continue + + print(colored(f"[{jsonl_name}]", Colors.CYAN)) + + count = 0 + seed_only = True + for line in jsonl_file.read_text(encoding="utf-8").splitlines(): + if not line.strip(): + continue + + try: + data = json.loads(line) + except json.JSONDecodeError: + continue + + file_path = data.get("file") + if not file_path: + # Seed / comment row — don't count as a real entry + continue + seed_only = False + + count += 1 + entry_type = data.get("type", "file") + reason = data.get("reason", "-") + + if entry_type == "directory": + print(f" {colored(f'{count}.', Colors.GREEN)} [DIR] {file_path}") + else: + print(f" {colored(f'{count}.', Colors.GREEN)} {file_path}") + print(f" {colored('→', Colors.YELLOW)} {reason}") + + if seed_only: + print(f" {colored('(no curated entries yet — only seed row)', Colors.YELLOW)}") + + print() + + return 0 diff --git a/.trellis/scripts/common/task_queue.py b/.trellis/scripts/common/task_queue.py new file mode 100644 index 0000000..f7485e2 --- /dev/null +++ b/.trellis/scripts/common/task_queue.py @@ -0,0 +1,188 @@ +#!/usr/bin/env python3 +""" +Task queue utility functions. + +Provides: + list_tasks_by_status - List tasks by status + list_pending_tasks - List tasks with pending status + list_tasks_by_assignee - List tasks by assignee + list_my_tasks - List tasks assigned to current developer + get_task_stats - Get P0/P1/P2/P3 counts +""" + +from __future__ import annotations + +from pathlib import Path + +from .paths import ( + get_repo_root, + get_developer, + get_tasks_dir, +) +from .tasks import iter_active_tasks + + +# ============================================================================= +# Internal helper +# ============================================================================= + +def _task_to_dict(t) -> dict: + """Convert TaskInfo to the dict format callers expect.""" + return { + "priority": t.priority, + "id": t.raw.get("id", ""), + "title": t.title, + "status": t.status, + "assignee": t.assignee or "-", + "dir": t.dir_name, + "children": list(t.children), + "parent": t.parent, + } + + +# ============================================================================= +# Public Functions +# ============================================================================= + +def list_tasks_by_status( + filter_status: str | None = None, + repo_root: Path | None = None +) -> list[dict]: + """List tasks by status. + + Args: + filter_status: Optional status filter. + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + List of task info dicts with keys: priority, id, title, status, assignee. + """ + if repo_root is None: + repo_root = get_repo_root() + + tasks_dir = get_tasks_dir(repo_root) + results = [] + + for t in iter_active_tasks(tasks_dir): + if filter_status and t.status != filter_status: + continue + results.append(_task_to_dict(t)) + + return results + + +def list_pending_tasks(repo_root: Path | None = None) -> list[dict]: + """List pending tasks. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + List of task info dicts. + """ + return list_tasks_by_status("planning", repo_root) + + +def list_tasks_by_assignee( + assignee: str, + filter_status: str | None = None, + repo_root: Path | None = None +) -> list[dict]: + """List tasks assigned to a specific developer. + + Args: + assignee: Developer name. + filter_status: Optional status filter. + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + List of task info dicts. + """ + if repo_root is None: + repo_root = get_repo_root() + + tasks_dir = get_tasks_dir(repo_root) + results = [] + + for t in iter_active_tasks(tasks_dir): + if (t.assignee or "-") != assignee: + continue + if filter_status and t.status != filter_status: + continue + results.append(_task_to_dict(t)) + + return results + + +def list_my_tasks( + filter_status: str | None = None, + repo_root: Path | None = None +) -> list[dict]: + """List tasks assigned to current developer. + + Args: + filter_status: Optional status filter. + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + List of task info dicts. + + Raises: + ValueError: If developer not set. + """ + if repo_root is None: + repo_root = get_repo_root() + + developer = get_developer(repo_root) + if not developer: + raise ValueError("Developer not set") + + return list_tasks_by_assignee(developer, filter_status, repo_root) + + +def get_task_stats(repo_root: Path | None = None) -> dict[str, int]: + """Get task statistics. + + Args: + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Dict with keys: P0, P1, P2, P3, Total. + """ + if repo_root is None: + repo_root = get_repo_root() + + tasks_dir = get_tasks_dir(repo_root) + stats = {"P0": 0, "P1": 0, "P2": 0, "P3": 0, "Total": 0} + + for t in iter_active_tasks(tasks_dir): + if t.priority in stats: + stats[t.priority] += 1 + stats["Total"] += 1 + + return stats + + +def format_task_stats(stats: dict[str, int]) -> str: + """Format task stats as string. + + Args: + stats: Stats dict from get_task_stats. + + Returns: + Formatted string like "P0:0 P1:1 P2:2 P3:0 Total:3". + """ + return f"P0:{stats['P0']} P1:{stats['P1']} P2:{stats['P2']} P3:{stats['P3']} Total:{stats['Total']}" + + +# ============================================================================= +# Main Entry (for testing) +# ============================================================================= + +if __name__ == "__main__": + stats = get_task_stats() + print(format_task_stats(stats)) + print() + print("Pending tasks:") + for task in list_pending_tasks(): + print(f" {task['priority']}|{task['id']}|{task['title']}|{task['status']}|{task['assignee']}") diff --git a/.trellis/scripts/common/task_store.py b/.trellis/scripts/common/task_store.py new file mode 100644 index 0000000..d4b527a --- /dev/null +++ b/.trellis/scripts/common/task_store.py @@ -0,0 +1,746 @@ +#!/usr/bin/env python3 +""" +Task CRUD operations. + +Provides: + ensure_tasks_dir - Ensure tasks directory exists + cmd_create - Create a new task + cmd_archive - Archive completed task + cmd_set_branch - Set git branch for task + cmd_set_base_branch - Set PR target branch + cmd_set_scope - Set scope for PR title + cmd_add_subtask - Link child task to parent + cmd_remove_subtask - Unlink child task from parent +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from datetime import datetime +from pathlib import Path + +from .config import ( + get_packages, + get_session_auto_commit, + is_monorepo, + resolve_package, + validate_package, +) +from .git import run_git +from .io import read_json, write_json +from .log import Colors, colored +from .paths import ( + DIR_ARCHIVE, + DIR_TASKS, + DIR_WORKFLOW, + FILE_TASK_JSON, + generate_task_date_prefix, + get_developer, + get_repo_root, + get_tasks_dir, +) +from .safe_commit import ( + print_gitignore_warning, + safe_archive_paths_to_add, + safe_git_add, +) +from .task_utils import ( + archive_task_complete, + find_task_by_name, + resolve_task_dir, + run_task_hooks, +) + + +# ============================================================================= +# Helper Functions +# ============================================================================= + +def _slugify(title: str) -> str: + """Convert title to slug (only works with ASCII).""" + result = title.lower() + result = re.sub(r"[^a-z0-9]", "-", result) + result = re.sub(r"-+", "-", result) + result = result.strip("-") + return result + + +def ensure_tasks_dir(repo_root: Path) -> Path: + """Ensure tasks directory exists.""" + tasks_dir = get_tasks_dir(repo_root) + archive_dir = tasks_dir / "archive" + + if not tasks_dir.exists(): + tasks_dir.mkdir(parents=True) + print(colored(f"Created tasks directory: {tasks_dir}", Colors.GREEN), file=sys.stderr) + + if not archive_dir.exists(): + archive_dir.mkdir(parents=True) + + return tasks_dir + + +def _find_archived_task_by_dir_name(tasks_dir: Path, dir_name: str) -> Path | None: + """Find an archived task directory with the exact active-task dir name.""" + archive_dir = tasks_dir / DIR_ARCHIVE + if not archive_dir.is_dir(): + return None + + for month_dir in sorted(archive_dir.iterdir()): + if not month_dir.is_dir(): + continue + candidate = month_dir / dir_name + if candidate.is_dir(): + return candidate + + return None + + +def _repo_relative_path(path: Path, repo_root: Path) -> str: + """Format a path relative to the repo root when possible.""" + try: + return path.relative_to(repo_root).as_posix() + except ValueError: + return str(path) + + +# ============================================================================= +# Sub-agent platform detection + JSONL seeding +# ============================================================================= + +# Config directories of platforms that consume implement.jsonl / check.jsonl. +# Keep in sync with src/types/ai-tools.ts AI_TOOLS entries — these are the +# platforms listed in workflow.md's "agent-capable" Skill Routing block +# (Class-1 hook-inject + Class-2 pull-based preludes). Kilo / Antigravity / +# Windsurf are NOT in this list: they do not consume JSONL. +_SUBAGENT_CONFIG_DIRS: tuple[str, ...] = ( + ".claude", + ".cursor", + ".codex", + ".kiro", + ".gemini", + ".opencode", + ".qoder", + ".codebuddy", + ".factory", # Factory Droid + ".github/copilot", + ".pi", # Pi Agent +) + +_SEED_EXAMPLE = ( + "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. " + "Put spec/research files only — no code paths. " + "Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. " + "Delete this line once real entries are added." +) + + +def _has_subagent_platform(repo_root: Path) -> bool: + """Return True if any sub-agent-capable platform is configured. + + Detected by probing well-known config directories at the repo root. Used + only to decide whether ``task.py create`` should seed empty + ``implement.jsonl`` / ``check.jsonl`` files. + """ + for config_dir in _SUBAGENT_CONFIG_DIRS: + if (repo_root / config_dir).is_dir(): + return True + return False + + +def _write_seed_jsonl(path: Path) -> None: + """Write a one-line seed JSONL file with a self-describing ``_example``. + + The seed row has no ``file`` field, so downstream consumers (hooks + + preludes) that iterate entries via ``item.get("file")`` naturally skip + it. The row exists purely as an in-file prompt for the AI curator. + """ + seed = {"_example": _SEED_EXAMPLE} + path.write_text(json.dumps(seed, ensure_ascii=False) + "\n", encoding="utf-8") + + +def _default_prd_content(title: str, description: str | None = None) -> str: + """Return the default PRD skeleton created with every task.""" + goal = (description or "").strip() or "TBD." + heading = title.strip() or "Untitled task" + return f"""# {heading} + +## Goal + +{goal} + +## Requirements + +- TBD + +## Acceptance Criteria + +- [ ] TBD + +## Notes + +- Keep `prd.md` focused on requirements, constraints, and acceptance criteria. +- Lightweight tasks can remain PRD-only. +- For complex tasks, add `design.md` for technical design and `implement.md` for execution planning before `task.py start`. +""" + + +# ============================================================================= +# Command: create +# ============================================================================= + +def cmd_create(args: argparse.Namespace) -> int: + """Create a new task.""" + repo_root = get_repo_root() + + if not args.title: + print(colored("Error: title is required", Colors.RED), file=sys.stderr) + return 1 + + # Validate --package (CLI source: fail-fast) + package: str | None = getattr(args, "package", None) + if not is_monorepo(repo_root): + # Single-repo: ignore --package, no package prefix + if package: + print(colored(f"Warning: --package ignored in single-repo project", Colors.YELLOW), file=sys.stderr) + package = None + elif package: + if not validate_package(package, repo_root): + packages = get_packages(repo_root) + available = ", ".join(sorted(packages.keys())) if packages else "(none)" + print(colored(f"Error: unknown package '{package}'. Available: {available}", Colors.RED), file=sys.stderr) + return 1 + else: + # Inferred: default_package → None (no task.json yet for create) + package = resolve_package(repo_root=repo_root) + + # Default assignee to current developer + assignee = args.assignee + if not assignee: + assignee = get_developer(repo_root) + if not assignee: + print(colored("Error: No developer set. Run init_developer.py first or use --assignee", Colors.RED), file=sys.stderr) + return 1 + + ensure_tasks_dir(repo_root) + + # Get current developer as creator + creator = get_developer(repo_root) or assignee + + # Generate slug if not provided + slug = args.slug or _slugify(args.title) + if not slug: + print(colored("Error: could not generate slug from title", Colors.RED), file=sys.stderr) + return 1 + + # Create task directory with MM-DD-slug format + tasks_dir = get_tasks_dir(repo_root) + date_prefix = generate_task_date_prefix() + dir_name = f"{date_prefix}-{slug}" + task_dir = tasks_dir / dir_name + task_json_path = task_dir / FILE_TASK_JSON + + archived_task_dir = _find_archived_task_by_dir_name(tasks_dir, dir_name) + if archived_task_dir: + print(colored(f"Error: Task already archived: {dir_name}", Colors.RED), file=sys.stderr) + print(f"Archived at: {_repo_relative_path(archived_task_dir, repo_root)}", file=sys.stderr) + print("Use a new slug if you intend to create a new task.", file=sys.stderr) + return 1 + + if task_dir.exists(): + print(colored(f"Warning: Task directory already exists: {dir_name}", Colors.YELLOW), file=sys.stderr) + else: + task_dir.mkdir(parents=True) + + today = datetime.now().strftime("%Y-%m-%d") + + # Record current branch as base_branch (PR target) + _, branch_out, _ = run_git(["branch", "--show-current"], cwd=repo_root) + current_branch = branch_out.strip() or "main" + + task_data = { + "id": slug, + "name": slug, + "title": args.title, + "description": args.description or "", + "status": "planning", + "dev_type": None, + "scope": None, + "package": package, + "priority": args.priority, + "creator": creator, + "assignee": assignee, + "createdAt": today, + "completedAt": None, + "branch": None, + "base_branch": current_branch, + "worktree_path": None, + "commit": None, + "pr_url": None, + "subtasks": [], + "children": [], + "parent": None, + "relatedFiles": [], + "notes": "", + "meta": {}, + } + + write_json(task_json_path, task_data) + + prd_path = task_dir / "prd.md" + if not prd_path.exists(): + prd_path.write_text( + _default_prd_content(args.title, args.description), + encoding="utf-8", + ) + + # Seed implement.jsonl / check.jsonl for sub-agent-capable platforms. + # Agent curates real entries during planning when the task needs them. + # Agent-less platforms (Kilo / Antigravity / Windsurf) skip this — they + # load specs via the trellis-before-dev skill instead of JSONL. + seeded_jsonl = False + if _has_subagent_platform(repo_root): + for jsonl_name in ("implement.jsonl", "check.jsonl"): + jsonl_path = task_dir / jsonl_name + if not jsonl_path.exists(): + _write_seed_jsonl(jsonl_path) + seeded_jsonl = True + + # Handle --parent: establish bidirectional link + if args.parent: + parent_dir = resolve_task_dir(args.parent, repo_root) + parent_json_path = parent_dir / FILE_TASK_JSON + if not parent_json_path.is_file(): + print(colored(f"Warning: Parent task.json not found: {args.parent}", Colors.YELLOW), file=sys.stderr) + else: + parent_data = read_json(parent_json_path) + if parent_data: + # Add child to parent's children list + parent_children = parent_data.get("children", []) + if dir_name not in parent_children: + parent_children.append(dir_name) + parent_data["children"] = parent_children + write_json(parent_json_path, parent_data) + + # Set parent in child's task.json + task_data["parent"] = parent_dir.name + write_json(task_json_path, task_data) + + print(colored(f"Linked as child of: {parent_dir.name}", Colors.GREEN), file=sys.stderr) + + # Auto-activate the new task so the per-turn breadcrumb fires planning + # state. Best-effort: gracefully degrade if no session identity (CLI run + # outside an AI session) — the task is still created, the user can run + # task.py start later. Pointer is session-scoped so this never affects + # other AI sessions. + try: + from .active_task import resolve_context_key, set_active_task + if resolve_context_key(): + try: + rel_dir = task_dir.relative_to(repo_root).as_posix() + except ValueError: + rel_dir = str(task_dir) + set_active_task(rel_dir, repo_root) + except Exception: + pass + + print(colored(f"Created task: {dir_name}", Colors.GREEN), file=sys.stderr) + print("", file=sys.stderr) + print(colored("Next steps:", Colors.BLUE), file=sys.stderr) + print(" - Fill prd.md with requirements and acceptance criteria", file=sys.stderr) + print(" - Lightweight task: PRD-only is valid", file=sys.stderr) + print(" - Complex task: add design.md and implement.md before task.py start", file=sys.stderr) + if seeded_jsonl: + print( + " - Curate implement.jsonl / check.jsonl as spec/research manifests when sub-agents need context", + file=sys.stderr, + ) + print(" - Use /trellis:continue or phase context to decide the next step", file=sys.stderr) + print("", file=sys.stderr) + + # Output relative path for script chaining + print(f"{DIR_WORKFLOW}/{DIR_TASKS}/{dir_name}") + + run_task_hooks("after_create", task_json_path, repo_root) + return 0 + + +# ============================================================================= +# Command: archive +# ============================================================================= + +def cmd_archive(args: argparse.Namespace) -> int: + """Archive completed task.""" + repo_root = get_repo_root() + task_name = args.name + + if not task_name: + print(colored("Error: Task name is required", Colors.RED), file=sys.stderr) + return 1 + + tasks_dir = get_tasks_dir(repo_root) + + # Resolve task directory (supports task name, relative path, or absolute path) + task_dir = resolve_task_dir(task_name, repo_root) + + if not task_dir or not task_dir.is_dir(): + print(colored(f"Error: Task not found: {task_name}", Colors.RED), file=sys.stderr) + print("Active tasks:", file=sys.stderr) + # Import lazily to avoid circular dependency + from .tasks import iter_active_tasks + for t in iter_active_tasks(tasks_dir): + print(f" - {t.dir_name}/", file=sys.stderr) + return 1 + + dir_name = task_dir.name + task_json_path = task_dir / FILE_TASK_JSON + + # Update status before archiving + today = datetime.now().strftime("%Y-%m-%d") + # Names of child task dirs whose task.json gets modified below; passed + # into safe_archive_paths_to_add so they're staged in this commit. + modified_children: list[str] = [] + if task_json_path.is_file(): + data = read_json(task_json_path) + if data: + data["status"] = "completed" + data["completedAt"] = today + write_json(task_json_path, data) + + # Handle subtask relationships on archive. + # Keep this task in its parent's children list so progress + # counters (children_progress) stay consistent — children + # missing from the active set are treated as completed. + task_children = data.get("children", []) + + # If this is a parent, clear parent field in all children + if task_children: + for child_name in task_children: + child_dir_path = find_task_by_name(child_name, tasks_dir) + if child_dir_path: + child_json = child_dir_path / FILE_TASK_JSON + if child_json.is_file(): + child_data = read_json(child_json) + if child_data: + child_data["parent"] = None + write_json(child_json, child_data) + modified_children.append(child_dir_path.name) + + # Clear any session that still points at this task before the path moves. + from .active_task import clear_task_from_sessions + clear_task_from_sessions(str(task_dir), repo_root) + + # Archive + result = archive_task_complete(task_dir, repo_root) + if "archived_to" in result: + archive_dest = Path(result["archived_to"]) + year_month = archive_dest.parent.name + print(colored(f"Archived: {dir_name} -> archive/{year_month}/", Colors.GREEN), file=sys.stderr) + + # Auto-commit unless --no-commit + if not getattr(args, "no_commit", False): + if not _auto_commit_archive(dir_name, repo_root, modified_children): + print( + colored( + "Archive moved on disk, but git auto-commit did not complete. " + "Resolve `git status` before continuing.", + Colors.RED, + ), + file=sys.stderr, + ) + return 1 + + # Return the archive path + print(f"{DIR_WORKFLOW}/{DIR_TASKS}/{DIR_ARCHIVE}/{year_month}/{dir_name}") + + # Run hooks with the archived path + archived_json = archive_dest / FILE_TASK_JSON + run_task_hooks("after_archive", archived_json, repo_root) + return 0 + + return 1 + + +def _auto_commit_archive( + task_name: str, + repo_root: Path, + modified_children: list[str] | None = None, +) -> bool: + """Stage Trellis-owned task paths and commit after archive. + + Scoped narrowly to the archived task's source + destination paths + plus any child task dirs whose ``task.json`` was edited (parent → + children relationship update). Dirty changes in OTHER active task + dirs are NOT bundled into the archive commit. + + If ``.gitignore`` blocks the paths, we warn + skip — we do NOT + retry with ``git add -f``. The warning explicitly forbids + ``git add -f .trellis/`` (which would fan out to caches/backups) + and points users at ``session_auto_commit: false``. + + Honors ``session_auto_commit`` in ``.trellis/config.yaml``: when + set to ``false``, this function returns immediately without + touching git (the archive directory move on disk is unaffected). + """ + if not get_session_auto_commit(repo_root): + print( + "[OK] session_auto_commit: false — skipping git stage/commit.", + file=sys.stderr, + ) + return True + + source_rel = f"{DIR_WORKFLOW}/{DIR_TASKS}/{task_name}" + rc, tracked_out, _ = run_git( + ["ls-files", "--", source_rel], + cwd=repo_root, + ) + source_was_tracked = rc == 0 and bool(tracked_out.strip()) + + paths = safe_archive_paths_to_add( + repo_root, task_name=task_name, modified_children=modified_children + ) + if not paths: + print("[OK] No task changes to commit.", file=sys.stderr) + return True + + success, _, err = safe_git_add(paths, repo_root) + if not success: + if err and "ignored by" in err.lower(): + print_gitignore_warning(paths) + else: + print( + f"[WARN] git add failed: {err.strip() if err else 'unknown error'}", + file=sys.stderr, + ) + return not source_was_tracked + + # Belt-and-suspenders for the phantom-delete bug: `safe_git_add` uses + # `git add` (no -A) which only stages additions/modifications. The + # source task directory was moved away by `shutil.move`, so its files + # need an explicit `git rm --cached` to stage the deletions in this + # same commit — otherwise they sit as uncommitted "phantom deletes" + # against HEAD until something later picks them up. + # + # `--ignore-unmatch` makes this a no-op when the task was never tracked + # (e.g. archiving a task that lived only in working tree). + run_git( + ["rm", "-r", "--cached", "--ignore-unmatch", "--", source_rel], + cwd=repo_root, + ) + + rc, _, _ = run_git( + ["diff", "--cached", "--quiet", "--", *paths, source_rel], + cwd=repo_root, + ) + if rc == 0: + print("[OK] No task changes to commit.", file=sys.stderr) + return True + + commit_msg = f"chore(task): archive {task_name}" + rc, _, err = run_git(["commit", "-m", commit_msg], cwd=repo_root) + if rc == 0: + print(f"[OK] Auto-committed: {commit_msg}", file=sys.stderr) + return True + else: + print(f"[WARN] Auto-commit failed: {err.strip()}", file=sys.stderr) + return not source_was_tracked + + +# ============================================================================= +# Command: add-subtask +# ============================================================================= + +def cmd_add_subtask(args: argparse.Namespace) -> int: + """Link a child task to a parent task.""" + repo_root = get_repo_root() + + parent_dir = resolve_task_dir(args.parent_dir, repo_root) + child_dir = resolve_task_dir(args.child_dir, repo_root) + + parent_json_path = parent_dir / FILE_TASK_JSON + child_json_path = child_dir / FILE_TASK_JSON + + if not parent_json_path.is_file(): + print(colored(f"Error: Parent task.json not found: {args.parent_dir}", Colors.RED), file=sys.stderr) + return 1 + + if not child_json_path.is_file(): + print(colored(f"Error: Child task.json not found: {args.child_dir}", Colors.RED), file=sys.stderr) + return 1 + + parent_data = read_json(parent_json_path) + child_data = read_json(child_json_path) + + if not parent_data or not child_data: + print(colored("Error: Failed to read task.json", Colors.RED), file=sys.stderr) + return 1 + + # Check if child already has a parent + existing_parent = child_data.get("parent") + if existing_parent: + print(colored(f"Error: Child task already has a parent: {existing_parent}", Colors.RED), file=sys.stderr) + return 1 + + # Add child to parent's children list + parent_children = parent_data.get("children", []) + child_dir_name = child_dir.name + if child_dir_name not in parent_children: + parent_children.append(child_dir_name) + parent_data["children"] = parent_children + + # Set parent in child's task.json + child_data["parent"] = parent_dir.name + + # Write both + write_json(parent_json_path, parent_data) + write_json(child_json_path, child_data) + + print(colored(f"Linked: {child_dir.name} -> {parent_dir.name}", Colors.GREEN), file=sys.stderr) + return 0 + + +# ============================================================================= +# Command: remove-subtask +# ============================================================================= + +def cmd_remove_subtask(args: argparse.Namespace) -> int: + """Unlink a child task from a parent task.""" + repo_root = get_repo_root() + + parent_dir = resolve_task_dir(args.parent_dir, repo_root) + child_dir = resolve_task_dir(args.child_dir, repo_root) + + parent_json_path = parent_dir / FILE_TASK_JSON + child_json_path = child_dir / FILE_TASK_JSON + + if not parent_json_path.is_file(): + print(colored(f"Error: Parent task.json not found: {args.parent_dir}", Colors.RED), file=sys.stderr) + return 1 + + if not child_json_path.is_file(): + print(colored(f"Error: Child task.json not found: {args.child_dir}", Colors.RED), file=sys.stderr) + return 1 + + parent_data = read_json(parent_json_path) + child_data = read_json(child_json_path) + + if not parent_data or not child_data: + print(colored("Error: Failed to read task.json", Colors.RED), file=sys.stderr) + return 1 + + # Remove child from parent's children list + parent_children = parent_data.get("children", []) + child_dir_name = child_dir.name + if child_dir_name in parent_children: + parent_children.remove(child_dir_name) + parent_data["children"] = parent_children + + # Clear parent in child's task.json + child_data["parent"] = None + + # Write both + write_json(parent_json_path, parent_data) + write_json(child_json_path, child_data) + + print(colored(f"Unlinked: {child_dir.name} from {parent_dir.name}", Colors.GREEN), file=sys.stderr) + return 0 + + +# ============================================================================= +# Command: set-branch +# ============================================================================= + +def cmd_set_branch(args: argparse.Namespace) -> int: + """Set git branch for task.""" + repo_root = get_repo_root() + target_dir = resolve_task_dir(args.dir, repo_root) + branch = args.branch + + if not branch: + print(colored("Error: Missing arguments", Colors.RED)) + print("Usage: python task.py set-branch <task-dir> <branch-name>") + return 1 + + task_json = target_dir / FILE_TASK_JSON + if not task_json.is_file(): + print(colored(f"Error: task.json not found at {target_dir}", Colors.RED)) + return 1 + + data = read_json(task_json) + if not data: + return 1 + + data["branch"] = branch + write_json(task_json, data) + + print(colored(f"✓ Branch set to: {branch}", Colors.GREEN)) + return 0 + + +# ============================================================================= +# Command: set-base-branch +# ============================================================================= + +def cmd_set_base_branch(args: argparse.Namespace) -> int: + """Set the base branch (PR target) for task.""" + repo_root = get_repo_root() + target_dir = resolve_task_dir(args.dir, repo_root) + base_branch = args.base_branch + + if not base_branch: + print(colored("Error: Missing arguments", Colors.RED)) + print("Usage: python task.py set-base-branch <task-dir> <base-branch>") + print("Example: python task.py set-base-branch <dir> develop") + print() + print("This sets the target branch for PR (the branch your feature will merge into).") + return 1 + + task_json = target_dir / FILE_TASK_JSON + if not task_json.is_file(): + print(colored(f"Error: task.json not found at {target_dir}", Colors.RED)) + return 1 + + data = read_json(task_json) + if not data: + return 1 + + data["base_branch"] = base_branch + write_json(task_json, data) + + print(colored(f"✓ Base branch set to: {base_branch}", Colors.GREEN)) + print(f" PR will target: {base_branch}") + return 0 + + +# ============================================================================= +# Command: set-scope +# ============================================================================= + +def cmd_set_scope(args: argparse.Namespace) -> int: + """Set scope for PR title.""" + repo_root = get_repo_root() + target_dir = resolve_task_dir(args.dir, repo_root) + scope = args.scope + + if not scope: + print(colored("Error: Missing arguments", Colors.RED)) + print("Usage: python task.py set-scope <task-dir> <scope>") + return 1 + + task_json = target_dir / FILE_TASK_JSON + if not task_json.is_file(): + print(colored(f"Error: task.json not found at {target_dir}", Colors.RED)) + return 1 + + data = read_json(task_json) + if not data: + return 1 + + data["scope"] = scope + write_json(task_json, data) + + print(colored(f"✓ Scope set to: {scope}", Colors.GREEN)) + return 0 diff --git a/.trellis/scripts/common/task_utils.py b/.trellis/scripts/common/task_utils.py new file mode 100644 index 0000000..62c215e --- /dev/null +++ b/.trellis/scripts/common/task_utils.py @@ -0,0 +1,274 @@ +#!/usr/bin/env python3 +""" +Task utility functions. + +Provides: + is_safe_task_path - Validate task path is safe to operate on + find_task_by_name - Find task directory by name + resolve_task_dir - Resolve task directory from name, relative, or absolute path + archive_task_dir - Archive task to monthly directory + run_task_hooks - Run lifecycle hooks for task events +""" + +from __future__ import annotations + +import shutil +import sys +from datetime import datetime +from pathlib import Path + +from .paths import get_repo_root, get_tasks_dir + + +# ============================================================================= +# Path Safety +# ============================================================================= + +def is_safe_task_path(task_path: str, repo_root: Path | None = None) -> bool: + """Check if a relative task path is safe to operate on. + + Args: + task_path: Task path (relative to repo_root). + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + True if safe, False if dangerous. + """ + if repo_root is None: + repo_root = get_repo_root() + + normalized = task_path.replace("\\", "/") + + # Check empty or null + if not normalized or normalized == "null": + print("Error: empty or null task path", file=sys.stderr) + return False + + # Reject absolute paths + if Path(task_path).is_absolute(): + print(f"Error: absolute path not allowed: {task_path}", file=sys.stderr) + return False + + # Reject ".", "..", paths starting with "./" or "../", or containing ".." + if normalized in (".", "..") or normalized.startswith("./") or normalized.startswith("../") or ".." in normalized: + print(f"Error: path traversal not allowed: {task_path}", file=sys.stderr) + return False + + # Final check: ensure resolved path is not the repo root + abs_path = repo_root / Path(normalized) + if abs_path.exists(): + try: + resolved = abs_path.resolve() + root_resolved = repo_root.resolve() + if resolved == root_resolved: + print(f"Error: path resolves to repo root: {task_path}", file=sys.stderr) + return False + except (OSError, IOError): + pass + + return True + + +# ============================================================================= +# Task Lookup +# ============================================================================= + +def find_task_by_name(task_name: str, tasks_dir: Path) -> Path | None: + """Find task directory by name (exact or suffix match). + + Args: + task_name: Task name to find. + tasks_dir: Tasks directory path. + + Returns: + Absolute path to task directory, or None if not found. + """ + if not task_name or not tasks_dir or not tasks_dir.is_dir(): + return None + + # Try exact match first + exact_match = tasks_dir / task_name + if exact_match.is_dir(): + return exact_match + + # Try suffix match (e.g., "my-task" matches "01-21-my-task") + for d in tasks_dir.iterdir(): + if d.is_dir() and d.name.endswith(f"-{task_name}"): + return d + + return None + + +# ============================================================================= +# Archive Operations +# ============================================================================= + +def archive_task_dir(task_dir_abs: Path, repo_root: Path | None = None) -> Path | None: + """Archive a task directory to archive/{YYYY-MM}/. + + Args: + task_dir_abs: Absolute path to task directory. + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Path to archived directory, or None on error. + """ + if not task_dir_abs.is_dir(): + print(f"Error: task directory not found: {task_dir_abs}", file=sys.stderr) + return None + + # Get tasks directory (parent of the task) + tasks_dir = task_dir_abs.parent + archive_dir = tasks_dir / "archive" + year_month = datetime.now().strftime("%Y-%m") + month_dir = archive_dir / year_month + + # Create archive directory + try: + month_dir.mkdir(parents=True, exist_ok=True) + except (OSError, IOError) as e: + print(f"Error: Failed to create archive directory: {e}", file=sys.stderr) + return None + + # Move task to archive + task_name = task_dir_abs.name + dest = month_dir / task_name + + try: + shutil.move(str(task_dir_abs), str(dest)) + except (OSError, IOError, shutil.Error) as e: + print(f"Error: Failed to move task to archive: {e}", file=sys.stderr) + return None + + return dest + + +def archive_task_complete( + task_dir_abs: Path, + repo_root: Path | None = None +) -> dict[str, str]: + """Complete archive workflow: archive directory. + + Args: + task_dir_abs: Absolute path to task directory. + repo_root: Repository root path. Defaults to auto-detected. + + Returns: + Dict with archive result info. + """ + if not task_dir_abs.is_dir(): + print(f"Error: task directory not found: {task_dir_abs}", file=sys.stderr) + return {} + + archive_dest = archive_task_dir(task_dir_abs, repo_root) + if archive_dest: + return {"archived_to": str(archive_dest)} + + return {} + + +# ============================================================================= +# Task Directory Resolution +# ============================================================================= + +def resolve_task_dir(target_dir: str, repo_root: Path) -> Path: + """Resolve task directory to absolute path. + + Supports: + - Absolute path: /path/to/task + - Relative path: .trellis/tasks/01-31-my-task + - Task name: my-task (uses find_task_by_name for lookup) + + Args: + target_dir: Task directory specification. + repo_root: Repository root path. + + Returns: + Resolved absolute path. + """ + if not target_dir: + return Path() + + normalized = target_dir.replace("\\", "/") + while normalized.startswith("./"): + normalized = normalized[2:] + + # Absolute path + if Path(target_dir).is_absolute(): + return Path(target_dir) + + # Relative path (contains path separator or starts with .trellis) + if "/" in normalized or normalized.startswith(".trellis"): + return repo_root / Path(normalized) + + # Task name - try to find in tasks directory + tasks_dir = get_tasks_dir(repo_root) + found = find_task_by_name(target_dir, tasks_dir) + if found: + return found + + # Fallback to treating as relative path + return repo_root / Path(normalized) + + +# ============================================================================= +# Lifecycle Hooks +# ============================================================================= + +def run_task_hooks(event: str, task_json_path: Path, repo_root: Path) -> None: + """Run lifecycle hooks for a task event. + + Args: + event: Event name (e.g. "after_create"). + task_json_path: Absolute path to the task's task.json. + repo_root: Repository root for cwd and config lookup. + """ + import os + import subprocess + + from .config import get_hooks + from .log import Colors, colored + + commands = get_hooks(event, repo_root) + if not commands: + return + + env = {**os.environ, "TASK_JSON_PATH": str(task_json_path)} + + for cmd in commands: + try: + result = subprocess.run( + cmd, + shell=True, + cwd=repo_root, + env=env, + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + if result.returncode != 0: + print( + colored(f"[WARN] Hook failed ({event}): {cmd}", Colors.YELLOW), + file=sys.stderr, + ) + if result.stderr.strip(): + print(f" {result.stderr.strip()}", file=sys.stderr) + except Exception as e: + print( + colored(f"[WARN] Hook error ({event}): {cmd} — {e}", Colors.YELLOW), + file=sys.stderr, + ) + + +# ============================================================================= +# Main Entry (for testing) +# ============================================================================= + +if __name__ == "__main__": + repo = get_repo_root() + tasks = get_tasks_dir(repo) + + print(f"Tasks dir: {tasks}") + print(f"is_safe_task_path('.trellis/tasks/test'): {is_safe_task_path('.trellis/tasks/test', repo)}") + print(f"is_safe_task_path('../test'): {is_safe_task_path('../test', repo)}") diff --git a/.trellis/scripts/common/tasks.py b/.trellis/scripts/common/tasks.py new file mode 100644 index 0000000..7b44094 --- /dev/null +++ b/.trellis/scripts/common/tasks.py @@ -0,0 +1,112 @@ +""" +Task data access layer. + +Single source of truth for loading and iterating task directories. +Replaces scattered task.json parsing across 9+ files. + +Provides: + load_task — Load a single task by directory path + iter_active_tasks — Iterate all non-archived tasks (sorted) + get_all_statuses — Get {dir_name: status} map for children progress +""" + +from __future__ import annotations + +from collections.abc import Iterator +from pathlib import Path + +from .io import read_json +from .paths import FILE_TASK_JSON +from .types import TaskInfo + + +def load_task(task_dir: Path) -> TaskInfo | None: + """Load task from a directory containing task.json. + + Args: + task_dir: Absolute path to the task directory. + + Returns: + TaskInfo if task.json exists and is valid, None otherwise. + """ + task_json = task_dir / FILE_TASK_JSON + if not task_json.is_file(): + return None + + data = read_json(task_json) + if not data: + return None + + return TaskInfo( + dir_name=task_dir.name, + directory=task_dir, + title=data.get("title") or data.get("name") or "unknown", + status=data.get("status", "unknown"), + assignee=data.get("assignee", ""), + priority=data.get("priority", "P2"), + children=tuple(data.get("children", [])), + parent=data.get("parent"), + package=data.get("package"), + raw=data, + ) + + +def iter_active_tasks(tasks_dir: Path) -> Iterator[TaskInfo]: + """Iterate all active (non-archived) tasks, sorted by directory name. + + Skips the "archive" directory and directories without valid task.json. + + Args: + tasks_dir: Path to the tasks directory. + + Yields: + TaskInfo for each valid task. + """ + if not tasks_dir.is_dir(): + return + + for d in sorted(tasks_dir.iterdir()): + if not d.is_dir() or d.name == "archive": + continue + info = load_task(d) + if info is not None: + yield info + + +def get_all_statuses(tasks_dir: Path) -> dict[str, str]: + """Get a {dir_name: status} mapping for all active tasks. + + Useful for computing children progress without loading full TaskInfo. + + Args: + tasks_dir: Path to the tasks directory. + + Returns: + Dict mapping directory names to status strings. + """ + return {t.dir_name: t.status for t in iter_active_tasks(tasks_dir)} + + +def children_progress( + children: tuple[str, ...] | list[str], + all_statuses: dict[str, str], +) -> str: + """Format children progress string like " [2/3 done]". + + Args: + children: List of child directory names. + all_statuses: Status map from get_all_statuses(). + + Returns: + Formatted string, or "" if no children. + """ + if not children: + return "" + # A child missing from active statuses has been archived (cmd_archive + # sets status=completed before moving the dir). Count it as done so + # parent progress doesn't regress when children are archived. + done = sum( + 1 for c in children + if c not in all_statuses or all_statuses.get(c) in ("completed", "done") + ) + return f" [{done}/{len(children)} done]" diff --git a/.trellis/scripts/common/trellis_config.py b/.trellis/scripts/common/trellis_config.py new file mode 100644 index 0000000..5dbec7a --- /dev/null +++ b/.trellis/scripts/common/trellis_config.py @@ -0,0 +1,131 @@ +#!/usr/bin/env python3 +""" +Standalone reader for .trellis/config.yaml. + +Mirrors a minimal subset of common.config so callers (hooks, workflow_phase) +can read configuration without importing the full task/repo helpers. Returns +an empty dict on missing/malformed files so callers stay simple. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Optional + + +CONFIG_REL_PATH = ".trellis/config.yaml" + + +def _unquote(value: str) -> str: + if len(value) >= 2 and value[0] == value[-1] and value[0] in ('"', "'"): + return value[1:-1] + return value + + +def _strip_inline_comment(value: str) -> str: + """Strip ` # …` inline comments while preserving `#` inside quoted strings. + + YAML treats ` #` (space-hash) as a comment opener; bare `#` inside a token + is part of the value. Quoted strings are immune. + """ + in_quote: str | None = None + for idx, ch in enumerate(value): + if in_quote: + if ch == in_quote: + in_quote = None + continue + if ch in ('"', "'"): + in_quote = ch + continue + if ch == "#" and (idx == 0 or value[idx - 1].isspace()): + return value[:idx] + return value + + +def _next_content_line(lines: list[str], start: int) -> tuple[int, str]: + i = start + while i < len(lines): + stripped = lines[i].strip() + if stripped and not stripped.startswith("#"): + return i, lines[i] + i += 1 + return i, "" + + +def _parse_yaml_block( + lines: list[str], start: int, min_indent: int, target: dict +) -> int: + i = start + current_list: list | None = None + + while i < len(lines): + line = lines[i] + stripped = line.strip() + + if not stripped or stripped.startswith("#"): + i += 1 + continue + + indent = len(line) - len(line.lstrip()) + if indent < min_indent: + break + + if stripped.startswith("- "): + if current_list is not None: + current_list.append(_unquote(stripped[2:].strip())) + i += 1 + elif ":" in stripped: + key, _, value = stripped.partition(":") + key = key.strip() + value = _strip_inline_comment(value).strip() + value = _unquote(value) + current_list = None + + if value: + target[key] = value + i += 1 + else: + next_i, next_line = _next_content_line(lines, i + 1) + if next_i >= len(lines): + target[key] = {} + i = next_i + elif next_line.strip().startswith("- "): + current_list = [] + target[key] = current_list + i += 1 + else: + next_indent = len(next_line) - len(next_line.lstrip()) + if next_indent > indent: + nested: dict = {} + target[key] = nested + i = _parse_yaml_block(lines, i + 1, next_indent, nested) + else: + target[key] = {} + i += 1 + else: + i += 1 + + return i + + +def parse_simple_yaml(content: str) -> dict: + """Parse a small subset of YAML. See common.config for full doc.""" + lines = content.splitlines() + result: dict = {} + _parse_yaml_block(lines, 0, 0, result) + return result + + +def read_trellis_config(repo_root: Optional[Path] = None) -> dict: + """Read .trellis/config.yaml. Returns {} on missing or malformed file.""" + root = repo_root or Path.cwd() + config_file = root / CONFIG_REL_PATH + try: + content = config_file.read_text(encoding="utf-8") + except (FileNotFoundError, OSError): + return {} + try: + parsed = parse_simple_yaml(content) + except Exception: + return {} + return parsed if isinstance(parsed, dict) else {} diff --git a/.trellis/scripts/common/types.py b/.trellis/scripts/common/types.py new file mode 100644 index 0000000..5802e10 --- /dev/null +++ b/.trellis/scripts/common/types.py @@ -0,0 +1,110 @@ +""" +Core type definitions for Trellis task data. + +Provides: + TaskData — TypedDict for task.json shape (read-path type hints only) + TaskInfo — Frozen dataclass for loaded task (the public API type) + AgentRecord — TypedDict for registry.json agent entries +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import TypedDict + + +# ============================================================================= +# task.json shape (TypedDict — used only for read-path type hints) +# ============================================================================= + +class TaskData(TypedDict, total=False): + """Shape of task.json on disk. + + Used only for type annotations when reading task.json. + Writes must use the original dict to avoid losing unknown fields. + """ + + id: str + name: str + title: str + description: str + status: str + dev_type: str + scope: str | None + package: str | None + priority: str + creator: str + assignee: str + createdAt: str + completedAt: str | None + branch: str | None + base_branch: str | None + worktree_path: str | None + commit: str | None + pr_url: str | None + subtasks: list[str] + children: list[str] + parent: str | None + relatedFiles: list[str] + notes: str + meta: dict + + +# ============================================================================= +# Loaded task object (frozen dataclass — the public API type) +# ============================================================================= + +@dataclass(frozen=True) +class TaskInfo: + """Immutable view of a loaded task. + + Created by load_task() / iter_active_tasks(). + Contains the commonly accessed fields; the original dict + is preserved in `raw` for write-back and uncommon field access. + """ + + dir_name: str + directory: Path + title: str + status: str + assignee: str + priority: str + children: tuple[str, ...] + parent: str | None + package: str | None + raw: dict # original dict — use for writes and uncommon fields + + @property + def name(self) -> str: + """Task name (id or name field).""" + return self.raw.get("name") or self.raw.get("id") or self.dir_name + + @property + def description(self) -> str: + return self.raw.get("description", "") + + @property + def branch(self) -> str | None: + return self.raw.get("branch") + + @property + def meta(self) -> dict: + return self.raw.get("meta", {}) + + +# ============================================================================= +# registry.json agent entry +# ============================================================================= + +class AgentRecord(TypedDict, total=False): + """Shape of an agent entry in registry.json.""" + + id: str + pid: int + task_dir: str + worktree_path: str + branch: str + platform: str + started_at: str + status: str diff --git a/.trellis/scripts/common/workflow_phase.py b/.trellis/scripts/common/workflow_phase.py new file mode 100644 index 0000000..2d32931 --- /dev/null +++ b/.trellis/scripts/common/workflow_phase.py @@ -0,0 +1,212 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Workflow Phase Extraction. + +Extracts step-level content from .trellis/workflow.md and optionally filters +platform-specific blocks. + +Platform marker syntax in workflow.md: + + [Claude Code, Cursor, ...] + agent-capable content + [/Claude Code, Cursor, ...] + +Provides: + get_phase_index - Extract the Phase Index section (no --step) + get_step - Extract a single step (#### X.X) section + filter_platform - Strip platform blocks that don't include the given name +""" + +from __future__ import annotations + +import re + +from .paths import DIR_WORKFLOW, get_repo_root + + +def _workflow_md_path(): + return get_repo_root() / DIR_WORKFLOW / "workflow.md" + +# Match a line that *is* a platform marker: "[A, B, C]" or "[/A, B, C]" +_MARKER_RE = re.compile(r"^\[(/?)([A-Za-z][^\[\]]*)\]\s*$") + +# Step heading: "#### 1.0 Title" or "#### 1.0 ..." +_STEP_HEADING_RE = re.compile(r"^####\s+(\d+\.\d+)\b.*$") + +# Phase Index starts here; Phase 1/2/3 step bodies follow; ends at Breadcrumbs. +_PHASE_INDEX_HEADING = "## Phase Index" + + +def _read_workflow() -> str: + path = _workflow_md_path() + if not path.exists(): + raise FileNotFoundError(f"workflow.md not found: {path}") + return path.read_text(encoding="utf-8") + + +def _parse_marker(line: str) -> tuple[bool, list[str]] | None: + """Parse a platform marker line. + + Returns: + (is_closing, [platform_names]) if line is a marker, else None. + """ + m = _MARKER_RE.match(line) + if not m: + return None + is_closing = m.group(1) == "/" + names = [p.strip() for p in m.group(2).split(",") if p.strip()] + return is_closing, names + + +def get_phase_index() -> str: + """Return the compact Phase Index summary from workflow.md. + + SessionStart and no-step phase context use this small summary as their + orientation payload. Detailed Phase 1/2/3 instructions are loaded with + ``get_step`` on demand. ``[workflow-state:STATUS]`` tag blocks are + consumed by the per-turn hook, so they're stripped from this output. + """ + text = _read_workflow() + lines = text.splitlines() + + start: int | None = None + end: int | None = None + for i, line in enumerate(lines): + stripped = line.strip() + if start is None and stripped == _PHASE_INDEX_HEADING: + start = i + continue + if start is not None and stripped == "## Phase 1: Plan": + end = i + break + + if start is None: + return "" + if end is None: + end = len(lines) + + section = "\n".join(lines[start:end]).rstrip() + # Strip [workflow-state:STATUS]...[/workflow-state:STATUS] blocks since + # they're injected separately by inject-workflow-state.py per-turn. + import re as _re + tag_re = _re.compile( + r"\[workflow-state:([A-Za-z0-9_-]+)\]\s*\n.*?\n\s*\[/workflow-state:\1\]\n?", + _re.DOTALL, + ) + return tag_re.sub("", section).rstrip() + "\n" + + +def get_step(step_id: str) -> str: + """Return the `#### X.X` section matching step_id (header + body). + + Body ends at the next `####` or `---` or `##` heading (whichever comes first). + """ + text = _read_workflow() + lines = text.splitlines() + + start: int | None = None + for i, line in enumerate(lines): + m = _STEP_HEADING_RE.match(line) + if m and m.group(1) == step_id: + start = i + break + if start is None: + return "" + + end: int = len(lines) + for j in range(start + 1, len(lines)): + line = lines[j] + if line.startswith("#### "): + end = j + break + if line.startswith("## "): + end = j + break + # Horizontal rule at column 0 + if line.strip() == "---": + end = j + break + + return "\n".join(lines[start:end]).rstrip() + "\n" + + +def _platform_matches(platform: str, block_names: list[str]) -> bool: + """Case-insensitive fuzzy match: accept 'cursor', 'Cursor', 'claude-code', 'Claude Code'.""" + needle = platform.lower().replace("-", "").replace("_", "").replace(" ", "") + for name in block_names: + hay = name.lower().replace("-", "").replace("_", "").replace(" ", "") + if needle == hay: + return True + return False + + +def resolve_effective_platform(platform: str, config: dict) -> str: + """Map ``codex`` to a dispatch-mode-namespaced virtual platform name. + + When ``--platform codex`` is passed, return ``"codex-inline"`` (default) + or ``"codex-sub-agent"`` based on ``.trellis/config.yaml`` ``codex.dispatch_mode``. + ``filter_platform`` then surfaces blocks whose marker lists include the + namespaced name (e.g. ``[codex-sub-agent, ...]`` or ``[codex-inline, Kilo, + Antigravity, Windsurf]``). + + Default is ``inline`` because Codex sub-agents run with ``fork_turns="none"`` + isolation and can't inherit the parent session's task context — inline + keeps the main agent in charge so context isn't lost. Invalid / missing + values also fall back to inline. + + Other platforms are returned unchanged. + """ + if platform == "codex": + mode = "inline" + codex_cfg = config.get("codex") if isinstance(config, dict) else None + if isinstance(codex_cfg, dict): + cfg_mode = codex_cfg.get("dispatch_mode") + if cfg_mode in ("inline", "sub-agent"): + mode = cfg_mode + return f"codex-{mode}" + return platform + + +def filter_platform(content: str, platform: str) -> str: + """Keep lines outside any `[...]` block + lines inside blocks that include platform. + + Marker lines themselves are dropped from the output. + """ + lines = content.splitlines() + out: list[str] = [] + + in_block = False + keep_block = False + + for line in lines: + marker = _parse_marker(line) + if marker is not None: + is_closing, names = marker + if not is_closing: + in_block = True + keep_block = _platform_matches(platform, names) + else: + in_block = False + keep_block = False + continue # drop the marker line itself + + if in_block: + if keep_block: + out.append(line) + continue + out.append(line) + + # Collapse runs of 3+ blank lines that may arise from dropped markers + collapsed: list[str] = [] + blank_run = 0 + for line in out: + if line.strip() == "": + blank_run += 1 + if blank_run <= 2: + collapsed.append(line) + else: + blank_run = 0 + collapsed.append(line) + + return "\n".join(collapsed).rstrip() + "\n" diff --git a/.trellis/scripts/get_context.py b/.trellis/scripts/get_context.py new file mode 100644 index 0000000..0bde5bf --- /dev/null +++ b/.trellis/scripts/get_context.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python3 +""" +Get Session Context for AI Agent. + +Usage: + python get_context.py Output context in text format + python get_context.py --json Output context in JSON format +""" + +from __future__ import annotations + +from common.git_context import main + + +if __name__ == "__main__": + main() diff --git a/.trellis/scripts/get_developer.py b/.trellis/scripts/get_developer.py new file mode 100644 index 0000000..f8a89eb --- /dev/null +++ b/.trellis/scripts/get_developer.py @@ -0,0 +1,26 @@ +#!/usr/bin/env python3 +""" +Get current developer name. + +This is a wrapper that uses common/paths.py +""" + +from __future__ import annotations + +import sys + +from common.paths import get_developer + + +def main() -> None: + """CLI entry point.""" + developer = get_developer() + if developer: + print(developer) + else: + print("Developer not initialized", file=sys.stderr) + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/.trellis/scripts/hooks/linear_sync.py b/.trellis/scripts/hooks/linear_sync.py new file mode 100644 index 0000000..1fdce68 --- /dev/null +++ b/.trellis/scripts/hooks/linear_sync.py @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +"""Linear sync hook for Trellis task lifecycle. + +Syncs task events to Linear via the `linearis` CLI. + +Usage (called automatically by task.py hooks): + python .trellis/scripts/hooks/linear_sync.py create + python .trellis/scripts/hooks/linear_sync.py start + python .trellis/scripts/hooks/linear_sync.py archive + +Manual usage: + TASK_JSON_PATH=.trellis/tasks/<name>/task.json python .trellis/scripts/hooks/linear_sync.py sync + +Environment: + TASK_JSON_PATH - Absolute path to task.json (set by task.py) + +Configuration: + .trellis/hooks.local.json - Local config (gitignored), example: + { + "linear": { + "team": "TEAM_KEY", + "project": "Project Name", + "assignees": { + "dev-name": "linear-user-id" + } + } + } +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +from pathlib import Path + +# ─── Configuration ──────────────────────────────────────────────────────────── + +# Trellis priority → Linear priority (1=Urgent, 2=High, 3=Medium, 4=Low) +PRIORITY_MAP = {"P0": 1, "P1": 2, "P2": 3, "P3": 4} + +# Linear status names (must match your team's workflow) +STATUS_IN_PROGRESS = "In Progress" +STATUS_DONE = "Done" + + +def _load_config() -> dict: + """Load local hook config from .trellis/hooks.local.json.""" + task_json_path = os.environ.get("TASK_JSON_PATH", "") + if task_json_path: + # Walk up from task.json to find .trellis/ + trellis_dir = Path(task_json_path).parent.parent.parent + else: + trellis_dir = Path(".trellis") + + config_path = trellis_dir / "hooks.local.json" + try: + with open(config_path, encoding="utf-8") as f: + return json.load(f) + except (OSError, json.JSONDecodeError): + return {} + + +CONFIG = _load_config() +LINEAR_CFG = CONFIG.get("linear", {}) + +TEAM = LINEAR_CFG.get("team", "") +PROJECT = LINEAR_CFG.get("project", "") +ASSIGNEE_MAP = LINEAR_CFG.get("assignees", {}) + +# ─── Helpers ────────────────────────────────────────────────────────────────── + + +def _read_task() -> tuple[dict, str]: + path = os.environ.get("TASK_JSON_PATH", "") + if not path: + print("TASK_JSON_PATH not set", file=sys.stderr) + sys.exit(1) + with open(path, encoding="utf-8") as f: + return json.load(f), path + + +def _write_task(data: dict, path: str) -> None: + with open(path, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2, ensure_ascii=False) + f.write("\n") + + +def _linearis(*args: str) -> dict | None: + result = subprocess.run( + ["linearis", *args], + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + ) + if result.returncode != 0: + print(f"linearis error: {result.stderr.strip()}", file=sys.stderr) + sys.exit(1) + stdout = result.stdout.strip() + if stdout: + return json.loads(stdout) + return None + + +def _get_linear_issue(task: dict) -> str | None: + meta = task.get("meta") + if isinstance(meta, dict): + return meta.get("linear_issue") + return None + + +# ─── Actions ────────────────────────────────────────────────────────────────── + + +def cmd_create() -> None: + if not TEAM: + print("No linear.team configured in hooks.local.json", file=sys.stderr) + sys.exit(1) + + task, path = _read_task() + + # Skip if already linked + if _get_linear_issue(task): + print(f"Already linked: {_get_linear_issue(task)}") + return + + title = task.get("title") or task.get("name") or "Untitled" + args = ["issues", "create", title, "--team", TEAM] + + # Map priority + priority = PRIORITY_MAP.get(task.get("priority", ""), 0) + if priority: + args.extend(["-p", str(priority)]) + + # Set project + if PROJECT: + args.extend(["--project", PROJECT]) + + # Assign to Linear user + assignee = task.get("assignee", "") + linear_user_id = ASSIGNEE_MAP.get(assignee) + if linear_user_id: + args.extend(["--assignee", linear_user_id]) + + # Link to parent's Linear issue if available + parent_issue = _resolve_parent_linear_issue(task) + if parent_issue: + args.extend(["--parent-ticket", parent_issue]) + + result = _linearis(*args) + if result and "identifier" in result: + if not isinstance(task.get("meta"), dict): + task["meta"] = {} + task["meta"]["linear_issue"] = result["identifier"] + _write_task(task, path) + print(f"Created Linear issue: {result['identifier']}") + + +def cmd_start() -> None: + task, _ = _read_task() + issue = _get_linear_issue(task) + if not issue: + return + _linearis("issues", "update", issue, "-s", STATUS_IN_PROGRESS) + print(f"Updated {issue} -> {STATUS_IN_PROGRESS}") + cmd_sync() + + +def cmd_archive() -> None: + task, _ = _read_task() + issue = _get_linear_issue(task) + if not issue: + return + _linearis("issues", "update", issue, "-s", STATUS_DONE) + print(f"Updated {issue} -> {STATUS_DONE}") + + +def cmd_sync() -> None: + """Sync prd.md content to Linear issue description.""" + task, _ = _read_task() + issue = _get_linear_issue(task) + if not issue: + print("No linear_issue in meta, run create first", file=sys.stderr) + sys.exit(1) + + # Find prd.md next to task.json + task_json_path = os.environ.get("TASK_JSON_PATH", "") + prd_path = Path(task_json_path).parent / "prd.md" + if not prd_path.is_file(): + print(f"No prd.md found at {prd_path}", file=sys.stderr) + sys.exit(1) + + description = prd_path.read_text(encoding="utf-8").strip() + _linearis("issues", "update", issue, "-d", description) + print(f"Synced prd.md to {issue} description") + + +# ─── Parent Issue Resolution ───────────────────────────────────────────────── + + +def _resolve_parent_linear_issue(task: dict) -> str | None: + """Find parent task's Linear issue identifier.""" + parent_name = task.get("parent") + if not parent_name: + return None + + task_json_path = os.environ.get("TASK_JSON_PATH", "") + if not task_json_path: + return None + + current_task_dir = Path(task_json_path).parent + tasks_dir = current_task_dir.parent + parent_json = tasks_dir / parent_name / "task.json" + + if parent_json.exists(): + try: + with open(parent_json, encoding="utf-8") as f: + parent_task = json.load(f) + return _get_linear_issue(parent_task) + except (json.JSONDecodeError, OSError): + pass + return None + + +# ─── Main ───────────────────────────────────────────────────────────────────── + +if __name__ == "__main__": + action = sys.argv[1] if len(sys.argv) > 1 else "" + actions = { + "create": cmd_create, + "start": cmd_start, + "archive": cmd_archive, + "sync": cmd_sync, + } + fn = actions.get(action) + if fn: + fn() + else: + print(f"Unknown action: {action}", file=sys.stderr) + print(f"Valid actions: {', '.join(actions)}", file=sys.stderr) + sys.exit(1) diff --git a/.trellis/scripts/init_developer.py b/.trellis/scripts/init_developer.py new file mode 100644 index 0000000..557b289 --- /dev/null +++ b/.trellis/scripts/init_developer.py @@ -0,0 +1,51 @@ +#!/usr/bin/env python3 +""" +Initialize developer for workflow. + +Usage: + python init_developer.py <developer-name> + +This creates: + - .trellis/.developer file with developer info + - .trellis/workspace/<name>/ directory structure +""" + +from __future__ import annotations + +import sys + +from common.paths import ( + DIR_WORKFLOW, + FILE_DEVELOPER, + get_developer, +) +from common.developer import init_developer + + +def main() -> None: + """CLI entry point.""" + if len(sys.argv) < 2: + print(f"Usage: {sys.argv[0]} <developer-name>") + print() + print("Example:") + print(f" {sys.argv[0]} john") + sys.exit(1) + + name = sys.argv[1] + + # Check if already initialized + existing = get_developer() + if existing: + print(f"Developer already initialized: {existing}") + print() + print(f"To reinitialize, remove {DIR_WORKFLOW}/{FILE_DEVELOPER} first") + sys.exit(0) + + if init_developer(name): + sys.exit(0) + else: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/.trellis/scripts/task.py b/.trellis/scripts/task.py new file mode 100644 index 0000000..92ba674 --- /dev/null +++ b/.trellis/scripts/task.py @@ -0,0 +1,500 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Task Management Script. + +Usage: + python task.py create "<title>" [--slug <name>] [--assignee <dev>] [--priority P0|P1|P2|P3] [--parent <dir>] [--package <pkg>] + python task.py add-context <dir> <file> <path> [reason] # Add jsonl entry + python task.py validate <dir> # Validate jsonl files + python task.py list-context <dir> # List jsonl entries + python task.py start <dir> # Set active task + python task.py current [--source] # Show active task + python task.py finish # Clear active task + python task.py set-branch <dir> <branch> # Set git branch + python task.py set-base-branch <dir> <branch> # Set PR target branch + python task.py set-scope <dir> <scope> # Set scope for PR title + python task.py archive <task-dir> # Archive completed task + python task.py list # List active tasks + python task.py list-archive [month] # List archived tasks + python task.py add-subtask <parent-dir> <child-dir> # Link child to parent + python task.py remove-subtask <parent-dir> <child-dir> # Unlink child from parent +""" + +from __future__ import annotations + +import argparse +import sys + +from common.log import Colors, colored +from common.paths import ( + DIR_WORKFLOW, + DIR_TASKS, + FILE_TASK_JSON, + get_repo_root, + get_developer, + get_tasks_dir, + get_current_task, +) +from common.active_task import ( + clear_active_task, + resolve_active_task, + resolve_context_key, + set_active_task, +) +from common.io import read_json, write_json +from common.task_utils import resolve_task_dir, run_task_hooks +from common.tasks import iter_active_tasks, children_progress + +# Import command handlers from split modules (also re-exports for plan.py compatibility) +from common.task_store import ( + cmd_create, + cmd_archive, + cmd_set_branch, + cmd_set_base_branch, + cmd_set_scope, + cmd_add_subtask, + cmd_remove_subtask, +) +from common.task_context import ( + cmd_add_context, + cmd_validate, + cmd_list_context, +) + + +# ============================================================================= +# Command: start / finish +# ============================================================================= + +def cmd_start(args: argparse.Namespace) -> int: + """Set active task.""" + repo_root = get_repo_root() + task_input = args.dir + + if not task_input: + print(colored("Error: task directory or name required", Colors.RED)) + return 1 + + # Resolve task directory (supports task name, relative path, or absolute path) + full_path = resolve_task_dir(task_input, repo_root) + + if not full_path.is_dir(): + print(colored(f"Error: Task not found: {task_input}", Colors.RED)) + print("Hint: Use task name (e.g., 'my-task') or full path (e.g., '.trellis/tasks/01-31-my-task')") + return 1 + + # Convert to relative path for storage + try: + task_dir = full_path.relative_to(repo_root).as_posix() + except ValueError: + task_dir = str(full_path) + + task_json_path = full_path / FILE_TASK_JSON + + if not resolve_context_key(): + # Degraded mode: no session identity available. + # Hook didn't inject TRELLIS_CONTEXT_ID (common on Windows + Claude Code, + # --continue resume path, fork distribution, hooks disabled, etc.). Skip + # per-session pointer write; AI continues based on conversation context. + print(colored( + "ℹ Session identity not available; active-task pointer not persisted " + "this session (degraded mode). AI continues based on conversation context.", + Colors.YELLOW, + )) + print(colored( + "Hint: run inside an AI IDE/session that exposes session identity, " + "or set TRELLIS_CONTEXT_ID before running task.py start.", + Colors.YELLOW, + )) + + # Still flip task.json status: planning → in_progress so downstream phases proceed. + if task_json_path.is_file(): + data = read_json(task_json_path) + if data and data.get("status") == "planning": + data["status"] = "in_progress" + if write_json(task_json_path, data): + print(colored("✓ Status: planning → in_progress (degraded)", Colors.GREEN)) + run_task_hooks("after_start", task_json_path, repo_root) + return 0 + + active = set_active_task(task_dir, repo_root) + if active: + print(colored(f"✓ Current task set to: {task_dir}", Colors.GREEN)) + print(f"Source: {active.source}") + + if task_json_path.is_file(): + data = read_json(task_json_path) + if data and data.get("status") == "planning": + data["status"] = "in_progress" + if write_json(task_json_path, data): + print(colored("✓ Status: planning → in_progress", Colors.GREEN)) + + print() + print(colored("The hook will now inject context from this task's jsonl files.", Colors.BLUE)) + + run_task_hooks("after_start", task_json_path, repo_root) + return 0 + else: + print(colored("Error: Failed to set current task", Colors.RED)) + return 1 + + +def cmd_finish(args: argparse.Namespace) -> int: + """Clear active task.""" + repo_root = get_repo_root() + active = clear_active_task(repo_root) + current = active.task_path + + if not current: + print(colored("No current task set", Colors.YELLOW)) + return 0 + + # Resolve task.json path before clearing + task_json_path = repo_root / current / FILE_TASK_JSON + + print(colored(f"✓ Cleared current task (was: {current})", Colors.GREEN)) + print(f"Source: {active.source}") + + if task_json_path.is_file(): + run_task_hooks("after_finish", task_json_path, repo_root) + return 0 + + +def cmd_current(args: argparse.Namespace) -> int: + """Show active task.""" + repo_root = get_repo_root() + active = resolve_active_task(repo_root) + + if args.source: + print(f"Current task: {active.task_path or '(none)'}") + print(f"Source: {active.source}") + if active.stale: + print("State: stale") + return 0 if active.task_path else 1 + + if active.task_path: + print(active.task_path) + return 0 + + return 1 + + +# ============================================================================= +# Command: list +# ============================================================================= + +def cmd_list(args: argparse.Namespace) -> int: + """List active tasks.""" + repo_root = get_repo_root() + tasks_dir = get_tasks_dir(repo_root) + current_task = get_current_task(repo_root) + developer = get_developer(repo_root) + filter_mine = args.mine + filter_status = args.status + + if filter_mine: + if not developer: + print(colored("Error: No developer set. Run init_developer.py first", Colors.RED), file=sys.stderr) + return 1 + print(colored(f"My tasks (assignee: {developer}):", Colors.BLUE)) + else: + print(colored("All active tasks:", Colors.BLUE)) + print() + + # Single pass: collect all tasks via shared iterator + all_tasks = {t.dir_name: t for t in iter_active_tasks(tasks_dir)} + all_statuses = {name: t.status for name, t in all_tasks.items()} + + # Display tasks hierarchically + count = 0 + + def _print_task(dir_name: str, indent: int = 0) -> None: + nonlocal count + t = all_tasks[dir_name] + + # Apply --mine filter + if filter_mine and (t.assignee or "-") != developer: + return + + # Apply --status filter + if filter_status and t.status != filter_status: + return + + relative_path = f"{DIR_WORKFLOW}/{DIR_TASKS}/{dir_name}" + marker = "" + if relative_path == current_task: + marker = f" {colored('<- current', Colors.GREEN)}" + + # Children progress + progress = children_progress(t.children, all_statuses) + + # Package tag + pkg_tag = f" @{t.package}" if t.package else "" + + prefix = " " * indent + " - " + + if filter_mine: + print(f"{prefix}{dir_name}/ ({t.status}){pkg_tag}{progress}{marker}") + else: + print(f"{prefix}{dir_name}/ ({t.status}){pkg_tag}{progress} [{colored(t.assignee or '-', Colors.CYAN)}]{marker}") + count += 1 + + # Print children indented + for child_name in t.children: + if child_name in all_tasks: + _print_task(child_name, indent + 1) + + # Display only top-level tasks (those without a parent) + for dir_name in sorted(all_tasks.keys()): + if not all_tasks[dir_name].parent: + _print_task(dir_name) + + if count == 0: + if filter_mine: + print(" (no tasks assigned to you)") + else: + print(" (no active tasks)") + + print() + print(f"Total: {count} task(s)") + return 0 + + +# ============================================================================= +# Command: list-archive +# ============================================================================= + +def cmd_list_archive(args: argparse.Namespace) -> int: + """List archived tasks.""" + repo_root = get_repo_root() + tasks_dir = get_tasks_dir(repo_root) + archive_dir = tasks_dir / "archive" + month = args.month + + print(colored("Archived tasks:", Colors.BLUE)) + print() + + if month: + month_dir = archive_dir / month + if month_dir.is_dir(): + print(f"[{month}]") + for d in sorted(month_dir.iterdir()): + if d.is_dir(): + print(f" - {d.name}/") + else: + print(f" No archives for {month}") + else: + if archive_dir.is_dir(): + for month_dir in sorted(archive_dir.iterdir()): + if month_dir.is_dir(): + month_name = month_dir.name + count = sum(1 for d in month_dir.iterdir() if d.is_dir()) + print(f"[{month_name}] - {count} task(s)") + + return 0 + + +# ============================================================================= +# Help +# ============================================================================= + +def show_usage() -> None: + """Show usage help.""" + print("""Task Management Script + +Usage: + python task.py create <title> Create new task directory + python task.py create <title> --package <pkg> Create task for a specific package + python task.py create <title> --parent <dir> Create task as child of parent + python task.py add-context <dir> <jsonl> <path> [reason] Add entry to jsonl + python task.py validate <dir> Validate jsonl files + python task.py list-context <dir> List jsonl entries + python task.py start <dir> Set active task + python task.py current [--source] Show active task + python task.py finish Clear active task + python task.py set-branch <dir> <branch> Set git branch + python task.py set-base-branch <dir> <branch> Set PR target branch + python task.py set-scope <dir> <scope> Set scope for PR title + python task.py archive <task-dir> Archive completed task + python task.py add-subtask <parent> <child> Link child task to parent + python task.py remove-subtask <parent> <child> Unlink child from parent + python task.py list [--mine] [--status <status>] List tasks + python task.py list-archive [YYYY-MM] List archived tasks + +Monorepo options: + --package <pkg> Package name (validated against config.yaml packages) + +List options: + --mine, -m Show only tasks assigned to current developer + --status, -s <s> Filter by status (planning, in_progress, review, completed) + +Examples: + python task.py create "Add login feature" --slug add-login + python task.py create "Add login feature" --slug add-login --package cli + python task.py create "Child task" --slug child --parent .trellis/tasks/01-21-parent + python task.py add-context <dir> implement .trellis/spec/cli/backend/auth.md "Auth guidelines" + python task.py set-branch <dir> task/add-login + python task.py start .trellis/tasks/01-21-add-login + python task.py current --source + python task.py finish + python task.py archive add-login + python task.py add-subtask parent-task child-task # Link existing tasks + python task.py remove-subtask parent-task child-task + python task.py list # List all active tasks + python task.py list --mine # List my tasks only + python task.py list --mine --status in_progress # List my in-progress tasks +""") + + +# ============================================================================= +# Main Entry +# ============================================================================= + +def main() -> int: + """CLI entry point.""" + # Deprecation guard: `init-context` was removed in v0.5.0-beta.12. + # Detect early so argparse doesn't mask the real reason with a generic + # "invalid choice" error. + if len(sys.argv) >= 2 and sys.argv[1] == "init-context": + print( + colored( + "Error: `task.py init-context` was removed in v0.5.0-beta.12.", + Colors.RED, + ), + file=sys.stderr, + ) + print( + "implement.jsonl / check.jsonl are now seeded on `task.py create` for", + file=sys.stderr, + ) + print( + "sub-agent-capable platforms and curated by the AI during planning when needed.", + file=sys.stderr, + ) + print("See .trellis/workflow.md planning artifact guidance or run:", file=sys.stderr) + print( + " python ./.trellis/scripts/get_context.py --mode phase --step 1", + file=sys.stderr, + ) + print( + "Use `task.py add-context <dir> implement|check <path> <reason>` to append entries.", + file=sys.stderr, + ) + return 2 + + parser = argparse.ArgumentParser( + description="Task Management Script", + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + subparsers = parser.add_subparsers(dest="command", help="Commands") + + # create + p_create = subparsers.add_parser("create", help="Create new task") + p_create.add_argument("title", help="Task title") + p_create.add_argument("--slug", "-s", help="Task slug") + p_create.add_argument("--assignee", "-a", help="Assignee developer") + p_create.add_argument("--priority", "-p", default="P2", help="Priority (P0-P3)") + p_create.add_argument("--description", "-d", help="Task description") + p_create.add_argument("--parent", help="Parent task directory (establishes subtask link)") + p_create.add_argument("--package", help="Package name for monorepo projects") + + # add-context + p_add = subparsers.add_parser("add-context", help="Add context entry") + p_add.add_argument("dir", help="Task directory") + p_add.add_argument("file", help="JSONL file (implement|check)") + p_add.add_argument("path", help="File path to add") + p_add.add_argument("reason", nargs="?", help="Reason for adding") + + # validate + p_validate = subparsers.add_parser("validate", help="Validate context files") + p_validate.add_argument("dir", help="Task directory") + + # list-context + p_listctx = subparsers.add_parser("list-context", help="List context entries") + p_listctx.add_argument("dir", help="Task directory") + + # start + p_start = subparsers.add_parser("start", help="Set active task") + p_start.add_argument("dir", help="Task directory") + + # current + p_current = subparsers.add_parser("current", help="Show active task") + p_current.add_argument("--source", action="store_true", + help="Show active task source") + + # finish + subparsers.add_parser("finish", help="Clear active task") + + # set-branch + p_branch = subparsers.add_parser("set-branch", help="Set git branch") + p_branch.add_argument("dir", help="Task directory") + p_branch.add_argument("branch", help="Branch name") + + # set-base-branch + p_base = subparsers.add_parser("set-base-branch", help="Set PR target branch") + p_base.add_argument("dir", help="Task directory") + p_base.add_argument("base_branch", help="Base branch name (PR target)") + + # set-scope + p_scope = subparsers.add_parser("set-scope", help="Set scope") + p_scope.add_argument("dir", help="Task directory") + p_scope.add_argument("scope", help="Scope name") + + # archive + p_archive = subparsers.add_parser("archive", help="Archive task") + p_archive.add_argument("name", help="Task directory or name") + p_archive.add_argument("--no-commit", action="store_true", help="Skip auto git commit after archive") + + # list + p_list = subparsers.add_parser("list", help="List tasks") + p_list.add_argument("--mine", "-m", action="store_true", help="My tasks only") + p_list.add_argument("--status", "-s", help="Filter by status") + + # add-subtask + p_addsub = subparsers.add_parser("add-subtask", help="Link child task to parent") + p_addsub.add_argument("parent_dir", help="Parent task directory") + p_addsub.add_argument("child_dir", help="Child task directory") + + # remove-subtask + p_rmsub = subparsers.add_parser("remove-subtask", help="Unlink child task from parent") + p_rmsub.add_argument("parent_dir", help="Parent task directory") + p_rmsub.add_argument("child_dir", help="Child task directory") + + # list-archive + p_listarch = subparsers.add_parser("list-archive", help="List archived tasks") + p_listarch.add_argument("month", nargs="?", help="Month (YYYY-MM)") + + args = parser.parse_args() + + if not args.command: + show_usage() + return 1 + + commands = { + "create": cmd_create, + "add-context": cmd_add_context, + "validate": cmd_validate, + "list-context": cmd_list_context, + "start": cmd_start, + "current": cmd_current, + "finish": cmd_finish, + "set-branch": cmd_set_branch, + "set-base-branch": cmd_set_base_branch, + "set-scope": cmd_set_scope, + "archive": cmd_archive, + "add-subtask": cmd_add_subtask, + "remove-subtask": cmd_remove_subtask, + "list": cmd_list, + "list-archive": cmd_list_archive, + } + + if args.command in commands: + return commands[args.command](args) + else: + show_usage() + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.trellis/spec/backend/codex-continuation-contracts.md b/.trellis/spec/backend/codex-continuation-contracts.md new file mode 100644 index 0000000..7f10aa7 --- /dev/null +++ b/.trellis/spec/backend/codex-continuation-contracts.md @@ -0,0 +1,1396 @@ +# Codex Continuation and CPA Integration Contracts + +## Scenario: Responses continuation middleware and CPA integration + +### 1. Scope / Trigger +- Trigger this spec whenever work touches `/v1/responses` request handling, Codex encrypted reasoning replay, 516-style truncation detection, CPA integration, or proxy/egress deployment for Codex traffic. +- This is cross-layer work: request decoding, upstream SSE control flow, auth routing, Docker/Caddy networking, and downstream response compatibility all affect correctness. +- The SJC migration proved that replacing sub2api with CPA removes the old non-passthrough production path, but does not by itself solve the 516 continuation problem. + +### 2. Signatures +- HTTP endpoint: `POST /v1/responses` +- Request headers: + - `Content-Encoding: zstd` is accepted by the middleware and decoded before JSON parsing. + - `Content-Encoding` must not be forwarded after decode because the forwarded body is plain JSON. + - `Responses-API-Base` may override the upstream Responses base according to config mode. +- Continuation detector: + - `reasoning_tokens == 518 * n - 2` + - examples: `516`, `1034`, `1552`, `2070`, `2588` +- Production CPA route: + - Public base URL: `https://cpa.konbakuyomu.us/` + - CPA local bind: `127.0.0.1:8317` + - Required production Codex egress: `socks5://172.19.0.1:1082` + +### 3. Contracts +- A continuation round is allowed only when the terminal upstream response has the truncation-token fingerprint and the completed output includes replayable encrypted reasoning. +- Hidden continuation rounds must preserve the same selected OAuth account and proxy route. Do not rotate to another account inside one folded response; encrypted reasoning may be account-bound. +- Downstream clients must see one logical Responses stream, even if the middleware or CPA opens multiple upstream rounds. +- Reasoning items may be forwarded as reasoning, but tentative message/function-call output from a truncated round must stay buffered and must be discarded if a continuation round is opened. +- Final usage must distinguish agent-facing logical usage from billed upstream usage. Keep per-round metadata redacted; never log OAuth tokens, API keys, or encrypted reasoning payloads. +- CPA's ordinary stream chunk plugin surface is not enough for this feature because it can mutate/drop chunks after executor output, but cannot own upstream retry/continuation with the same auth context. A durable CPA-native implementation belongs in the Codex executor or in a new executor-level supervisor API. + +### 4. Validation & Error Matrix +- Unsupported request `Content-Encoding` -> return `400` with an explicit decode error. +- Invalid zstd body -> return `400`; log content type, encoding, and byte length, but not body content. +- Truncation fingerprint without encrypted reasoning -> do not continue; flush the natural terminal response. +- Upstream EOF before terminal event -> emit an incomplete terminal response and do not leak buffered tentative text/tool calls. +- Continuation cap reached -> stop and report the cap reason in metadata. +- CPA egress route not reachable from the CPA container -> fail deployment validation; attach CPA to the required Docker network rather than falling back to direct egress. + +### 5. Good/Base/Bad Cases +- Good: Round 1 ends with `reasoning_tokens=516`, has encrypted reasoning, and tentative text. The middleware replays reasoning plus a hidden marker into Round 2, discards Round 1 text, and downstream receives only the folded final answer. +- Base: A normal upstream response has no truncation fingerprint. The middleware acts as a transparent stream proxy. +- Bad: A downstream plugin sees `response.completed` and tries to start another upstream request after chunks have already been emitted. This leaks partial output and cannot guarantee same-auth replay. + +### 6. Tests Required +- Unit: detector accepts `518 * n - 2` values and rejects adjacent values. +- Unit: zstd request bodies decode before JSON parsing, and `Content-Encoding` is dropped from forwarded headers. +- Stream fixture: truncated round followed by clean round produces one created event, one terminal event, monotonic sequence numbers, two reasoning items, and only the final clean answer. +- Stream fixture: truncated function call is discarded; clean function call is flushed. +- Stream fixture: EOF without terminal event emits `response.incomplete` and does not leak buffered text. +- Integration: CPA production smoke must verify `/healthz`, authenticated `/v1/models`, authenticated `/v1/responses`, and egress evidence through `172.19.0.1:1082`. + +### 7. Wrong vs Correct + +#### Wrong +```text +client -> CPA ordinary StreamChunkInterceptor plugin -> open ad hoc second request +``` + +This is too late in the pipeline: the plugin only sees downstream-bound chunks and cannot safely reuse the executor-selected Codex auth/proxy context. + +#### Correct +```text +client -> continuation supervisor -> Codex executor same-auth upstream rounds -> folded downstream stream +``` + +The continuation owner must sit at executor level, where it can inspect raw upstream SSE, preserve auth/proxy identity, replay encrypted reasoning, and reconstruct the final logical response. + +## Scenario: CodexCont admin diagnostics dashboard and request summaries + +### 1. Scope / Trigger +- Trigger this spec whenever work touches CodexCont `/admin/*` routes, in-process diagnostics, SSE admin streams, dashboard UI, or production deployment of the CodexCont sidecar behind CPA. +- This is a cross-layer contract: `/v1/responses` lifecycle events feed `Diagnostics`, `Diagnostics` projects request summaries, Starlette exposes JSON/SSE admin APIs, and the static dashboard renders beginner-facing protection status. +- The admin dashboard is observability for the 516 continuation mitigation. It must never become a second control plane that mutates CPA, OAuth accounts, proxy routes, or request payloads. + +### 2. Signatures +- Admin routes: + - `GET /admin/healthz` -> service health and uptime. + - `GET /admin/status` -> counters, active request metadata, upstream health, and safe config summary. + - `GET /admin/requests?limit=N` -> recent request-level protection summaries. + - `GET /admin/logs?limit=N` -> recent redacted diagnostic log events. + - `GET /admin/logs/stream` -> SSE stream with `event: ready`, `event: request`, and `event: log`. + - `GET /admin/` -> static dashboard HTML. +- Request summary projection fields: + - `request_id`, `model`, `path`, `started_at`, `updated_at`, `ended_at`, `duration_ms` + - `status`, `protection`, `folded`, `passthrough`, `passthrough_reason` + - `rounds[]`, `latest_round`, `latest_reasoning_tokens` + - `first_truncation_round`, `first_truncation_reasoning_tokens`, `first_truncation_n`, `first_truncation_decision`, `continuation_count` + - `truncation_match`, `final_status`, `stopped_reason`, `failure_reason`, `failure_detail` + - optional safe `key_identity`: `known`, `name`, `id`, `preview`, `source`, + `enabled` +- Protection values: + - `protected_clean`, `auto_continued`, `risk_uncontinued`, `passthrough`, `failed`, `incomplete`, `processing` + +### 3. Contracts +- `Diagnostics` owns the request-summary projection. The frontend may format labels, but it must not re-derive protection status from raw log event names or ad hoc field parsing. +- Admin data is memory-only. Do not add persistent log files, databases, Redis, or CPA Manager dependencies for dashboard v1/v2 behavior. +- Request summaries and logs must not include request bodies, Authorization headers, API keys, OAuth tokens, encrypted reasoning content, or internal implementation-only fields such as `_started_perf`. +- If CodexCont is configured with a Key Policy state path, request summaries + may include safe key identity. The resolver must hash a bearer credential + only long enough to match Key Policy state, then discard the raw key and + expose only safe name/preview/source fields. +- `event: log` behavior is backward-compatible with the original dashboard stream. Adding request updates must use a separate `event: request` SSE event. +- The beginner-facing dashboard must distinguish "entered CodexCont protection and no continuation was needed" from "516/518n-2 was detected and a hidden continuation round was opened". +- When a continued request ends with a clean final round, `latest_reasoning_tokens` may be below 516. The dashboard must label it as latest-round reasoning and separately display the first 516/518n-2 trigger round from the request summary. +- The dashboard frontend must treat the SSE connection as recoverable browser state, not as a durable data source. Manual refresh and foreground resume (`visibilitychange`, `pageshow`, or stale `focus`) must re-fetch the JSON snapshots with no-store/cache-bust semantics and force-create a new `EventSource`. Late responses from older fetches must not overwrite newer snapshots. +- The refresh action must provide visible busy/completion feedback, and the realtime connection chip must animate in all states (`connected`, `connecting/reconnecting`, and `disconnected`) so an operator can see that the page is alive after returning from an idle tab. +- The dashboard's top-right refresh button light must be driven by the same + realtime state as the connection chip. During a refresh it may temporarily + show `syncing`, but after the label falls back to "refresh" the light must + keep the live state class (`live-ok`, `live-warn`, or `live-bad`) instead of + returning to a neutral grey dot. +- Admin snapshot refresh must keep a short minimum visible `syncing` duration + on manual/foreground refresh, just like the user page. Fast local admin + snapshot responses must not collapse the operator feedback into a single + imperceptible frame. +- When an SSE request update is still `processing`, the dashboard should + immediately refresh status counters and follow up with short delayed + `/admin/requests` snapshot reloads. Do not rely only on the next long polling + interval to clear processing rows. +- Production admin access must remain behind `cpa-admin.konbakuyomu.us` plus Cloudflare Access. Public `cpa.konbakuyomu.us` must not expose `/admin/*`, `/codexcont/*`, `/management.html`, or CPA management APIs. +- SJC is a small-disk host. Deployment must prefer uploading changed files plus single-service rebuild/restart; do not use Docker prune or broad filesystem cleanup as part of dashboard rollout. + +### 4. Validation & Error Matrix +- Invalid `limit` query on `/admin/requests` or `/admin/logs` -> fall back to safe defaults. +- Request summary retention exceeds configured cap -> discard oldest non-active summaries first. +- Active request summary is returned -> internal monotonic timer fields must be stripped before JSON/SSE output. +- `GET /admin/logs/stream?once=1` -> emits `ready`, recent `request` events, then recent `log` events, then ends. +- Upstream CPA health probe fails -> dashboard reports upstream unhealthy but admin routes still return safely. +- Browser tab is idle/backgrounded and returns later -> dashboard reconnects SSE and reloads snapshots without requiring a full page reload. +- Manual refresh is clicked while a previous fetch is slow -> the latest refresh wins; older fetch results are ignored instead of overwriting the visible table. +- Request row appears as `processing` -> short follow-up reloads update it to a + terminal state without requiring a full page refresh. +- Public API host exposes any admin path -> deployment validation fails; fix Caddy/admin proxy routing before accepting rollout. +- SJC free space is tight before rebuild -> verify `df -h /` and avoid pulls/prune; if rebuild needs new image layers and space is insufficient, pause rather than cleaning broad data. + +### 5. Good/Base/Bad Cases +- Good: A real Codex request appears in `/admin/requests` with `protection=protected_clean` or `auto_continued`, and no raw reasoning content is present. +- Base: An invalid JSON request returns `400` from `/v1/responses` and appears as `protection=failed`, `failure_reason=invalid_json_body`. +- Bad: The dashboard scans log strings like `round_decision` in JavaScript and guesses whether the request was protected. This duplicates backend contract logic and will drift. +- Bad: A production rollout fixes the page but exposes `/admin/requests` on `https://cpa.konbakuyomu.us/`. This leaks operational metadata and violates the public/admin boundary. + +### 6. Tests Required +- Unit: request summary projection covers `protected_clean`, `auto_continued`, `risk_uncontinued`, `passthrough`, `failed`, and retention behavior. +- Unit: redaction preserves numeric counters such as `reasoning_tokens` and `total_tokens`, while redacting bearer/API/OAuth/encrypted-content fields. +- Route smoke: `/admin/requests` returns summaries and `/admin/logs/stream?once=1` includes both `event: request` and `event: log`. +- Frontend smoke: desktop and mobile dashboard render without horizontal overflow, and simulated protection states are visibly distinct. +- Frontend smoke: dashboard HTML keeps the manual-refresh reconnect path, foreground-resume handler, and visible refresh/realtime animation hooks. +- Frontend smoke: dashboard HTML keeps processing-request follow-up reloads. +- Production smoke: `cpa-admin.konbakuyomu.us/codexcont/` reaches the dashboard through Cloudflare Access, while public `cpa.konbakuyomu.us/admin/*` and `/codexcont/*` return `404`. + +### 7. Wrong vs Correct + +#### Wrong +```text +raw log event -> frontend string matching -> protection label +``` + +This spreads the event contract into JavaScript and makes the beginner-facing status depend on incidental log wording. + +#### Correct +```text +/v1/responses lifecycle -> Diagnostics request summary -> /admin/requests + event: request -> dashboard label +``` + +`Diagnostics` is the single projection owner. The UI renders the explicit `protection` value and keeps raw logs as an advanced troubleshooting view only. + +## Scenario: CPA Key Policy, CPAMP, and user usage portal + +### 1. Scope / Trigger +- Trigger this spec whenever work touches `cpa_usage_portal/`, CPAMP monitoring + queries, CPA Key Policy state parsing, per-key quota display, user usage + events, or production routes for `cpa-usage.konbakuyomu.us`. +- This is a cross-layer contract: Key Policy state authenticates a raw + `cpa_...` key, CPA records the plugin principal into usage events, CPAMP + hashes that principal, the portal filters CPAMP analytics, and the frontend + renders only safe per-user summaries. + +### 2. Signatures +- Portal routes: + - `GET /healthz` + - `POST /api/session` + - `DELETE /api/session` + - `GET /api/me` + - `GET /api/usage?range=5h|24h|7d|month` + - `GET /api/events?range=5h|24h|7d|month&limit=N&before=...` + - `GET /api/events/stream` + - `GET /admin/` + - `GET /admin/api/keys` + - `PUT /admin/api/keys/limits` + - `PUT /admin/api/keys/{id}/limits` + - `POST /admin/api/keys/{id}/reset` + - `GET /admin/api/events?key_id=all|...&range=5h|24h|7d|month` +- Production user route: `https://cpa-usage.konbakuyomu.us/` +- Production local quota admin route: + `https://cpa-admin.konbakuyomu.us/usage-admin/` +- Production admin route for CPAMP: `https://cpa-admin.konbakuyomu.us/` +- Key Policy state path on SJC: + `/opt/codex-stacks/cpa/plugin-state/cpa-key-policy-state.json` +- Containers that read Key Policy state must use a mounted in-container path + such as `/data/plugin-state/cpa-key-policy-state.json`; host paths are not + valid from inside CodexCont or the usage portal unless explicitly mounted. + +### 3. Contracts +- CPA stays on the official image. Do not fork CPA to implement per-key usage + views. +- CPAMP stays on the official `seakee/cpa-manager-plus:latest` image. Do not + patch CPAMP source for user self-service behavior; add sidecars or routes + around it instead. +- CPA Key Policy stays as the official release plugin binary mounted into CPA. + Updating Key Policy should mean replacing the plugin binary and preserving + `plugin-state`, not editing CPA source. +- Keep CPA, CPAMP, CodexCont, and `cpa-usage-portal` as separate containers / + stacks on the shared `cpa_net`. Do not bundle them into one image because + independent updates are part of the maintenance contract. +- The user portal may mutate only its own local SQLite metadata: 5H/month + limits, reset watermarks, and audit entries. It may read Key Policy state and + query CPAMP monitoring, but it must not mutate CPA, OAuth accounts, proxy + routing, CPAMP source events, or Key Policy records. +- Ordinary users should receive Key Policy `cpa_...` keys. CPA native `sk...` + keys are compatibility/admin escape hatches and should not be treated as + self-service user credentials. +- There is no automatic one-to-one binding between native CPA `sk...` keys and + Key Policy `cpa_...` keys. If a future migration needs such a bridge, design + an explicit mapping layer and prove it cannot bypass Key Policy limits. +- The CPAMP login key and the CPA management key are different secrets. + `cpa-admin.konbakuyomu.us/management.html` currently points to CPAMP, so it + requires the CPAMP admin key. CPA-native management calls use the CPA + management key through the internal admin proxy path. +- Login validation uses the raw user key only once: + `sha256(trimmed_raw_cpa_key)` must match Key Policy `key_hash` + (`sha256:<hex>`). The raw key must not be stored, logged, or returned. +- CPAMP filtering for Key Policy keys must use `sha256(Key Policy id)`, not + `sha256(raw cpa_... key)`. CPA Key Policy authenticates requests with + `Principal = key.ID`, and CPAMP hashes CPA's usage-record principal. +- Keep raw-key hash and CPAMP usage hash as separate concepts in code and + tests. Session cookies may contain safe hash identifiers, but every request + must re-load Key Policy state and validate the raw-key hash still maps to an + enabled record. +- User APIs must never return raw API keys, full raw-key hashes, full CPAMP + usage hashes, OAuth tokens, CPA management keys, CPAMP admin keys, cookies, + Authorization headers, request bodies, response bodies, or encrypted + reasoning content. +- CPAMP `api_key_stats` must be projected before returning it to the browser. + Do not pass CPAMP rows through directly because they may include full + `api_key_hash` values. +- When Plus receives CPA usage records from executor/host-callback paths, it + must resolve the user key by `AuthID` first, then `APIKey`, then `Source`. + Executor records can carry the Key Policy id in `AuthID` while `Source` or + provider fields name an upstream account file. Mapping only by raw API/source + fields drops valid usage from `cpa-usage.konbakuyomu.us`. +- Plus usage projection must keep the user-visible model separate from the + internal upstream model. If CPA usage callbacks report the executor's + internal model instead of a client alias, Plus must project known executor + aliases such as `gpt-5.3-codex-spark -> gpt-5.4` into `model` and + `requested_model`, while preserving the internal value in `actual_model`. +- Key Policy model entries may be structured objects under `models[]`, not only + strings. The portal must parse clean aliases from `alias` / `model` / + `target_model` fields instead of rendering dicts as strings. +- Per-key prices are owned by Key Policy. The portal must parse + `input_price_per_million`, `output_price_per_million`, and + `cache_read_price_per_million` from each model entry, plus legacy + top-level `model_prices` forms for compatibility. +- A model entry whose input, output, cache-read, and cache-creation prices are + all zero is treated as unpriced for safety. Missing prices must not silently + turn into "free" usage unless a future explicit free-model policy is added. +- CPAMP can legitimately return `cost: 0` when its own global price book lacks + custom Codex aliases. For self-service user accounting, `/api/usage` and + `/api/events` should overlay costs using the current key's Key Policy price + book. Do not interpret CPAMP zero cost as "free" when Key Policy prices are + configured. +- CPAMP Management API is the source of truth for usage token projection. Its + public `cached_tokens` field is already the compatibility cached-input bucket + used by the main CPA/CPAMP dashboard: + `max(max(cached_tokens, cache_tokens) - cache_read_tokens - + cache_creation_tokens, 0)`. The portal must treat that field as a real + OpenAI/Codex cache hit even when fine-grained `cache_read_tokens` and + `cache_creation_tokens` are both zero. +- Keep CPAMP-compatible cached input separate from fine-grained cache + read/create fields in user-facing details. For OpenAI/Codex, a large + `cached_tokens` value with `cache_read_tokens = 0` is normal and must not be + displayed as "no cache read". +- `/api/me` must expose safe daily/weekly USD limits and a safe pricing + summary. It must also expose local 5H/month USD limits and reset points when + the portal SQLite has them. The user dashboard must show 5H, daily, weekly, + and monthly limits directly, not only as a selected-range hint. +- `usage-admin` bulk limit saves must validate every submitted key id and + numeric 5H/month value before reporting success. The UI should expose one + global save action for local limits and keep per-key soft reset actions + separate, because reset changes the local watermark rather than the limit + configuration. +- `usage-admin` all-key event mode (`key_id=all`) must merge enabled Key Policy + keys, attach only a safe key summary (`id`, `name`, `preview`, `enabled`), + sort newest first, and cap the merged result by the requested limit. +- The usage portal's selected time range controls both `/api/usage` aggregates + and the visible `/api/events` recent-request table. The page must also render + the active range label, because 24h and 7d can legitimately return identical + numbers when all retained usage happened in the last day. +- Supported portal ranges are `5h`, `24h`, `7d`, and `month`. `month` is the + current Asia/Shanghai calendar month. `5h`, `24h`, and `7d` are rolling + windows. +- A portal soft reset writes a reset watermark and narrows future CPAMP query + windows to `max(base_window_start, reset_at_ms)`. It must not delete CPAMP + rows or rewrite Key Policy's own historical usage display. +- All `/admin/*` portal routes require a proxy-injected admin header from + `cpa-admin.konbakuyomu.us/usage-admin/`. Public `cpa-usage.konbakuyomu.us` + must not be able to call these routes successfully. +- Public `cpa.konbakuyomu.us` must continue to block management, plugin, + admin, CodexCont dashboard, CPAMP, and usage-portal internals. +- The usage portal frontend must not rely on an old `/api/events/stream` + connection after tab idle. Manual refresh and foreground resume must rebuild + the `EventSource`, reload `/api/me`, `/api/usage`, and `/api/events` with + no-store/cache-bust semantics, and keep late fetch responses from replacing + newer data. +- The refresh button and realtime chip must expose visible state changes: + refresh shows busy/completion animation, while the realtime chip pulses for + connected, reconnecting, and disconnected states. +- The visible refresh state must not disappear just because the local or + cached API responds quickly. Keep a short minimum `syncing` state before the + completion confirmation, then use row/card highlights to show what changed + without flashing or blanking the table. +- The refresh button's small status light and the realtime chip must share the + same state transition. After the completion label returns to "refresh", the + light must still pulse as connected/reconnecting/error rather than reverting + to a grey idle light. +- Custom Governor/CodexCont dashboards must derive the visible `活跃` chip from + the current request list's non-stale `processing` rows rather than directly + rendering a backend `active_requests` counter. Backend counters can remain + high after abnormal communication; stale processing rows may stay in history + but must not keep the active chip inflated. +- CPAMP-aligned custom dashboards should use restrained status-dot animation + only. Do not reintroduce page sweep bars, refresh-button sweep lights, + metric-card bump animations, or broad row flash effects as the primary + realtime feedback. +- A realtime usage event should update the visible recent-request table + immediately and then schedule delayed snapshot refreshes, because CPAMP + aggregate views may update slightly after the event row appears. +- SJC is disk-constrained. Portal rollouts should upload changed files and + rebuild only `cpa-usage-portal`; do not use Docker prune or broad deletion. + +### 4. Validation & Error Matrix +- Unknown raw API key -> `401 invalid_api_key`. +- Native CPA `sk...` key submitted to the user portal -> `401 invalid_api_key` + unless it has been explicitly migrated into Key Policy; this is expected. +- Disabled Key Policy record -> `403 api_key_disabled` at login or + `401 key_not_available` for an existing session. +- Missing or invalid session -> `401 not_authenticated`. +- CPA management key submitted to CPAMP UI -> reject as an invalid admin key; + use the CPAMP admin key for CPAMP. +- CPAMP rows whose `api_key_hash` does not match the current key's CPAMP usage + hash -> drop them server-side. +- Browser tab is left idle and reopened -> usage portal reconnects SSE and + reloads own-key usage/events without a full page reload. +- Manual refresh happens during a slow previous refresh -> the latest refresh + owns the visible state; stale responses are ignored. +- Key Policy prices exist but CPAMP returns zero cost -> user portal shows + nonzero estimated cost from Key Policy prices and marks the source as + `key_policy`. +- `GET /api/events?range=24h` -> recent events are fetched from the 24h window; + omitting `range` keeps the compatibility default. +- Key Policy daily/weekly USD limits exist -> `/api/me` and the dashboard show + both values safely. +- Portal local 5H/month limits exist -> `/api/me` and the dashboard show both + values safely. +- `POST /admin/api/keys/{id}/reset` -> updates only portal reset watermarks; + CPAMP original rows remain visible in CPAMP itself. +- `GET /admin/api/keys` without the proxy-injected admin header -> `404`. +- `GET /api/usage` must not include the full raw-key hash or full policy-id + hash anywhere in the JSON response. +- CPA usage record with `AuthID=<key.id>`, `Source=<provider file>`, and + `Alias=<client-visible model>` -> Plus stores a `usage_events` row for + `<key.id>`, keeps the client-visible alias as `Model`/`RequestedModel`, and + keeps the provider/internal model as `ActualModel`. +- CPA usage record with `Model=<executor internal model>` and no usable visible + alias -> Plus applies the known executor usage alias table before pricing or + user-event projection. +- DNS for `cpa-usage.konbakuyomu.us` may be absent while the sidecar and Caddy + route are ready; verify with explicit host resolution before declaring the + route broken. + +### 5. Good/Base/Bad Cases +- Good: A user logs in with a `cpa_...` key; the portal validates + `sha256(raw key)` against Key Policy state, then queries CPAMP with + `sha256(key.id)` and shows only that key's events. +- Good: CPA and CPAMP are updated by pulling their official images while the + custom user portal is rebuilt separately from this repository. +- Base: A Key Policy key has no events yet. The portal still shows safe key + metadata and empty usage tables. +- Bad: The portal filters CPAMP with `sha256(raw key)` and shows zero events + even though CPAMP has usage records for the key id. +- Bad: `/api/usage` returns CPAMP `api_key_stats` unchanged and leaks a full + `api_key_hash` to the browser. +- Bad: Ordinary users log into CPAMP or create native `sk...` keys for + themselves. That expands the admin trust boundary and bypasses the intended + Key Policy user model. + +### 6. Tests Required +- Unit: raw key hash validates login while `record.cpamp_hash` equals + `sha256(policy id)` when `id` is present. +- Unit: CPAMP analytics calls use the policy-id hash, not the raw-key hash. +- Unit: usage/event projections reject another key's hash and do not return + full hash values. +- Unit: structured Key Policy `models[]` entries produce clean model aliases, + safe daily/weekly limits, and per-model prices. +- Unit: `/api/usage` and `/api/events` recompute nonzero costs from Key Policy + prices when CPAMP cost fields are zero. +- Unit: `usage.handle` maps executor/host-callback records by `AuthID` before + falling back to `APIKey` or `Source`, and preserves alias/internal-model + projection in the stored event. +- Unit: `usage.handle` maps known executor internal models to visible aliases + even when CPA fills both `Model` and `Alias` with the internal model. +- Unit: portal local SQLite stores 5H/month limits, applies reset watermarks, + and closes connections cleanly on Windows. +- Unit: `/admin/*` routes reject requests without the proxy-injected admin + header and expose safe quota projections when the header is present. +- Unit: redaction covers Authorization, cookies, API keys, tokens, management + keys, and encrypted reasoning fields while preserving numeric token counters. +- Unit: retention deletes old CPAMP `usage_events` in batches and does not run + `VACUUM`. +- Frontend smoke: user portal HTML keeps the forced stream reconnect, + foreground-resume handler, and refresh/realtime animation hooks. +- Production smoke: allowed Key Policy model succeeds, disallowed model is + rejected, CPAMP records real usage, the portal login succeeds, and + `/api/events` returns only own events. + +### 7. Wrong vs Correct + +#### Wrong +```text +user -> native sk... key -> user portal / CPAMP admin +``` + +Native CPA keys are not the quota-managed user identity in this deployment. + +#### Correct +```text +admin -> Key Policy creates cpa_... key -> user uses cpa_... for Codex and usage portal +``` + +The `cpa_...` key is both the request credential and the self-service usage +credential, while CPAMP remains admin-only. + +## Scenario: CPA Key Policy Plus native-key policy layer + +### 1. Scope / Trigger +- Trigger this spec whenever work touches `cpa_key_policy_plus_plugin/`, CPA + native `api-keys`, CPAMP alias integration, Plus policy/quota decisions, + user usage portal login, or over-limit `/v1/responses` behavior. +- This is cross-layer work: CPA config and CPAMP aliases feed Plus SQLite, + Plus frontend-auth metadata feeds model routing/executor behavior, usage + callbacks feed quota windows, and `cpa-usage.konbakuyomu.us` renders the + user-facing view. + +### 2. Signatures +- CPA config source: top-level `api-keys` in `/CLIProxyAPI/config.yaml`. +- CPAMP alias source: + `/CLIProxyAPI/plugin-state/cpamp-usage.sqlite`, table + `api_key_aliases(api_key_hash, alias, updated_at_ms)`. +- Plus config fields: + `native_keys_config_path`, `cpamp_alias_db_path`, + `codex_summary_db_path`, `codexcont_enabled`, and `codexcont_route`. +- Plus user API resource path on production: + `/v0/resource/plugins/cpa-key-policy-plus/user/api/session`, + `/me`, `/usage?range=24h`, `/events?range=24h&limit=N`, and + `/codexcont?limit=N`. +- Plus user page public host: + `https://cpa-usage.konbakuyomu.us/`. +- Executor denial body: + ```json + { + "error": { + "message": "CPA Key Policy+ 已拦截：<key name> 触发 <window>费用限额，已用 $<used> / 上限 $<limit>。", + "type": "rate_limit_exceeded", + "code": "five_hour_quota_exceeded", + "param": "5h" + } + } + ``` + +### 3. Contracts +- CPA/CPAMP owns native `sk-...` key lifecycle: create, delete, copy, and alias. + Plus is a passive policy layer and must not expose raw-key creation, + deletion, rotation, full-key copy, or alias editing controls. +- Plus stores only safe identity: `sha256:<hex>` hash, safe preview, source + flags, read-only alias/name, and strategy fields. It must not store raw + `sk-...` keys. +- Plus policy IDs for native keys use the native hash-derived + `native_<preview>` form. Alias is display/template metadata and must not be + the ledger primary key. +- New native keys default to enabled by current product decision. If exactly + one removed historical row has the same alias, Plus may inherit policy fields + and enabled state into the new native row, but usage history remains under + the old row. If the new row has no RPM or fee-window limits, the admin UI + must label it as missing/unlimited limits instead of hiding the risk. +- Removed native keys are marked `source_present=false`, disabled, hidden by + default, and retained for historical usage/protection summaries. +- Plus admin key reads must trigger native-key sync before listing, then default + to only current official native rows: + `source == native_cpa`, `source_present == true`, and `hidden == false`. + Legacy Plus rows and removed native rows may remain in SQLite for history or + diagnostics, but must not appear in the ordinary strategy table. The ordinary + admin page must request `/keys` without `include_removed`/`show_removed` and + must not expose a visible removed-row toggle. +- Plus SQLite access must use a shared opener with `busy_timeout` for both the + writable Plus DB and read-only auxiliary DBs. The writable Plus DB must limit + the Go `database/sql` pool to one open connection; CPAMP alias DB, executor + summary DB, and legacy import DB reads must use read-only opens where + possible. When using one writable connection, do not keep `Rows` open while + issuing writes on the same DB; collect IDs, close rows, then update. +- CPAMP alias sync must normalize `api_key_aliases.api_key_hash` whether it is + stored as bare SHA256 hex or with a case-insensitive `sha256:` prefix. +- Plus user login now accepts CPA native `sk-...` keys. Retired Plus + `cpa_...` keys should fail with migrated/retired guidance. +- Plus user APIs and HTML must never return raw keys, full hashes, bearer + headers, cookies, request/response bodies, or encrypted reasoning. +- `codexcont_enabled` and `codexcont_route` remain false for Plus. Plus may + read executor summaries through `codex_summary_db_path`, but the executor + plugin owns streaming continuation. +- Over-limit model calls must return an OpenAI-compatible error body with + Chinese key/window/used/limit details. Under the current official CPA + executor ABI, plugins cannot guarantee the final public HTTP status or + response headers on `/v1/responses`; treat the JSON body as the reliable + client-facing contract unless CPA core is changed. + +### 4. Validation & Error Matrix +- Native key appears in CPA config without a Plus policy -> insert enabled + strategy row and mark missing RPM/fee-window limits in the admin UI. +- Native key appears with one same-alias removed template -> copy policy fields + and enabled state; do not copy ledger usage. +- Native key appears with multiple same-alias removed templates -> insert + disabled row with conflict state for manual review. +- Existing native row that is only the old empty/default-disabled placeholder + -> upgrade to enabled on sync. A manually disabled row with actual strategy + settings must stay disabled. +- Native key disappears from CPA config -> set `source_present=false`, + `enabled=false`, `hidden=true`; do not delete history. +- Official key alias changes or deletions in CPAMP -> refreshing the Plus admin + key API updates the displayed alias and default key count without requiring a + CPA restart. +- Missing policy, removed source, disabled key, disallowed model, RPM limit, or + 5H/24H/7D/month fee limit -> structured policy denial with safe key name and + stable error code. +- Fee quota uses post-accounting blocking: when current window usage is already + `>= limit`, the next request is denied. Do not pre-charge or predict the + current request cost. +- A `$0.00` limit is explicit and must deny immediately; `nil` means unlimited. +- Public `cpa.konbakuyomu.us` exposes plugin/admin/resource paths -> deployment + is not accepted. +- Expecting true HTTP 429 from Plus executor without modifying CPA core -> + invalid assumption; production acceptance should check the error JSON body. + +### 5. Good/Base/Bad Cases +- Good: CPAMP has aliases `QQ的官key`, `kuma的官key`, and `阿伟的官key`; Plus + syncs the corresponding native rows as enabled policy records and + `cpa-usage.konbakuyomu.us` logs in with a native `sk-...` key. +- Good: CPAMP shows four official native keys; the Plus admin default key API + returns exactly those four current native rows with matching aliases, while + old `legacy_plus` / `cpa_...` rows remain hidden from the ordinary table. +- Good: Temporarily setting a key's 5H limit to `$0.00` makes the next + `/v1/responses` return an OpenAI-compatible error body naming the key and + `5小时费用限额`, then restoring the previous limit re-enables normal calls. +- Base: A newly created CPA native key has no alias/history; Plus shows it + enabled and clearly warns that RPM or fee-window limits are not configured. +- Bad: Plus stores raw `sk-...` keys or exposes a full-key copy button. CPAMP + already owns raw key lifecycle. +- Bad: Tests assert HTTP 429/header propagation from executor output under the + current CPA ABI. That can pass only with a CPA core change. + +### 6. Tests Required +- Go unit: native CPA config parsing and CPAMP `api_key_aliases` loading. +- Go unit: Plus SQLite opener applies busy timeout, writable DB uses one open + connection, read-only auxiliary opens reject writes, and transient writer + locks wait instead of immediately returning `SQLITE_BUSY`. +- Go unit: admin key listing triggers native sync, reflects alias/deletion + changes, hides legacy rows by default, and keeps removed-native diagnostics + separate from the ordinary table. +- Go unit: sync lifecycle for new, removed, inherited, and ambiguous native + keys; historical usage does not move across native hash IDs. +- Go unit: policy denials cover missing policy, source removed, disabled, + model allowlist, RPM, 5H, 24H, 7D, month, and explicit zero limits. +- Go unit: admin HTML has no create/delete/rotate/raw-key-copy lifecycle + controls, no removed-row toggle, and visible missing-limit hints; user HTML + points to native `sk-...` keys. +- Integration: production user API session/me/usage/events/codexcont works with + an enabled native key. +- Integration: normal `/v1/models` and `/v1/responses` succeed with an enabled + native key; over-limit `/v1/responses` returns the structured error body. + +### 7. Wrong vs Correct + +#### Wrong +```text +CPA native key -> Plus raw-key store -> Plus alias/key lifecycle controls +``` + +This duplicates CPAMP's job and increases secret exposure. + +#### Correct +```text +CPA config api-keys + CPAMP aliases -> Plus native-key sync + -> Plus strategy/quota rows keyed by native hash -> cpa-usage user portal +``` + +Plus owns policy and accounting, not the raw key lifecycle. + +#### Wrong +```go +db.SetMaxOpenConns(1) +rows, _ := db.QueryContext(ctx, "select id from keys") +for rows.Next() { + db.ExecContext(ctx, "update keys set hidden=1 where id=?", id) +} +``` + +This can self-deadlock when the single writable SQLite connection is still +owned by the active query. + +#### Correct +```go +rows, _ := db.QueryContext(ctx, "select id from keys") +ids := collectIDsAndClose(rows) +for _, id := range ids { + db.ExecContext(ctx, "update keys set hidden=1 where id=?", id) +} +``` + +Close read cursors before writes on the same single-connection Plus DB. + +## Scenario: CPA Key Policy Plus historical key authority migration + +### 1. Scope / Trigger +- Trigger this spec whenever work touches `cpa_key_policy_plus_plugin/`, the + `cpa-key-policy-plus` CPA plugin config, `cpa-usage.konbakuyomu.us`, per-key + quota windows, retired Plus-owned `cpa_...` key compatibility, or migration + cleanup from the old `cpa-key-policy` plugin. +- This is cross-layer work: CPA dynamic plugin loading, old Key Policy JSON + import, Plus SQLite state, Caddy public/admin routing, user cookies, and + CodexCont/Governor deployment boundaries must agree. + +### 2. Signatures +- CPA plugin artifact: + `/CLIProxyAPI/plugins/linux/amd64/cpa-key-policy-plus.so`. +- CPA plugin config: + `plugins.configs.cpa-key-policy-plus` with `enabled`, `priority`, + `exclusive_auth`, `state_db_path`, `key_policy_state_path`, + `legacy_quota_db_path`, `governor_state_db_path`, `session_secret`, + `codexcont_enabled`, `codexcont_route`, `codexcont_url`, and `fail_mode`. +- SQLite tables owned by Plus: `keys`, `usage_events`, `reset_watermarks`, + `active_requests`, `active_sessions`, `audit_log`, `codexcont_summaries`, + and `settings`. `active_sessions` is retained for schema compatibility and + delete cleanup, but is not an enforcement source after the RPM-only cutover. +- Admin resource: `GET /v0/resource/plugins/cpa-key-policy-plus/admin`. +- Admin management routes: + - `GET /v0/management/plugins/cpa-key-policy-plus/keys` + - `GET /v0/management/plugins/cpa-key-policy-plus/models` + - `POST /v0/management/plugins/cpa-key-policy-plus/keys/create` + - `PUT /v0/management/plugins/cpa-key-policy-plus/keys/save` + - `PUT /v0/management/plugins/cpa-key-policy-plus/keys/limits` + - `POST /v0/management/plugins/cpa-key-policy-plus/keys/reset` + - `POST /v0/management/plugins/cpa-key-policy-plus/keys/delete` +- User resource: `GET /v0/resource/plugins/cpa-key-policy-plus/user`. +- Admin convenience route: + `https://cpa-admin.konbakuyomu.us/key-policy-plus/`. +- Admin API alias: + `https://cpa-admin.konbakuyomu.us/key-policy-plus/api/*` rewrites to the + corresponding Plus management route. This alias is admin-host only and must + not exist on `cpa.konbakuyomu.us`. +- User route: `https://cpa-usage.konbakuyomu.us/`. +- User API session creation is GET-only on CPA resource routes and sends the + raw user key in `X-CPA-Key-Policy-Plus-Key`; do not put the key in the URL. +- User session cookie name: `cpa_key_policy_plus_session`. Session creation + must refresh compatible cookies for `/`, + `/v0/resource/plugins/cpa-key-policy-plus/user`, and + `/key-policy-plus-user` so stale path-specific cookies from earlier routes + do not survive a successful login. + +### 3. Contracts +- This scenario is historical-migration context. In the current native-key + design, CPA/CPAMP owns raw native `sk-...` keys and aliases; Plus owns only + policy, quota, prices, resets, user sessions, and user usage projections. + The old `cpa-key-policy` plugin must stay disabled in config. +- Plus may load old Key Policy JSON and legacy Governor/usage-admin SQLite + watermarks as import/audit material, but ordinary admin/user surfaces must not + present retired Plus-owned `cpa_...` rows as current keys. +- Plus must never store or return raw API keys, Authorization headers, full key + hashes, cookies, request bodies, response bodies, OAuth tokens, or encrypted + reasoning content. +- User session validation must tolerate multiple cookies with the same + `cpa_key_policy_plus_session` name. Browsers may send both a stale + path-specific cookie and a fresh root cookie for plugin resource API paths; + the backend must try every candidate session token and accept the first valid + one instead of failing on the first invalid token. +- CPA plugin `ResourceRoute` is GET-only in the current host. The Plus admin + HTML may be served from a resource route, but create/save/reset mutations + must go through `/key-policy-plus/api/*` -> CPA management routes. Do not + send `POST` or `PUT` to `/v0/resource/plugins/cpa-key-policy-plus/admin/api/*`. +- The admin proxy route for `/key-policy-plus/api/*` must inject the CPA + management key from a mounted secret or equivalent process environment; do + not commit the raw key to Caddyfile, Trellis docs, or git. CPAMP iframe + context must not be relied on to add an Authorization header for embedded + plugin HTML, because the plugin page owns its own `fetch()` calls. +- The model catalog endpoint returns safe `ModelOption` projections only: + `id`, optional display metadata, `source`, and `known`. It may merge CPA host + model hints with already configured Plus models, but it must preserve unknown + configured models instead of deleting them when online discovery is empty or + stale. +- `5h`, `24h`, and `7d` are rolling USD windows. `month` is the current + Asia/Shanghai calendar month. Reset writes a soft watermark and does not + delete historical `usage_events`. +- The Plus user dashboard primary live view is fixed to `24h`. Do not expose a + top-level `5h/24h/7d/month` selector on the ordinary user page; keep + four-window quota visibility in side-by-side cards instead. The user APIs may + continue accepting range parameters for compatibility and future callers. +- Refresh cancellation is browser control flow, not a user-visible sync + failure. When a manual refresh, focus/pageshow refresh, or visibility change + aborts an older in-flight user-page fetch, the page must ignore that aborted + work instead of writing usage/protection error notices. +- Ordinary user throttling is RPM-only plus model allowlist and quota windows. + `concurrency` and `max_active_sessions` payload fields are compatibility + fields only: create/save handlers must accept stale payloads but persist and + return both values as `0`, and frontend auth must not read them. +- Plus no longer owns native key deletion. Deleting a current key happens in + CPA/CPAMP; the next Plus sync marks the native row removed/hidden for + ordinary views while preserving `usage_events` plus `codexcont_summaries` for + billing and troubleshooting history. Any legacy Plus-owned hard-delete route + is compatibility-only and must not be used for native key lifecycle. +- CPAMP native key aliases are stored in `api_key_aliases` inside CPAMP's + `usage.sqlite`, which runs in WAL mode in production. CPA must mount the + containing CPAMP data directory read-only, not only the bare `usage.sqlite` + file, so Plus can see `usage.sqlite-wal` / `usage.sqlite-shm` and newly + written aliases. A file-only bind can look healthy but show stale aliases such + as a hash preview instead of a newly renamed key. +- Plus alias loading may try multiple read-only SQLite paths and must treat a + missing `api_key_aliases` table as an empty source, not as a fatal key sync + failure. The CPA config `api-keys` list remains the current key source of + truth; aliases are display metadata matched by native key SHA256. +- Archive/restore has been retired. Stale archive routes may remain as + compatibility guards, but must return `410 archive_removed_use_delete` + instead of mutating key state. +- `exclusive_auth: true` lets Plus participate in CPA frontend auth. In the + current CPA host, policy rejection may be surfaced as CPA's generic `401` + `Missing API key` response because `frontendAuth` returns unauthenticated. + Treat that as an expected wrapper unless CPA adds typed auth-denial payloads. +- Current production request chain is `Codex -> Caddy -> CPA -> Plus policy -> + cpa-codexcont-executor -> upstream`. Do not reintroduce the Docker + CodexCont sidecar or Governor into the public `/v1/responses` path after the + executor plugin has passed production smoke. +- `cpa-admin.konbakuyomu.us/usage-admin/` must no longer serve stale local + limit controls after Plus cutover; return `404` or redirect to the Plus admin + page. + +### 4. Validation & Error Matrix +- Plus plugin missing/wrong architecture -> CPA logs do not show + `plugin loaded plugin_id=cpa-key-policy-plus`; deployment is not accepted. +- Old Key Policy enabled alongside Plus exclusive auth -> ordinary key + authority is ambiguous; disable old Key Policy before accepting cutover. +- Current valid native `sk-...` key -> `/v1/models`, `/v1/responses`, and user + session login succeed. Retired `cpa_...` keys should fail with migrated or + retired guidance. +- Valid full `cpa_...` key plus a stale same-name path-specific session cookie + -> user session login and `/user/api/me` still succeed; stale cookies must + not create a persistent `not_authenticated` loop after a successful login. +- Shortened key preview or rotated full key -> login fails; only the full key + shown at create/rotation can match the stored hash. +- Disabled/removed/disallowed/over-RPM/over-quota request -> CPA/Plus rejects + before upstream execution; the executor must not call the upstream provider + for a denied key. +- Stale create/save payload includes `concurrency` or `max_active_sessions` -> + Plus ignores the requested values and persists `0`. +- Stale archive route call -> `410 archive_removed_use_delete`. +- Native key removal in CPA/CPAMP -> Plus default admin key list no longer + returns that key after sync; history remains internal. +- `POST/PUT /v0/resource/plugins/cpa-key-policy-plus/admin/api/*` -> fails + before reaching plugin logic; this is a deployment/config bug if the admin + page depends on it. +- `/key-policy-plus/api/*` without a working CPA management-key injection -> + `401 missing management key` or `invalid_admin_key`; deployment is not + accepted until the alias returns safe Plus JSON and create/save/reset work. +- CPA/host model discovery unavailable -> `/models` still returns the union of + currently configured Plus model names and prices, with a warning instead of + stripping key allowlists. +- `cpa-usage.konbakuyomu.us` exposes only the user resource and user APIs. +- Public `cpa.konbakuyomu.us/v0/resource/plugins/*`, `/usage-admin*`, + `/codexcont*`, `/governor*`, `/management*`, `/key-policy-plus*`, and + `/admin*` -> `404`. + +### 5. Good/Base/Bad Cases +- Good: CPA logs show `cpa-key-policy-plus` loaded, Plus default admin key list + mirrors current CPA native keys, `cpa-usage` login works with a current native + key, and `/v1/responses` succeeds through the CPA -> Plus -> executor chain. +- Good: A browser with an old + `Path=/v0/resource/plugins/cpa-key-policy-plus/user` session cookie can log + in again; the new response refreshes all compatible paths and `/api/me` + accepts the fresh token even if the stale token is sent first. +- Base: A key has no usage yet. User login still shows key metadata, configured + models, prices, and empty usage tables. +- Good: The admin page is a resource HTML page, while its mutations use + `/key-policy-plus/api/*` and reach Plus management handlers with the CPA + management key injected by the admin proxy. +- Good: An unwanted key is removed in CPA/CPAMP; Plus ordinary key list drops it + after sync, the key can no longer log in or authenticate requests, while + historical usage and CodexCont summaries still exist by safe `key_id`. +- Bad: `cpa-usage` still reads `cpa_usage_portal` SQLite as the authority after + Plus is enabled. That preserves the split-brain limit problem. +- Bad: A retired key is only hidden or archived. That keeps a confusing second + lifecycle path and can make admins think a key was fully removed when the + permission row still exists. +- Bad: The Plus admin page tries to create keys through + `/v0/resource/plugins/cpa-key-policy-plus/admin/api/keys/create`. The current + CPA host treats ResourceRoute as GET-only, so writes fail before plugin code. +- Bad: Caddy is changed to route public `/v1/responses` directly to CPA before + executor-level folding exists. That bypasses the current 516/518n-2 + mitigation. + +### 6. Tests Required +- Go unit: old state import remains history-only, native key preservation, + raw-key hash login, rotation/deletion invalidation through native sync, + negative value validation, model allowlist, RPM, rolling/natural-month quota + windows, soft reset, retired concurrency/session fields forced to zero, and + cost projection. +- Go unit: admin/user HTML resources are `no-store`, user login uses + `X-CPA-Key-Policy-Plus-Key`, and responses do not leak raw keys/full hashes. +- Go unit: user session creation sets `cpa_key_policy_plus_session` cookies on + the root path and known user-resource aliases, and session lookup succeeds + when a stale same-name cookie appears before a fresh valid cookie in the + `Cookie` header. +- Go unit: user events and CodexCont summaries are filtered to the current key. +- Go unit: Plus user HTML has no range dropdown, fixes usage/events requests to + `range=24h`, keeps `24H / 7D` and `5H / 本月` quota cards, and ignores + refresh-cancel aborts before rendering sync errors. +- Go unit: admin HTML points mutations at `/key-policy-plus/api`, model + normalization preserves unknown configured models, and create/save/reset + through the admin alias persist settings. +- Go unit: native deletion from CPA config hides the key from ordinary Plus + admin responses, preserves usage and Codex summaries, and stale archive routes + return `410 archive_removed_use_delete`. +- Frontend/Playwright: Plus admin can create a key, select discovered models, + edit per-model prices, save, hard-delete a key, reload, and keep dense tables + horizontally scrollable on 390px without page-level overflow. +- Production smoke: plugin SHA256 matches the built artifact, CPA logs show + Plus loaded, Plus default admin key count matches current CPA native keys, + `cpa-usage` login works, admin backend `/key-policy-plus/` and + `/key-policy-plus/api/models` work, `usage-admin` backend returns `404`, and + public blocked paths return `404`. +- Production smoke: a tiny authenticated `/v1/responses` request succeeds + through the current CPA -> Plus -> executor route before handoff. + +### 7. Wrong vs Correct + +#### Wrong +```text +Plus enabled -> public /v1/responses -> Governor or Docker sidecar +``` + +This reintroduces a retired control plane and makes the request path harder to +reason about. + +#### Correct +```text +Codex -> Caddy -> CPA -> Plus policy -> cpa-codexcont-executor -> upstream +``` + +#### Wrong +```text +admin HTML -> POST /v0/resource/plugins/cpa-key-policy-plus/admin/api/keys/create +``` + +This depends on a mutating ResourceRoute, but the current CPA host dispatches +resource plugin routes as GET-only browser resources. + +#### Correct +```text +admin HTML -> /key-policy-plus/api/keys/create +admin proxy -> /v0/management/plugins/cpa-key-policy-plus/keys/create +``` + +The admin proxy injects the CPA management key from a mounted secret or process +environment, and the public API host still blocks `/key-policy-plus*`. + +#### Wrong +```text +admin "deletes" a key by setting enabled=false or archived=true +``` + +This leaves a permission row behind and keeps the old archive lifecycle alive. + +#### Correct +```text +admin HTML -> /key-policy-plus/api/keys/delete {"id":"...","confirm":"delete"} +Plus -> delete keys/reset_watermarks/active_sessions, keep safe history +``` + +Hard deletion removes the authority entry while preserving billing and +diagnostic records. + +#### Wrong +```text +Codex window count -> reject requests via max_active_sessions +``` + +Normal Codex conversations can reuse or fan out window/session metadata in ways +that make this limit noisy and hard to explain. + +#### Correct +```text +frontend auth -> enabled/deleted check -> model allowlist -> RPM -> quota windows +``` + +After the RPM-only cutover, concurrency/session fields are accepted only for +backward-compatible payload decoding and must be stored as zero. + +#### Wrong +```text +Cookie: cpa_key_policy_plus_session=stale-path-token; cpa_key_policy_plus_session=fresh-root-token +backend -> verify only the first same-name cookie -> 401 not_authenticated +``` + +Browser cookie path precedence can put an older path-specific cookie before the +fresh root cookie on plugin resource API requests, causing a login loop even +after session creation succeeds. + +#### Correct +```text +session creation -> Set-Cookie for root and known plugin/user aliases +session lookup -> collect all same-name session cookies -> accept first valid token +``` + +The backend must treat same-name cookies as a compatibility set, not as a +single trusted value. + +Separate the key authority migration from the continuation-owner migration. + +## Scenario: CodexCont executor-only CPA plugin + +### 1. Scope / Trigger +- Trigger this spec whenever work touches `cpa_codexcont_executor_plugin/`, + CPA executor routing for `/v1/responses`, or Plus protection-summary reads + from an executor store. +- This plugin replaces the Docker CodexCont sidecar only. It must not own + `cpa-usage.konbakuyomu.us`, ordinary user sessions, quota windows, RPM, or + `/user/api/*`. + +### 2. Signatures +- CPA plugin id: `cpa-codexcont-executor`. +- Artifact: `/CLIProxyAPI/plugins/linux/amd64/cpa-codexcont-executor.so`. +- Config keys: `enabled`, `route_enabled`, `state_db_path`, `fail_mode`, + `upstream_model`, `upstream_model_aliases`, `truncation_step`, + `max_continue`, and `marker_text`. +- Plus optional read-only bridge: + `plugins.configs.cpa-key-policy-plus.codex_summary_db_path`. +- Management routes are internal observability only: + `GET /plugins/cpa-codexcont-executor/status` and + `GET /plugins/cpa-codexcont-executor/summaries`. +- CPAMP/admin resource: + `GET /v0/resource/plugins/cpa-codexcont-executor/admin` renders a read-only + realtime rolling monitor for executor health and safe summaries. +- Resource API aliases for that monitor: + `GET /v0/resource/plugins/cpa-codexcont-executor/admin/api/status` and + `GET /v0/resource/plugins/cpa-codexcont-executor/admin/api/summaries`. + +### 3. Contracts +- `route_enabled=false` -> `model.route` returns unhandled. CPA keeps the + normal upstream path and the executor plugin does not provide continuation + protection. +- `route_enabled=true` -> only streaming Responses-style requests are routed to + the executor. Non-stream requests remain unhandled by this plugin. +- The executor stream owner opens upstream rounds through CPA host callbacks, + folds them into one downstream SSE stream, and preserves one logical terminal + event for the client. +- Upstream model aliasing belongs inside the executor plugin. If + `upstream_model` or `upstream_model_aliases` maps a client-visible model to a + provider-registered internal model, the host callback model and request body + model must be rewritten for upstream, while downstream SSE and safe summaries + keep the client-visible model. +- CPA host callbacks may return SSE as line-sized chunks without trailing + newlines. The executor parser must accept standalone `event:`, `data:`, + comment, and blank line chunks as complete SSE lines. +- The executor may persist only safe summaries: request id, key id, model, + protection state, round counters, reasoning counters, continuation count, + stopped/failure reason, timestamps, and safe diagnostics such as model alias + evidence and read byte counts. It must not persist or return request bodies, + response bodies, raw keys, Authorization headers, OAuth tokens, cookies, or + encrypted reasoning. +- The executor may register a CPAMP admin menu/resource for read-only + monitoring. This is the replacement for Governor's CodexCont protection + monitor only; it must not expose `/user`, `/user/api/*`, key editing, quota + editing, or ordinary user self-service. +- CPAMP must show exactly one executor sidebar entry. Internal management + routes such as `/plugins/cpa-codexcont-executor/status` and `/summaries` + must not set `Menu`; only the `/admin` resource may set + `Menu: CodexCont Executor`. +- If CPA/CPAMP requires resource-adjacent capabilities for resource menu + registration, the executor may declare non-exclusive + `frontend_auth_provider=true` and `usage_plugin=true` only as compatibility + shims. In that case `frontend_auth.authenticate` must always return + unauthenticated and `usage.handle` must be a no-op response. These shims must + not authenticate requests, persist usage events, calculate costs, mutate + quota windows, or become billing sources. +- CPA calls `plugin.reconfigure` after the initial `plugin.register` and + decodes the response through the same registration path. Executor plugins + must return full metadata and capabilities from both methods; returning only a + lightweight configured acknowledgement makes CPA mark the plugin + unregistered and drops CPAMP resource routes. +- Plus may read the executor SQLite store through `codex_summary_db_path` for + `/user/api/codexcont`; read failures degrade only protection summaries and + must not affect login, quota, `/user/api/usage`, or `/user/api/events`. +- After production traffic is verified on the executor plugin, the old Docker + sidecar chain must be retired all the way through operational entry points: + remove the stopped `codexcont` container and image explicitly, disable or + rename the default CodexCont compose file so it cannot be recreated by a + plain `docker compose up`, remove admin proxy routes that target + `codexcont:8787` or `cpa-governor`, and set Plus `codexcont_enabled: false` + so user-summary reads use the executor SQLite bridge instead of the old + sidecar admin API. Keep legacy Governor state only as read-only import/audit + material when Plus still needs it. + +### 4. Validation & Error Matrix +- Plugin registers frontend auth or user resources -> reject the change; Plus + owns the user portal. +- Plugin registers key/quota mutations or ordinary user resource APIs -> reject + the change; the CPAMP resource is observability-only. +- Any non-admin executor management route sets a CPAMP `Menu` label -> reject; + this creates duplicate sidebar entries and can expose raw JSON as a page. +- `frontend_auth.authenticate` authenticates a request, or `usage.handle` stores + records / changes quota or cost state -> reject; these capabilities are only + menu-registration shims. +- `route_enabled=false` but `model.route` handles a request -> reject; the + switch is not one-click safe. +- Missing executor summary DB -> Plus falls back to sidecar/local summaries or + returns an empty protection list, while usage APIs continue to pass. +- Upstream EOF before terminal event -> executor emits `response.incomplete` + and must not leak buffered tentative message/function-call output. +- Host stream returns line-sized SSE chunks -> executor must still emit the + terminal event; treating this as EOF/incomplete is a regression. +- Upstream alias configured -> upstream host callback uses the internal model + in callback metadata and body; downstream stream and summaries keep the + client-visible model. +- Executor default config must include stable Codex visible-model aliases for + currently exposed Codex client models. In this deployment, `gpt-5.4` and + `gpt-5.5` both default to the provider-registered upstream + `gpt-5.3-codex-spark`; production YAML may repeat or override those mappings, + but missing YAML entries must not make `gpt-5.5` fall through to an + unregistered upstream model. +- When routing to `gpt-5.3-codex-spark`, the executor must filter upstream-only + incompatible built-in tools such as `image_generation` before the host + callback. It must preserve custom/function tools, remove an emptied `tools` + array, clear any `tool_choice` that points to the filtered built-in, and keep + safe diagnostics such as `filtered_tool_types` without storing the request + body. +- A public `/v1/responses` failure that reaches the executor/host callback and + returns `authentication_error` with `auth_unavailable` plus wording such as + `Encountered invalidated oauth token for user` is a CPA Codex OAuth account + credential problem, not a Plus native-key sync problem. Verify + `/CLIProxyAPI/config.yaml` native key hashes, Plus admin key projection, and + `auth-dir` Codex account files separately before blaming API keys. +- Truncation fingerprint without encrypted reasoning -> executor must not open + a continuation round and must report `no_encrypted_content` metadata. +- After final sidecar cleanup, any live production config still containing + `reverse_proxy codexcont:8787`, a default + `/opt/codex-stacks/codexcont/docker-compose.yaml`, a `codexcont` Docker + container/image, or Plus `codexcont_enabled: true` is a rollback hazard and + must be fixed before calling the migration closed. + +### 5. Good/Base/Bad Cases +- Good: `cpa-usage.konbakuyomu.us` still serves Plus, while public + `/v1/responses` can later route through CPA and the executor plugin for + continuation protection. +- Good: The CPAMP plugin menu has `CodexCont Executor`, and it shows a + polling realtime monitor backed by executor status/summaries without key or + quota controls. +- Good: `/v0/resource/plugins/cpa-codexcont-executor/admin` is routable inside + the admin boundary, while `/v0/resource/plugins/cpa-codexcont-executor/status` + is not a resource page and returns `404` through resource dispatch. +- Good: A `gpt-5.5` streaming request opens the host callback with + `gpt-5.3-codex-spark`, while downstream events, summaries, and diagnostics + preserve the client-visible `gpt-5.5`. +- Base: executor plugin loaded with `route_enabled=false`; no request is + handled by the executor, and CPA remains usable without continuation folding. +- Base: All official native keys fail with the same `invalidated oauth token` + message even though Plus lists the right keys and executor aliasing is active. + Refresh or replace the CPA Codex OAuth auth JSON; do not rotate CPA native + keys to fix an upstream account-token failure. +- Bad: executor plugin registers `/user` or `/user/api/session`. That creates a + second user portal and conflicts with Key Policy Plus ownership. + +### 6. Tests Required +- Go unit: registration has executor/model-router capability but no frontend + auth, usage plugin, or user resources. +- Go unit: CPAMP admin monitor resource exists, returns `no-store` HTML, polls + status/summaries, and does not contain user/key/quota control endpoints. +- Go unit: executor management registration exposes exactly one CPAMP menu + entry, and non-admin management routes have empty `Menu` fields. +- Go unit: `plugin.reconfigure` returns full registration metadata and + capabilities, not only `{"configured": true}`. +- Go unit: executor `usage.handle` returns an observability-only no-op and does + not store billing or quota data. +- Go unit: executor `frontend_auth.authenticate` returns unauthenticated and is + non-exclusive. +- Go unit: route switch disabled/enabled behavior and non-stream fallback. +- Go unit: upstream model aliasing rewrites host callback metadata/body while + preserving downstream client-visible model and safe diagnostics. +- Go unit: default executor config aliases `gpt-5.5` to the configured Codex + upstream model and keeps `gpt-5.5` visible in emitted streams and summaries. +- Go unit: SSE parser accepts host callback line-chunked streams without + trailing newlines. +- Go unit: stream folding covers auto continuation, max continuation, missing + encrypted reasoning, upstream EOF, upstream error, monotonic sequence + numbers, and reconstructed proxy metadata. +- Go unit: Plus reads executor summaries by key id through + `codex_summary_db_path`, filters other users, and fails soft when the DB is + missing. + +### 7. Wrong vs Correct + +#### Wrong +```text +cpa-codexcont-executor -> registers user page -> cpa-usage host points there +``` + +This recreates the ownership confusion between usage portal and continuation +engine. + +#### Correct +```text +cpa-key-policy-plus -> owns cpa-usage and /user/api/* +cpa-codexcont-executor -> owns streaming Responses continuation and read-only CPAMP monitor only +Plus -> optional read-only summary bridge for display +``` + +The executor replaces the Docker sidecar and Governor's CodexCont monitor, not +the Plus user portal or key/quota controls. + +## Scenario: Local linux/amd64 Go plugin build toolchain + +### 1. Scope / Trigger +- Trigger this spec whenever building or rebuilding the Linux CPA plugin + artifacts for `cpa-codexcont-executor` or `cpa-key-policy-plus` from this + Windows/WSL workspace. +- This is an infra contract because build reproducibility, WSL placement, disk + usage, and production artifact SHA evidence all affect rollout safety. + +### 2. Signatures +- Preferred WSL Go binary: + `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go`. +- Project-scoped fallback WSL Go binary: + `/mnt/d/Dev/20_Software/_LocalRuntime/CodexCont/go-sdk-1.22.6/bin/go`. +- Cached Go tarball, if re-extraction is ever needed: + `/mnt/d/Dev/20_Software/_LocalRuntime/go/downloads/go1.22.6.linux-amd64.tar.gz`. +- Expected version for the current CPA plugin builds: + `go version go1.22.6 linux/amd64`. +- CPA plugin artifact build command shape: + `CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags cliproxy_plugin -buildmode=c-shared -o <plugin>.so .` + +### 3. Contracts +- Do not repeatedly download Go into WSL or `/tmp` for plugin builds when the + `_LocalRuntime` toolchain exists. +- Build/test commands must either call the preferred Go binary explicitly or + prepend its `bin` directory to `PATH` for that one command/session. +- If plain `go` is not in WSL `PATH`, that is not a blocker and must not start + a new download. Use the explicit `_LocalRuntime` path instead. +- If the extracted toolchain is missing but the cached tarball exists, ask + before re-extracting and place it under `_LocalRuntime`, not a throwaway + `/tmp/codex-go*` directory. +- Record plugin artifact SHA256 hashes after every production-bound rebuild. +- CLIProxyAPI CPA plugins are C ABI shared objects built with + `-tags cliproxy_plugin -buildmode=c-shared`. Do not use Go + `-buildmode=plugin`; that produces the wrong plugin ABI for this host. +- Do not commit local build directories or generated `.so` / `.h` files from + plugin builds. Keep them ignored and record only the production SHA evidence + in the Trellis task. + +### 4. Validation & Error Matrix +- Preferred Go path exists and reports `go1.22.6 linux/amd64` -> use it for + `go test ./...` and plugin builds. +- Plain WSL `go` is absent -> continue with the explicit `_LocalRuntime` Go + path; do not download. +- Preferred path missing but fallback path exists and reports the expected + version -> use the fallback and record that choice in task evidence. +- Both extracted toolchains missing -> pause before network download; check the + cached tarball and confirm the intended `_LocalRuntime` extraction target. +- Build artifact SHA not recorded -> rollout evidence is incomplete. +- Build uses `-buildmode=plugin` instead of `-buildmode=c-shared` -> artifact + is invalid for CPA deployment even if `go build` exits successfully. + +### 5. Good/Base/Bad Cases +- Good: A Linux plugin build uses + `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go`, + tests pass, `file` reports an ELF x86-64 shared object, and SHA256 is logged. +- Base: WSL has no global `go`; explicit `_LocalRuntime` Go still works. +- Bad: A helper script silently downloads Go again into `/tmp` because + `command -v go` returned empty. +- Bad: A local `build/` directory or generated plugin `.so` / `.h` appears in + `git status` as an untracked production artifact. + +### 6. Tests Required +- Shell smoke: `command -v go || true` plus the preferred explicit Go path + `version` check before any WSL plugin build. +- Go unit: run `go test ./...` in both plugin packages with the selected Go + binary. +- Artifact check: run `file` and `sha256sum` on production-bound `.so` files. +- Git check: `git status --short` must not include plugin build artifacts. + +### 7. Wrong vs Correct + +#### Wrong +```bash +command -v go || curl -fsSL https://go.dev/dl/go1.22.6.linux-amd64.tar.gz | tar -xz -C /tmp +``` + +This redownloads a large toolchain, hides the chosen compiler path, and leaves +throwaway state outside the project runtime convention. + +#### Correct +```bash +/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go test ./... +``` + +The build uses the already-provisioned local runtime toolchain and produces +repeatable evidence. + +## Scenario: CPA Governor plugin and CodexCont Engine rollout + +### 1. Scope / Trigger +- Trigger this spec whenever work touches `cpa_governor_plugin/`, Governor + plugin deployment, CPA plugin routing, `cpa-usage.konbakuyomu.us` user + routing, or CodexCont Engine routes. +- This is cross-layer work: CPA dynamic plugin loading, Key Policy state + import, SQLite usage state, Caddy public/admin routing, and CodexCont Engine + health all have to agree. +- Governor is owned by this repository. CPA, CPAMP, and CPA Key Policy remain + official upstream artifacts and must not be forked for this integration. + +### 2. Signatures +- CPA plugin artifact: + `/CLIProxyAPI/plugins/linux/amd64/cpa-governor.so`. +- CPA plugin config: + `plugins.configs.cpa-governor` with `enabled`, `priority`, + `exclusive_auth`, `state_db_path`, `key_policy_state_path`, + `session_secret`, `codexcont_enabled`, `codexcont_route`, + `codexcont_url`, and `fail_mode`. +- Admin resource: + `GET /v0/resource/plugins/cpa-governor/admin`. +- User resource: + `GET /v0/resource/plugins/cpa-governor/user`. +- Admin proxy convenience routes: + `https://cpa-admin.konbakuyomu.us/governor/` and + `https://cpa-admin.konbakuyomu.us/governor-user/`. +- Embedded admin CodexCont data channel: + `GET /governor/codexcont/admin/status`, + `GET /governor/codexcont/admin/requests?limit=N`, and + `GET /governor/codexcont/admin/logs/stream?once=1`. These are admin-host + only proxy paths to CodexCont `/admin/*`; the retired standalone + `/codexcont/` dashboard must return `404`. +- User portal route: + `https://cpa-usage.konbakuyomu.us/`. +- CodexCont Engine: + `GET /engine/healthz` and `POST /engine/v1/responses/analyze`. + +### 3. Contracts +- Deploy Governor first in passive mode unless a test-key executor cutover has + already passed: + `codexcont_enabled: true`, `codexcont_route: false`, + `exclusive_auth: false`. +- In passive mode, Governor may provide admin/user UI, key visibility, usage + storage, CodexCont health, and safe request projections, but it must not be + described as the exclusive quota enforcer or the executor-level 516 owner. +- Public `cpa.konbakuyomu.us` must block `/v0/resource/plugins/*`, + `/v0/management*`, `/admin*`, `/codexcont*`, `/governor*`, and related + management paths. +- `cpa-usage.konbakuyomu.us` may expose only the Governor user page and + `.../user/api/*`; it must return 404 for Governor admin resources and other + management paths. +- CPA plugin `ResourceRoute` dispatch is GET-only in the current CPA host. + User self-service APIs under `/v0/resource/plugins/cpa-governor/user/api/*` + must therefore use GET requests. Session creation passes the raw user key in + `X-CPA-Governor-Key` (or `X-CPA-User-Key`) so it does not collide with + CPAMP/management `Authorization` headers on embedded plugin pages. A direct + route may still accept `Authorization: Bearer <cpa_...>` as a fallback. Do + not put the key in the URL, and do not implement user-resource mutations as + POST unless they move behind a management route or another authenticated + proxy surface. +- User resource responses and plugin HTML must send `Cache-Control: no-store`. + CPAMP can keep a tab alive across plugin upgrades, so stale HTML/JS must not + be cached by the browser or an intermediate admin proxy. +- `cpa-admin.konbakuyomu.us/governor/` is protected by Cloudflare Access and + may route through the local admin proxy to CPA's plugin resource endpoint. + The Governor admin resource is read-only daily observability: it may show + CodexCont status, request summaries, hit rounds, latest reasoning counters, + continuation counts, failures, and advanced logs, but it must not expose Key + management, request-management tabs, or server-side CodexCont save controls. + Persistent Governor settings live in CPA's plugin configuration drawer. +- Do not put CPA management keys, API keys, OAuth tokens, cookies, or + encrypted reasoning into Caddy rewrites, browser URLs, Trellis docs, or git. +- The Governor user portal must explain key identity clearly: Key Policy + `cpa_...` full keys are accepted, native CPA `sk...` keys and shortened + previews are rejected with human-readable messages. +- When updating the plugin binary, record the SHA256 and verify CPA logs show + the plugin loaded and registered from the platform directory. +- Dense admin/user tables on mobile must keep a stable minimum table width + inside an overflowed panel. Do not let tables shrink until short fields turn + vertical. +- On SJC, rebuild/restart only the necessary self-owned service or plugin. + Do not use Docker prune and do not pull official images as a side effect of + a Governor-only rollout. + +### 4. Validation & Error Matrix +- CPA plugin file missing or wrong architecture -> CPA logs do not show + `plugin loaded plugin_id=cpa-governor`; deployment is not accepted. +- Governor admin API returns no Key Policy keys -> verify + `key_policy_state_path` and plugin-state mount before accepting the UI. +- `codexcont_route=false` -> production `/v1/responses` must still pass + through the existing working route and return a real successful response. +- Public `cpa.konbakuyomu.us/v0/resource/plugins/cpa-governor/admin` returns + 200 -> rollback Caddy public block before accepting the rollout. +- Public `cpa.konbakuyomu.us/governor/` or any + `/governor/codexcont/admin/*` path returns 200 -> rollback Caddy public + block before accepting the rollout. +- Public `cpa-usage.konbakuyomu.us/v0/resource/plugins/cpa-governor/admin` + returns 200 -> rollback user-host route before accepting the rollout. +- User API without session -> `401`; invalid CPA user key -> `401 + invalid_api_key`. +- User session with a native `sk...` key -> `401 + native_cpa_key_not_supported` and a message telling the user to use the full + Key Policy `cpa_...` key. +- User session with a shortened `cpa_...` preview -> `401 + key_preview_not_usable` and a message telling the user to use the full key + shown at create/rotation time. +- User session with a full but rotated/stale `cpa_...` key -> `401 + invalid_api_key`; validate by hashing the pasted key and comparing it with + the current Key Policy state before blaming Governor sync. +- Embedded CPAMP plugin page sends a CPAMP management bearer token in + `Authorization` plus user key in `X-CPA-Governor-Key` -> Governor must use + the dedicated user-key header and return `200` for a valid current key. +- `POST` to a user resource API -> CPA returns `404` before the plugin; the + browser UI must call these resource APIs with GET. +- Mobile Playwright snapshot shows table columns narrower than practical text + width or vertical labels -> add panel overflow/min-width and revalidate. + +### 5. Good/Base/Bad Cases +- Good: Governor is loaded by CPA, the sidebar `CPA Governor` page shows the + read-only CodexCont protection dashboard, user page opens on `cpa-usage`, the + embedded admin data channel works, retired `/codexcont/` returns 404, public + API admin paths return 404, and real `/v1/responses` still succeeds. +- Base: Governor user page opens but no user is logged in. `/user/api/me` + returns `401 not_authenticated`, and the page waits for a raw `cpa_...` key. +- Base: A Key Policy key is rotated. The old full key is unrecoverable and + should fail login; only the newly generated full key shown in the rotation + dialog can match the current `key_hash`. +- Bad: The user portal uses `Authorization` as its only login transport inside + CPAMP. The admin shell may already use that header, causing valid user keys + to be interpreted as invalid. +- Bad: Caddy sends public `/v1/responses` to Governor before the executor-level + continuation path is validated. This can bypass the known-good CodexCont + fold path. +- Bad: A deployment runs `docker compose up` against the CPA stack with + `pull_policy: always` during a plugin-only change on the small SJC disk. + This may pull new layers and introduce unrelated official-image drift. + +### 6. Tests Required +- Go unit: key hashing, Key Policy import, quota windows, pricing, redaction, + store usage events, admin/user handler responses. +- Go unit: user login must prefer `X-CPA-Governor-Key` over `Authorization`, + read headers case-insensitively, and mark JSON/HTML responses `no-store`. +- Go unit: admin HTML must be read-only and must not contain Key management, + request-detail tabs, CodexCont save actions, or spinner/diagonal animation + hooks. +- Go unit: user CodexCont summaries must be filtered to the current session key + by safe identity and must not leak other users' request ids. +- Python unit: CodexCont Engine summary projection and route smoke. +- Build verification: linux/amd64 `.so` SHA256 recorded and `file` reports an + ELF x86-64 shared object compatible with the Debian/glibc CPA image. +- Server smoke: `/healthz`, authenticated `/v1/models`, authenticated + `/v1/responses`, Governor admin/user pages, user-host 401/404 boundaries, + public API 404 boundaries, disk free space. +- Playwright: desktop and 390px mobile snapshots for Governor admin and user + pages; dense tables must be horizontally scrollable instead of vertically + compressed. + +### 7. Wrong vs Correct + +#### Wrong +```text +plugin UI added -> route every public path to /v0/resource/plugins/* +``` + +This exposes internal management resources and bypasses the public/admin host +boundary. + +#### Correct +```text +cpa-admin host -> Governor admin resource +cpa-usage host -> Governor user resource only +cpa API host -> official API plus explicit admin/plugin blocks +``` + +Each hostname exposes only the surface that matches its trust boundary. diff --git a/.trellis/spec/backend/database-guidelines.md b/.trellis/spec/backend/database-guidelines.md new file mode 100644 index 0000000..b61aa78 --- /dev/null +++ b/.trellis/spec/backend/database-guidelines.md @@ -0,0 +1,51 @@ +# Database Guidelines + +> Database patterns and conventions for this project. + +--- + +## Overview + +<!-- +Document your project's database conventions here. + +Questions to answer: +- What ORM/query library do you use? +- How are migrations managed? +- What are the naming conventions for tables/columns? +- How do you handle transactions? +--> + +(To be filled by the team) + +--- + +## Query Patterns + +<!-- How should queries be written? Batch operations? --> + +(To be filled by the team) + +--- + +## Migrations + +<!-- How to create and run migrations --> + +(To be filled by the team) + +--- + +## Naming Conventions + +<!-- Table names, column names, index names --> + +(To be filled by the team) + +--- + +## Common Mistakes + +<!-- Database-related mistakes your team has made --> + +(To be filled by the team) diff --git a/.trellis/spec/backend/directory-structure.md b/.trellis/spec/backend/directory-structure.md new file mode 100644 index 0000000..9bb253d --- /dev/null +++ b/.trellis/spec/backend/directory-structure.md @@ -0,0 +1,54 @@ +# Directory Structure + +> How backend code is organized in this project. + +--- + +## Overview + +<!-- +Document your project's backend directory structure here. + +Questions to answer: +- How are modules/packages organized? +- Where does business logic live? +- Where are API endpoints defined? +- How are utilities and helpers organized? +--> + +(To be filled by the team) + +--- + +## Directory Layout + +``` +<!-- Replace with your actual structure --> +src/ +├── ... +└── ... +``` + +--- + +## Module Organization + +<!-- How should new features/modules be organized? --> + +(To be filled by the team) + +--- + +## Naming Conventions + +<!-- File and folder naming rules --> + +(To be filled by the team) + +--- + +## Examples + +<!-- Link to well-organized modules as examples --> + +(To be filled by the team) diff --git a/.trellis/spec/backend/error-handling.md b/.trellis/spec/backend/error-handling.md new file mode 100644 index 0000000..bcd5533 --- /dev/null +++ b/.trellis/spec/backend/error-handling.md @@ -0,0 +1,51 @@ +# Error Handling + +> How errors are handled in this project. + +--- + +## Overview + +<!-- +Document your project's error handling conventions here. + +Questions to answer: +- What error types do you define? +- How are errors propagated? +- How are errors logged? +- How are errors returned to clients? +--> + +(To be filled by the team) + +--- + +## Error Types + +<!-- Custom error classes/types --> + +(To be filled by the team) + +--- + +## Error Handling Patterns + +<!-- Try-catch patterns, error propagation --> + +(To be filled by the team) + +--- + +## API Error Responses + +<!-- Standard error response format --> + +(To be filled by the team) + +--- + +## Common Mistakes + +<!-- Error handling mistakes your team has made --> + +(To be filled by the team) diff --git a/.trellis/spec/backend/index.md b/.trellis/spec/backend/index.md new file mode 100644 index 0000000..bfc2cd2 --- /dev/null +++ b/.trellis/spec/backend/index.md @@ -0,0 +1,39 @@ +# Backend Development Guidelines + +> Best practices for backend development in this project. + +--- + +## Overview + +This directory contains guidelines for backend development. Fill in each file with your project's specific conventions. + +--- + +## Guidelines Index + +| Guide | Description | Status | +|-------|-------------|--------| +| [Directory Structure](./directory-structure.md) | Module organization and file layout | To fill | +| [Database Guidelines](./database-guidelines.md) | ORM patterns, queries, migrations | To fill | +| [Error Handling](./error-handling.md) | Error types, handling strategies | To fill | +| [Quality Guidelines](./quality-guidelines.md) | Code standards, forbidden patterns | To fill | +| [Logging Guidelines](./logging-guidelines.md) | Structured logging, log levels | To fill | +| [Codex Continuation Contracts](./codex-continuation-contracts.md) | Responses continuation, CPA integration, and egress contracts | Active | + +--- + +## How to Fill These Guidelines + +For each guideline file: + +1. Document your project's **actual conventions** (not ideals) +2. Include **code examples** from your codebase +3. List **forbidden patterns** and why +4. Add **common mistakes** your team has made + +The goal is to help AI assistants and new team members understand how YOUR project works. + +--- + +**Language**: All documentation should be written in **English**. diff --git a/.trellis/spec/backend/logging-guidelines.md b/.trellis/spec/backend/logging-guidelines.md new file mode 100644 index 0000000..bb930df --- /dev/null +++ b/.trellis/spec/backend/logging-guidelines.md @@ -0,0 +1,51 @@ +# Logging Guidelines + +> How logging is done in this project. + +--- + +## Overview + +<!-- +Document your project's logging conventions here. + +Questions to answer: +- What logging library do you use? +- What are the log levels and when to use each? +- What should be logged? +- What should NOT be logged (PII, secrets)? +--> + +(To be filled by the team) + +--- + +## Log Levels + +<!-- When to use each level: debug, info, warn, error --> + +(To be filled by the team) + +--- + +## Structured Logging + +<!-- Log format, required fields --> + +(To be filled by the team) + +--- + +## What to Log + +<!-- Important events to log --> + +(To be filled by the team) + +--- + +## What NOT to Log + +<!-- Sensitive data, PII, secrets --> + +(To be filled by the team) diff --git a/.trellis/spec/backend/quality-guidelines.md b/.trellis/spec/backend/quality-guidelines.md new file mode 100644 index 0000000..c1e1065 --- /dev/null +++ b/.trellis/spec/backend/quality-guidelines.md @@ -0,0 +1,51 @@ +# Quality Guidelines + +> Code quality standards for backend development. + +--- + +## Overview + +<!-- +Document your project's quality standards here. + +Questions to answer: +- What patterns are forbidden? +- What linting rules do you enforce? +- What are your testing requirements? +- What code review standards apply? +--> + +(To be filled by the team) + +--- + +## Forbidden Patterns + +<!-- Patterns that should never be used and why --> + +(To be filled by the team) + +--- + +## Required Patterns + +<!-- Patterns that must always be used --> + +(To be filled by the team) + +--- + +## Testing Requirements + +<!-- What level of testing is expected --> + +(To be filled by the team) + +--- + +## Code Review Checklist + +<!-- What reviewers should check --> + +(To be filled by the team) diff --git a/.trellis/spec/guides/code-reuse-thinking-guide.md b/.trellis/spec/guides/code-reuse-thinking-guide.md new file mode 100644 index 0000000..bb789e9 --- /dev/null +++ b/.trellis/spec/guides/code-reuse-thinking-guide.md @@ -0,0 +1,223 @@ +# Code Reuse Thinking Guide + +> **Purpose**: Stop and think before creating new code - does it already exist? + +--- + +## The Problem + +**Duplicated code is the #1 source of inconsistency bugs.** + +When you copy-paste or rewrite existing logic: +- Bug fixes don't propagate +- Behavior diverges over time +- Codebase becomes harder to understand + +--- + +## Before Writing New Code + +### Step 1: Search First + +```bash +# Search for similar function names +grep -r "functionName" . + +# Search for similar logic +grep -r "keyword" . +``` + +### Step 2: Ask These Questions + +| Question | If Yes... | +|----------|-----------| +| Does a similar function exist? | Use or extend it | +| Is this pattern used elsewhere? | Follow the existing pattern | +| Could this be a shared utility? | Create it in the right place | +| Am I copying code from another file? | **STOP** - extract to shared | + +--- + +## Common Duplication Patterns + +### Pattern 1: Copy-Paste Functions + +**Bad**: Copying a validation function to another file + +**Good**: Extract to shared utilities, import where needed + +### Pattern 2: Similar Components + +**Bad**: Creating a new component that's 80% similar to existing + +**Good**: Extend existing component with props/variants + +### Pattern 3: Repeated Constants + +**Bad**: Defining the same constant in multiple files + +**Good**: Single source of truth, import everywhere + +### Pattern 4: Repeated Payload Field Extraction + +**Bad**: Multiple consumers cast the same JSON/event fields locally: + +```typescript +const description = (ev as { description?: string }).description; +const context = (ev as { context?: ContextEntry[] }).context; +``` + +This is duplicated contract logic even when the code is only two lines. Each +consumer now has its own definition of what a valid payload means. + +**Good**: Put the decoder, type guard, or projection next to the data owner: + +```typescript +if (isThreadEvent(ev)) { + renderThreadEvent(ev); +} +``` + +**Rule**: If the same untyped payload field is read in 2+ places, create a +shared type guard / normalizer / projection before adding a third reader. + +--- + +## When to Abstract + +**Abstract when**: +- Same code appears 3+ times +- Logic is complex enough to have bugs +- Multiple people might need this + +**Don't abstract when**: +- Only used once +- Trivial one-liner +- Abstraction would be more complex than duplication + +--- + +## After Batch Modifications + +When you've made similar changes to multiple files: + +1. **Review**: Did you catch all instances? +2. **Search**: Run grep to find any missed +3. **Consider**: Should this be abstracted? + +### Reducers Should Use Exhaustive Structure + +When state is derived from action-like values (`action`, `kind`, `status`, +`phase`), prefer a reducer with one `switch` over scattered `if/else` updates. + +```typescript +// BAD - action-specific state transitions are hard to audit +if (action === "opened") { ... } +else if (action === "comment") { ... } +else if (action === "status") { ... } + +// GOOD - one reducer owns the transition table +switch (event.action) { + case "opened": + ... + return; + case "comment": + ... + return; +} +``` + +This matters when the event log is the source of truth. A reducer is the +documented replay model; display code and commands should not duplicate pieces +of that replay model. + +--- + +## Checklist Before Commit + +- [ ] Searched for existing similar code +- [ ] No copy-pasted logic that should be shared +- [ ] No repeated untyped payload field extraction outside a shared decoder +- [ ] Constants defined in one place +- [ ] Similar patterns follow same structure +- [ ] Reducer/action transitions live in one reducer or command dispatcher + +--- + +## Gotcha: Python if/elif/else Exhaustive Check + +**Problem**: Python's if/elif/else chains have no compile-time exhaustive check. When you add a new value to a `Literal` type (e.g., `Platform`), existing if/elif/else chains silently fall through to `else` with wrong defaults. + +**Symptom**: New platform works partially — some methods return Claude defaults instead of platform-specific values. No error is raised. + +**Example** (`cli_adapter.py`): +```python +# BAD: "gemini" falls through to else, returns "claude" +@property +def cli_name(self) -> str: + if self.platform == "opencode": + return "opencode" + else: + return "claude" # gemini silently gets "claude"! + +# GOOD: explicit branch for every platform +@property +def cli_name(self) -> str: + if self.platform == "opencode": + return "opencode" + elif self.platform == "gemini": + return "gemini" + else: + return "claude" +``` + +**Prevention**: When adding a new value to a Python `Literal` type, search for ALL if/elif/else chains that switch on that type and add explicit branches. Don't rely on `else` being correct for new values. + +--- + +## Gotcha: Asymmetric Mechanisms Producing Same Output + +**Problem**: When two different mechanisms must produce the same file set (e.g., recursive directory copy for init vs. manual `files.set()` for update), structural changes (renaming, moving, adding subdirectories) only propagate through the automatic mechanism. The manual one silently drifts. + +**Symptom**: Init works perfectly, but update creates files at wrong paths or misses files entirely. + +**Prevention**: +- **Best**: Eliminate the asymmetry — have the manual path call the automatic one (e.g., `collectTemplateFiles()` calls `getAllScripts()` instead of maintaining its own list) +- **If asymmetry is unavoidable**: Add a regression test that compares outputs from both mechanisms +- When migrating directory structures, search for ALL code paths that reference the old structure + +**Real example**: `trellis update` had a manual `files.set()` list for 11 scripts that `getAllScripts()` already tracked. Fix: replaced the manual list with a `for..of getAllScripts()` loop. See `update.ts` refactor in v0.4.0-beta.3. + +--- + +## Template File Registration (Trellis-specific) + +When adding new files to `src/templates/trellis/scripts/`: + +**Single registration point**: `src/templates/trellis/index.ts` + +1. Add `export const xxxScript = readTemplate("scripts/path/file.py");` +2. Add to `getAllScripts()` Map + +That's it. `commands/update.ts` uses `getAllScripts()` directly — no manual sync needed. + +**Why this matters**: Without registration in `getAllScripts()`, `trellis update` won't sync the file to user projects. Bug fixes and features won't propagate. + +**History**: Before v0.4.0-beta.3, `update.ts` had its own hand-maintained file list that frequently fell out of sync with `getAllScripts()`. This caused 11 Python files to be silently skipped during `trellis update`. The fix was to eliminate the duplicate list and use `getAllScripts()` as the single source of truth. + +### Quick Checklist for New Scripts + +```bash +# After adding a new .py file, verify it's in getAllScripts(): +grep -l "newFileName" src/templates/trellis/index.ts # Should match +``` + +### Template Sync Convention + +`.trellis/scripts/` (dogfooded) and `packages/cli/src/templates/trellis/scripts/` (template) must stay identical. After editing `.trellis/scripts/`, always sync: + +```bash +rsync -av --delete --exclude='__pycache__' .trellis/scripts/ packages/cli/src/templates/trellis/scripts/ +``` + +**Gotcha**: Running rsync with wrong source/destination paths can create nested garbage directories (e.g., `.trellis/scripts/packages/cli/...`). Always double-check paths before running. diff --git a/.trellis/spec/guides/cross-layer-thinking-guide.md b/.trellis/spec/guides/cross-layer-thinking-guide.md new file mode 100644 index 0000000..9686546 --- /dev/null +++ b/.trellis/spec/guides/cross-layer-thinking-guide.md @@ -0,0 +1,327 @@ +# Cross-Layer Thinking Guide + +> **Purpose**: Think through data flow across layers before implementing. + +--- + +## The Problem + +**Most bugs happen at layer boundaries**, not within layers. + +Common cross-layer bugs: + +- API returns format A, frontend expects format B +- Database stores X, service transforms to Y, but loses data +- Multiple layers implement the same logic differently + +--- + +## Before Implementing Cross-Layer Features + +### Step 1: Map the Data Flow + +Draw out how data moves: + +``` +Source → Transform → Store → Retrieve → Transform → Display +``` + +For each arrow, ask: + +- What format is the data in? +- What could go wrong? +- Who is responsible for validation? + +### Step 2: Identify Boundaries + +| Boundary | Common Issues | +| --------------------- | --------------------------------- | +| API ↔ Service | Type mismatches, missing fields | +| Service ↔ Database | Format conversions, null handling | +| Backend ↔ Frontend | Serialization, date formats | +| Component ↔ Component | Props shape changes | + +### Step 3: Define Contracts + +For each boundary: + +- What is the exact input format? +- What is the exact output format? +- What errors can occur? + +--- + +## Common Cross-Layer Mistakes + +### Mistake 1: Implicit Format Assumptions + +**Bad**: Assuming date format without checking + +**Good**: Explicit format conversion at boundaries + +### Mistake 2: Scattered Validation + +**Bad**: Validating the same thing in multiple layers + +**Good**: Validate once at the entry point + +### Mistake 3: Leaky Abstractions + +**Bad**: Component knows about database schema + +**Good**: Each layer only knows its neighbors + +### Mistake 4: Every Consumer Parses The Same Payload + +**Bad**: A command reads JSONL events and casts fields inline: + +```typescript +const thread = (ev as { thread?: string }).thread; +const labels = (ev as { labels?: string[] }).labels; +``` + +This looks local, but it means every consumer owns a private version of the +event contract. The next field change will update one command and miss another. + +**Good**: Decode once at the event boundary, then export typed projections: + +```typescript +if (!isThreadEvent(ev)) return false; +return ev.thread === filter.thread; +``` + +**Rule**: For append-only logs, JSON streams, RPC payloads, or config files, +create one owner for: + +- event / payload type definitions +- type guards and normalization from `unknown` +- metadata projections used by UI commands +- reducers that replay state from the source of truth + +Rendering code may format fields, but it must not redefine the payload contract. + +--- + +## Checklist for Cross-Layer Features + +Before implementation: + +- [ ] Mapped the complete data flow +- [ ] Identified all layer boundaries +- [ ] Defined format at each boundary +- [ ] Decided where validation happens + +After implementation: + +- [ ] Tested with edge cases (null, empty, invalid) +- [ ] Verified error handling at each boundary +- [ ] Checked data survives round-trip +- [ ] Checked that consumers import shared decoders / projections instead of + casting payload fields locally +- [ ] Checked that derived state points back to the source event identifier + (`seq`, `id`, `version`) instead of inventing a second cursor + +--- + +## Cross-Platform Template Consistency + +In Trellis, command templates (e.g., `record-session.md`) exist in **multiple platforms** with identical or near-identical content. This is a cross-layer boundary. + +### Checklist: After Modifying Any Command Template + +- [ ] Find all platforms with the same command: `find src/templates/*/commands/trellis/ -name "<command>.*"` +- [ ] Update all platform copies (Markdown `.md` and TOML `.toml`) +- [ ] For Gemini TOML: adapt line continuations (`\\` vs `\`) and triple-quoted strings +- [ ] Run `/trellis:check-cross-layer` to verify nothing was missed + +**Real-world example**: Updated `record-session.md` in Claude to use `--mode record`, but forgot iFlow, Kilo, OpenCode, and Gemini — caught by cross-layer check. + +--- + +## Generated Runtime Template Upgrade Consistency + +Some generated files are both documentation and runtime input. In Trellis, +`.trellis/workflow.md` is parsed by `get_context.py`, `workflow_phase.py`, +SessionStart filters, and per-turn hooks. Template changes must be validated +against both fresh init and upgrade paths. + +### Checklist: After Modifying A Runtime-Parsed Template + +- [ ] Identify every runtime parser that reads the template, not just the file + writer that installs it +- [ ] Check whether relevant syntax lives outside obvious managed regions + such as tag blocks +- [ ] Verify fresh `init` output and a versioned `update` scenario that writes + the older `.trellis/.version` +- [ ] Add an upgrade regression using an older pristine template fixture, then + assert the installed file reaches the current packaged shape +- [ ] Update the backend spec that owns the runtime contract + +--- + +## Versioned Documentation Boundary + +Versioned documentation is a cross-layer boundary: source paths, `docs.json` +version routing, and the rendered version selector must all describe the same +release line. + +### Checklist: Before Editing Versioned Docs + +- [ ] Identify the target release line: stable, beta, or RC +- [ ] Verify the edited MDX path matches that line: + - stable: `docs-site/{start,advanced,...}` and `docs-site/zh/{start,advanced,...}` + - beta: `docs-site/beta/**` and `docs-site/zh/beta/**` + - RC: `docs-site/rc/**` and `docs-site/zh/rc/**` +- [ ] Verify `docs.json` navigation points the version label to the same paths +- [ ] Grep the opposite tree for release-line-specific terms before committing +- [ ] Treat beta content appearing under root release paths as a source-path bug, + not a rendering bug + +**Real-world example**: A beta-only task workflow change documented +`prd.md` + `design.md` + `implement.md`, task-creation consent, and Codex +mode banners under root `start/` and `advanced/` paths. The docs site then +served 0.6 beta behavior under the Release selector. The fix was to restore root +release docs, move the 0.6 content to `beta/` and `zh/beta/`, and add a grep +audit for beta markers against the root release tree. + +**Real-world example**: Codex inline mode changed workflow platform markers from +`[Codex]` / `[Kilo, Antigravity, Windsurf]` to `[codex-sub-agent]` / +`[codex-inline, Kilo, Antigravity, Windsurf]`. Fresh init was correct, but +`trellis update` only merged `[workflow-state:*]` blocks and preserved stale +markers outside those blocks. Result: upgraded projects got new hook scripts +but old workflow routing, so `get_context.py --mode phase --platform codex` +could return empty Phase 2.1 detail. + +--- + +## Mode-Detection Probe Checklist + +When a CLI auto-detects a mode by probing a remote resource (e.g., checking if `index.json` exists to decide marketplace vs direct download): + +### Before implementing: + +- [ ] Probe runs in **ALL** code paths that use the result (interactive, `-y`, `--flag` combos) +- [ ] 404 vs transient error are distinguished — don't treat both as "not found" +- [ ] Transient errors **abort or retry**, never silently switch modes +- [ ] Shared state (caches, prefetched data) is **reset** when context changes (e.g., user switches source) +- [ ] **Shortcut paths** (e.g., `--template` skipping picker) must have the same error-handling quality as the probed path — check that downstream functions don't call catch-all wrappers + +### After implementing: + +- [ ] Trace every path from probe result to the mode-decision branch — no fallthrough +- [ ] External format contracts (giget URI, raw URLs) are tested or at least documented as comments +- [ ] Metadata reads consume a complete response or use a streaming parser — never parse a fixed-size prefix as full JSON +- [ ] When reconstructing a composite identifier from parsed parts, verify **all** fields are included and in the **correct position** (e.g., `provider:repo/path#ref` not `provider:repo#ref/path`) +- [ ] Verify that **action functions** called after a shortcut don't internally use the old catch-all fetch — they must use the probe-quality variant when error distinction matters + +**Real-world example**: Custom registry flow had 8 bugs across 3 review rounds: (1) probe only ran in interactive mode, (2) transient errors fell through to wrong mode, (3) giget URI had `#ref` in wrong position, (4) prefetched templates leaked across source switches, (5) `--template` shortcut bypassed probe but `downloadTemplateById` internally used catch-all `fetchTemplateIndex`, turning timeouts into "Template not found". + +**Real-world example**: Agent-session update hints fetched npm `latest` metadata with `response.read(4096)` and then parsed it as complete JSON. The `@mindfoldhq/trellis` package metadata exceeded 4 KB, so the JSON was truncated, parse failed silently, and the first session injection showed no update hint. Fix: read the complete response before parsing, and add a regression where `version` is followed by an 8 KB metadata tail. + +--- + +## Cross-Platform Template Consistency + +In Trellis, command templates (e.g., `record-session.md`) exist in **multiple platforms** with identical or near-identical content. This is a cross-layer boundary. + +### Checklist: After Modifying Any Command Template + +- [ ] Find all platforms with the same command: `find src/templates/*/commands/trellis/ -name "<command>.*"` +- [ ] Update all platform copies (Markdown `.md` and TOML `.toml`) +- [ ] For Gemini TOML: adapt line continuations (`\\` vs `\`) and triple-quoted strings +- [ ] Run `/trellis:check-cross-layer` to verify nothing was missed + +**Real-world example**: Updated `record-session.md` in Claude to use `--mode record`, but forgot iFlow, Kilo, OpenCode, and Gemini — caught by cross-layer check. + +--- + +## Generated Runtime Template Upgrade Consistency + +Some generated files are both documentation and runtime input. In Trellis, +`.trellis/workflow.md` is parsed by `get_context.py`, `workflow_phase.py`, +SessionStart filters, and per-turn hooks. Template changes must be validated +against both fresh init and upgrade paths. + +### Checklist: After Modifying A Runtime-Parsed Template + +- [ ] Identify every runtime parser that reads the template, not just the file + writer that installs it +- [ ] Check whether relevant syntax lives outside obvious managed regions + such as tag blocks +- [ ] Verify fresh `init` output and a versioned `update` scenario that writes + the older `.trellis/.version` +- [ ] Add an upgrade regression using an older pristine template fixture, then + assert the installed file reaches the current packaged shape +- [ ] Update the backend spec that owns the runtime contract + +**Real-world example**: Codex inline mode changed workflow platform markers from +`[Codex]` / `[Kilo, Antigravity, Windsurf]` to `[codex-sub-agent]` / +`[codex-inline, Kilo, Antigravity, Windsurf]`. Fresh init was correct, but +`trellis update` only merged `[workflow-state:*]` blocks and preserved stale +markers outside those blocks. Result: upgraded projects got new hook scripts +but old workflow routing, so `get_context.py --mode phase --platform codex` +could return empty Phase 2.1 detail. + +--- + +## Mode-Detection Probe Checklist + +When a CLI auto-detects a mode by probing a remote resource (e.g., checking if `index.json` exists to decide marketplace vs direct download): + +### Before implementing: +- [ ] Probe runs in **ALL** code paths that use the result (interactive, `-y`, `--flag` combos) +- [ ] 404 vs transient error are distinguished — don't treat both as "not found" +- [ ] Transient errors **abort or retry**, never silently switch modes +- [ ] Shared state (caches, prefetched data) is **reset** when context changes (e.g., user switches source) +- [ ] **Shortcut paths** (e.g., `--template` skipping picker) must have the same error-handling quality as the probed path — check that downstream functions don't call catch-all wrappers + +### After implementing: +- [ ] Trace every path from probe result to the mode-decision branch — no fallthrough +- [ ] External format contracts (giget URI, raw URLs) are tested or at least documented as comments +- [ ] Metadata reads consume a complete response or use a streaming parser — never parse a fixed-size prefix as full JSON +- [ ] When reconstructing a composite identifier from parsed parts, verify **all** fields are included and in the **correct position** (e.g., `provider:repo/path#ref` not `provider:repo#ref/path`) +- [ ] Verify that **action functions** called after a shortcut don't internally use the old catch-all fetch — they must use the probe-quality variant when error distinction matters + +**Real-world example**: Custom registry flow had 8 bugs across 3 review rounds: (1) probe only ran in interactive mode, (2) transient errors fell through to wrong mode, (3) giget URI had `#ref` in wrong position, (4) prefetched templates leaked across source switches, (5) `--template` shortcut bypassed probe but `downloadTemplateById` internally used catch-all `fetchTemplateIndex`, turning timeouts into "Template not found". + +**Real-world example**: Agent-session update hints fetched npm `latest` metadata with `response.read(4096)` and then parsed it as complete JSON. The `@mindfoldhq/trellis` package metadata exceeded 4 KB, so the JSON was truncated, parse failed silently, and the first session injection showed no update hint. Fix: read the complete response before parsing, and add a regression where `version` is followed by an 8 KB metadata tail. + +--- + +## When to Create Flow Documentation + +Create detailed flow docs when: + +- Feature spans 3+ layers +- Multiple teams are involved +- Data format is complex +- Feature has caused bugs before + +--- + +## Event Log / Projection Boundary + +Append-only logs are cross-layer contracts. A single event travels through: + +``` +CLI input → event writer → events.jsonl → reader → filter → reducer → display +``` + +### Checklist: After Adding A New Event Kind Or Field + +- [ ] Add the event kind to the central event taxonomy +- [ ] Add a typed event variant or type guard at the event layer +- [ ] Add normalization helpers for array/object fields that come from + user input or JSON +- [ ] Keep `seq` / `id` assignment in the event writer only +- [ ] Make filters and reducers consume the typed event guard, not local casts +- [ ] Make display code consume reducer output or typed events, not raw JSON +- [ ] Add at least one regression that proves history replay and live filtering + use the same filter model + +**Real-world example**: Thread channels added `kind: "thread"`, `description`, +`context`, labels, and `lastSeq`. The first implementation replayed thread +state correctly, but several commands still re-parsed event payload fields with +local casts. The fix was to make the core event layer own `ThreadChannelEvent` +and `isThreadEvent`, make `reduceChannelMetadata` the only channel metadata +projection, and make `reduceThreads` the only thread replay reducer. diff --git a/.trellis/spec/guides/index.md b/.trellis/spec/guides/index.md new file mode 100644 index 0000000..56c6d77 --- /dev/null +++ b/.trellis/spec/guides/index.md @@ -0,0 +1,97 @@ +# Thinking Guides + +> **Purpose**: Expand your thinking to catch things you might not have considered. + +--- + +## Why Thinking Guides? + +**Most bugs and tech debt come from "didn't think of that"**, not from lack of skill: + +- Didn't think about what happens at layer boundaries → cross-layer bugs +- Didn't think about code patterns repeating → duplicated code everywhere +- Didn't think about edge cases → runtime errors +- Didn't think about future maintainers → unreadable code + +These guides help you **ask the right questions before coding**. + +--- + +## Available Guides + +| Guide | Purpose | When to Use | +|-------|---------|-------------| +| [Code Reuse Thinking Guide](./code-reuse-thinking-guide.md) | Identify patterns and reduce duplication | When you notice repeated patterns | +| [Cross-Layer Thinking Guide](./cross-layer-thinking-guide.md) | Think through data flow across layers | Features spanning multiple layers | + +--- + +## Quick Reference: Thinking Triggers + +### When to Think About Cross-Layer Issues + +- [ ] Feature touches 3+ layers (API, Service, Component, Database) +- [ ] Data format changes between layers +- [ ] Multiple consumers need the same data +- [ ] You're not sure where to put some logic +- [ ] You are adding an event kind, JSONL record, RPC payload, or config field +- [ ] UI / command code starts casting raw payload fields directly + +→ Read [Cross-Layer Thinking Guide](./cross-layer-thinking-guide.md) + +### When to Think About Code Reuse + +- [ ] You're writing similar code to something that exists +- [ ] You see the same pattern repeated 3+ times +- [ ] You're adding a new field to multiple places +- [ ] **You're modifying any constant or config** +- [ ] **You're creating a new utility/helper function** ← Search first! +- [ ] Two files read the same untyped payload field with local casts +- [ ] Multiple branches update the same derived state from `kind` / `action` + +→ Read [Code Reuse Thinking Guide](./code-reuse-thinking-guide.md) + +### When Verifying AI Cross-Review Results + +- [ ] Reviewer claims "user input can be malicious" → Check the actual data source (internal manifest? user config? external API?) +- [ ] Reviewer flags "missing validation" → Is the data from a trusted internal source? +- [ ] Reviewer says "behavior change" → Read the code comments — is it intentional design? +- [ ] Reviewer identifies a "bug" in test → Mentally delete the feature being tested — does the test still pass? If yes → tautological test + +**Common AI reviewer false-positive patterns**: +1. **Trust boundary confusion**: Treating internal data (bundled JSON manifests) as untrusted external input +2. **Ignoring design comments**: Flagging intentional behavior documented in code comments as bugs +3. **Variable misreading**: Not tracing a variable to its actual definition (e.g., Map keyed by path vs name) + +**Verification rule**: Every CRITICAL/WARNING finding must be verified against the actual code before prioritizing. Budget ~35% false-positive rate for AI reviews. + +--- + +## Pre-Modification Rule (CRITICAL) + +> **Before changing ANY value, ALWAYS search first!** + +```bash +# Search for the value you're about to change +grep -r "value_to_change" . +``` + +This single habit prevents most "forgot to update X" bugs. + +--- + +## How to Use This Directory + +1. **Before coding**: Skim the relevant thinking guide +2. **During coding**: If something feels repetitive or complex, check the guides +3. **After bugs**: Add new insights to the relevant guide (learn from mistakes) + +--- + +## Contributing + +Found a new "didn't think of that" moment? Add it to the relevant guide. + +--- + +**Core Principle**: 30 minutes of thinking saves 3 hours of debugging. diff --git a/.trellis/tasks/00-bootstrap-guidelines/prd.md b/.trellis/tasks/00-bootstrap-guidelines/prd.md new file mode 100644 index 0000000..9bee139 --- /dev/null +++ b/.trellis/tasks/00-bootstrap-guidelines/prd.md @@ -0,0 +1,126 @@ +# Bootstrap Task: Fill Project Development Guidelines + +**You (the AI) are running this task. The developer does not read this file.** + +The developer just ran `trellis init` on this project for the first time. +`.trellis/` now exists with empty spec scaffolding, and this bootstrap task +exists under `.trellis/tasks/`. When they want to work on it, they should start +this task from a session that provides Trellis session identity. + +**Your job**: help them populate `.trellis/spec/` with the team's real +coding conventions. Every future AI session — this project's +`trellis-implement` and `trellis-check` sub-agents — auto-loads spec files +listed in per-task jsonl manifests. Empty spec = sub-agents write generic +code. Real spec = sub-agents match the team's actual patterns. + +Don't dump instructions. Open with a short greeting, figure out if the repo +has any existing convention docs (CLAUDE.md, .cursorrules, etc.), and drive +the rest conversationally. + +--- + +## Status (update the checkboxes as you complete each item) + +- [ ] Fill backend guidelines +- [ ] Add code examples + +--- + +## Spec files to populate + + +### Backend guidelines + +| File | What to document | +|------|------------------| +| `.trellis/spec/backend/directory-structure.md` | Where different file types go (routes, services, utils) | +| `.trellis/spec/backend/database-guidelines.md` | ORM, migrations, query patterns, naming conventions | +| `.trellis/spec/backend/error-handling.md` | How errors are caught, logged, and returned | +| `.trellis/spec/backend/logging-guidelines.md` | Log levels, format, what to log | +| `.trellis/spec/backend/quality-guidelines.md` | Code review standards, testing requirements | + + +### Thinking guides (already populated) + +`.trellis/spec/guides/` contains general thinking guides pre-filled with +best practices. Customize only if something clearly doesn't fit this project. + +--- + +## How to fill the spec + +### Step 1: Import from existing convention files first (preferred) + +Search the repo for existing convention docs. If any exist, read them and +extract the relevant rules into the matching `.trellis/spec/` files — +usually much faster than documenting from scratch. + +| File / Directory | Tool | +|------|------| +| `CLAUDE.md` / `CLAUDE.local.md` | Claude Code | +| `AGENTS.md` | Codex / Claude Code / agent-compatible tools | +| `.cursorrules` | Cursor | +| `.cursor/rules/*.mdc` | Cursor (rules directory) | +| `.windsurfrules` | Windsurf | +| `.clinerules` | Cline | +| `.roomodes` | Roo Code | +| `.github/copilot-instructions.md` | GitHub Copilot | +| `.vscode/settings.json` → `github.copilot.chat.codeGeneration.instructions` | VS Code Copilot | +| `CONVENTIONS.md` / `.aider.conf.yml` | aider | +| `CONTRIBUTING.md` | General project conventions | +| `.editorconfig` | Editor formatting rules | + +### Step 2: Analyze the codebase for anything not covered by existing docs + +Scan real code to discover patterns. Before writing each spec file: +- Find 2-3 real examples of each pattern in the codebase. +- Reference real file paths (not hypothetical ones). +- Document anti-patterns the team clearly avoids. + +### Step 3: Document reality, not ideals + +**Critical**: write what the code *actually does*, not what it should do. +Sub-agents match the spec, so aspirational patterns that don't exist in the +codebase will cause sub-agents to write code that looks out of place. + +If the team has known tech debt, document the current state — improvement +is a separate conversation, not a bootstrap concern. + +--- + +## Quick explainer of the runtime (share when they ask "why do we need spec at all") + +- Every AI coding task spawns two sub-agents: `trellis-implement` (writes + code) and `trellis-check` (verifies quality). +- Each task has `implement.jsonl` / `check.jsonl` manifests listing which + spec files to load. +- The platform hook auto-injects those spec files + the task's `prd.md` + into every sub-agent prompt, so the sub-agent codes/reviews per team + conventions without anyone pasting them manually. +- Source of truth: `.trellis/spec/`. That's why filling it well now pays + off forever. + +--- + +## Completion + +When the developer confirms the checklist items above are done with real +examples (not placeholders), guide them to run: + +```bash +python ./.trellis/scripts/task.py finish +python ./.trellis/scripts/task.py archive 00-bootstrap-guidelines +``` + +After archive, every new developer who joins this project will get a +`00-join-<slug>` onboarding task instead of this bootstrap task. + +--- + +## Suggested opening line + +"Welcome to Trellis! Your init just set me up to help you fill the project +spec — a one-time setup so every future AI session follows the team's +conventions instead of writing generic code. Before we start, do you have +any existing convention docs (CLAUDE.md, .cursorrules, CONTRIBUTING.md, +etc.) I can pull from, or should I scan the codebase from scratch?" diff --git a/.trellis/tasks/00-bootstrap-guidelines/task.json b/.trellis/tasks/00-bootstrap-guidelines/task.json new file mode 100644 index 0000000..9eb6e63 --- /dev/null +++ b/.trellis/tasks/00-bootstrap-guidelines/task.json @@ -0,0 +1,28 @@ +{ + "id": "00-bootstrap-guidelines", + "name": "00-bootstrap-guidelines", + "title": "Bootstrap Guidelines", + "description": "Fill in project development guidelines for AI agents", + "status": "in_progress", + "dev_type": "docs", + "scope": null, + "package": null, + "priority": "P1", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-01", + "completedAt": null, + "branch": null, + "base_branch": null, + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [ + ".trellis/spec/backend/" + ], + "notes": "First-time setup task created by trellis init (backend project)", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/check.jsonl b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/check.jsonl new file mode 100644 index 0000000..d3b4a2a --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/check.jsonl @@ -0,0 +1,2 @@ +{"file": ".trellis/spec/backend/codex-continuation-contracts.md", "reason": "Validate continuation metadata, secret logging, and production CPA route invariants."} +{"file": ".trellis/spec/guides/cross-layer-thinking-guide.md", "reason": "Validate status/log event contracts from backend to frontend."} diff --git a/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/design.md b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/design.md new file mode 100644 index 0000000..0d79f99 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/design.md @@ -0,0 +1,65 @@ +# Design + +## Architecture + +Dashboard functionality lives inside the existing CodexCont Starlette app. CPA remains an official upstream service and is not forked or extended for this first version. + +Production data path remains unchanged: + +`Codex -> cpa.konbakuyomu.us/v1/responses -> Caddy -> CodexCont -> CPA -> OpenAI` + +Admin view path: + +`Browser -> Cloudflare Access -> cpa-admin.konbakuyomu.us/codexcont/ -> cpa-admin-proxy -> codexcont:8787/admin/` + +## Backend Contracts + +- `GET /admin/healthz` returns `{ "ok": true }` plus process uptime. +- `GET /admin/status` returns process metrics, recent counters, upstream health, and a safe config summary. +- `GET /admin/requests?limit=N` returns recent request-level protection summaries. Each summary is a redacted, + memory-only projection of internal diagnostics events keyed by the generated request id. +- `GET /admin/logs?limit=N` returns the newest redacted in-memory log events. +- `GET /admin/logs/stream` uses `text/event-stream`; new log events are emitted as `event: log`, and request + summary updates are emitted as `event: request`. +- `GET /admin/` serves the static dashboard. Static assets can be embedded or served under `/admin/static/...`; no Node build is required. + +## Metrics And Logs + +- Add a small diagnostics module responsible for: + - monotonic service start time and uptime + - bounded ring buffer + - subscriber queues for SSE + - request counters and active request tracking + - continuation/truncation/failure counters + - log redaction for sensitive-looking strings and headers +- Instrument these points: + - request accepted / passthrough / fold start + - round decision with reasoning token count and continuation decision + - continuation opened + - request finished cleanly or failed + - upstream health probe result +- Use generated request IDs for dashboard correlation only; do not expose upstream tokens or reasoning payloads. +- Add a request summary projection inside `Diagnostics` so frontend code does not re-derive protection meaning from raw log fields. +- Request protection result values: + - `protected_clean`: folded request completed without a truncation fingerprint. + - `auto_continued`: a truncation fingerprint was detected and a hidden continuation round opened. + - `risk_uncontinued`: a truncation fingerprint was detected but continuation was blocked by a guard. + - `passthrough`: request did not enter folding protection. + - `failed`, `incomplete`, `processing`: failure, incomplete upstream ending, or still active. + +## Frontend + +- Single static operational dashboard, not a landing page. +- Chinese-first operational copy. +- Compact top band: CodexCont health, CPA health, continuation config, active requests, SSE connection status. +- Metrics grid: total requests, folded/protected requests, continuations, truncation hits, failures. +- Primary table: recent requests with protection result chips, model, rounds, reasoning token count, continuation count, final result, and expandable round details. +- Advanced log table remains available below the primary request view, with filters, pause/autoscroll, clear local view, and Chinese event labels. +- Styling is plain CSS with restrained colors, max 8px card radius, stable dimensions, responsive grid, and no decorative gradient/orb background. + +## Deployment Contract + +- Public API host `cpa.konbakuyomu.us` must not expose `/admin` or `/codexcont`. +- Existing `cpa-admin-proxy` should route `/codexcont/` to `codexcont:8787/admin/` with path stripping; all other paths continue to CPA management. +- Cloudflare Access remains the outer auth layer. CodexCont admin routes do not implement their own login for v1. +- Server rollout must back up current CodexCont stack and `cpa-admin-proxy` config before changes. diff --git a/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/implement.jsonl b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/implement.jsonl new file mode 100644 index 0000000..98b4e35 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/implement.jsonl @@ -0,0 +1,3 @@ +{"file": ".trellis/spec/backend/codex-continuation-contracts.md", "reason": "Continuation and CPA integration constraints for instrumenting /v1/responses without changing fold semantics."} +{"file": ".trellis/spec/guides/cross-layer-thinking-guide.md", "reason": "Dashboard events cross backend, SSE, and frontend rendering boundaries."} +{"file": ".trellis/spec/guides/code-reuse-thinking-guide.md", "reason": "Diagnostics owns one event/metrics projection instead of duplicated payload parsing."} diff --git a/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/implement.md b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/implement.md new file mode 100644 index 0000000..24d1a4a --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/implement.md @@ -0,0 +1,167 @@ +# Implementation Plan + +## 1. Task And Spec Prep + +- Confirm clean Git state and current Trellis task. +- Read applicable Trellis specs before code edits. +- Keep sensitive deployment values out of Trellis artifacts. + +## 2. Backend Diagnostics + +- Add a diagnostics module with a bounded in-memory ring buffer, metrics snapshot, subscriber queues, and redaction helpers. +- Add admin routes to the Starlette app. +- Add lightweight instrumentation in `handle_responses` and `fold_stream` for lifecycle and continuation events. +- Add upstream health probe from `/admin/status` using the configured CPA upstream base. + +## 3. Frontend + +- Add static dashboard HTML/CSS/JS served by CodexCont. +- Use EventSource for `/admin/logs/stream` and fetch `/admin/status` periodically. +- Include filters, pause/autoscroll controls, local clear, connection state, and responsive layout. + +## 4. Local Validation + +- Add/extend tests for diagnostics ring buffer, redaction, SSE stream, admin routes, and current middleware behavior. +- Run the existing test suite. +- Use Playwright or a local browser smoke to capture desktop and mobile dashboard states. + +## 5. Server Rollout + +- Back up current `/opt/codex-stacks/codexcont` and `/opt/codex-stacks/cpa-admin-tunnel` config files to a root-only timestamped path. +- Rebuild/restart only the CodexCont stack; do not prune Docker. +- Update `cpa-admin-proxy` so `/codexcont/` proxies to `codexcont:8787/admin/`. +- Verify public API host still blocks admin paths. +- Verify `cpa-admin.konbakuyomu.us/codexcont/` loads through Cloudflare Access. + +## 6. Acceptance Evidence + +- Record local test results, server route checks, and dashboard live-log proof in `implement.md`. +- Specifically record whether this active Codex conversation or another real `/v1/responses` request appears in live logs. +- Commit code and Trellis artifacts with narrow staging. + +## 7. V2 Chinese Protection Dashboard + +- Add request-level summary projection in `middleware.diagnostics.Diagnostics`. +- Add `GET /admin/requests` and `event: request` SSE updates. +- Replace the English log-first dashboard with a Chinese request-first dashboard. +- Keep raw logs as an advanced section and preserve the existing `event: log` SSE stream. +- Validate all protection states locally: protected clean, auto-continued, risk-uncontinued, passthrough, failed, incomplete/processing where possible. + +### V2 Local Implementation + +- Added memory-only recent request summaries with protection results: + - `protected_clean` + - `auto_continued` + - `risk_uncontinued` + - `passthrough` + - `failed` + - `incomplete` + - `processing` +- Added `GET /admin/requests?limit=N`. +- Extended `/admin/logs/stream` so it still emits `event: log` and also emits `event: request`. +- Rebuilt `middleware/dashboard.html` as a Chinese request-first operational page with raw logs moved to an advanced section. +- Updated README and README_zh dashboard documentation. + +### V2 Local Validation + +- `.venv\Scripts\python.exe -m py_compile middleware\app.py middleware\admin.py middleware\diagnostics.py middleware\proxy.py middleware\config.py tests\test_middleware.py`: passed. +- `.venv\Scripts\python.exe tests\test_middleware.py`: `136/136 checks passed`. +- `git diff --check`: passed; only existing Windows LF/CRLF warnings were reported. +- Playwright CLI + Edge validated `http://127.0.0.1:8797/admin/`: + - page title `CodexCont 保护状态面板` + - desktop 1440px horizontal overflow `false` + - mobile 390px horizontal overflow `false` + - simulated request states rendered: protected clean, auto-continued, risk-uncontinued, passthrough, failed + - console errors/warnings: `0` +- Controlled local invalid JSON request returned HTTP `400` and appeared in `/admin/requests` with `protection=failed`, `failure_reason=invalid_json_body`. +- Request summary output strips internal timing field `_started_perf` before reaching admin APIs or SSE. + +### V2 Server Rollout + +- Backup path: `/root/codexcont-dashboard-v2-backups/20260701T085718Z`. +- Disk before rebuild: `/dev/sda1` about `8.7G used / 820M available / 92%`. +- Uploaded updated production files: + - `/opt/codex-stacks/codexcont/app/middleware/admin.py` + - `/opt/codex-stacks/codexcont/app/middleware/app.py` + - `/opt/codex-stacks/codexcont/app/middleware/dashboard.html` + - `/opt/codex-stacks/codexcont/app/middleware/diagnostics.py` +- Rebuilt/restarted only `codexcont` with `docker compose up -d --build --no-deps codexcont`. +- No Docker prune, image prune, recursive delete, or bulk directory deletion was used. + +### V2 Server Validation + +- `codexcont`: running after rebuild. +- Disk after rollout: `/dev/sda1` about `8.7G used / 833M available / 92%`. +- Local admin proxy: + - `http://127.0.0.1:8327/codexcont/status`: HTTP `200`, `ok=True`, CPA upstream health `200`, `log_retention=800`. + - `http://127.0.0.1:8327/codexcont/requests?limit=5`: HTTP `200`. + - `http://127.0.0.1:8327/codexcont/`: contains `CodexCont 保护状态面板`, `最近请求`, and `高级日志`. + - `http://127.0.0.1:8327/codexcont/logs/stream?once=1`: emitted `ready`, `request`, and `log` events. +- Public API domain: + - `https://cpa.konbakuyomu.us/healthz`: HTTP `200`. + - `https://cpa.konbakuyomu.us/admin/requests`: HTTP `404`. + - `https://cpa.konbakuyomu.us/codexcont/requests`: HTTP `404`. + - `https://cpa.konbakuyomu.us/management.html`: HTTP `404`. +- Cloudflare Access: + - `https://cpa-admin.konbakuyomu.us/codexcont/`: unauthenticated request returned HTTP `302` to Cloudflare Access. +- Live proof: + - Real current `gpt-5.5` Codex traffic appeared in recent requests with `protection=protected_clean`. + - A controlled public invalid JSON request returned HTTP `400` and appeared in recent requests with `protection=failed`, `failure_reason=invalid_json_body`. + - Final `/codexcont/requests?limit=5` check reported `has_internal_perf=False`. + +## Rollback + +- Restore backed-up CodexCont stack files and restart `codexcont`. +- Restore backed-up `cpa-admin-proxy` config if `/codexcont/` routing breaks CPA management access. +- Caddy public API routing should not need rollback if it remains unchanged. + +## Execution Evidence + +### Local Implementation + +- Added `middleware.diagnostics` as the single in-memory owner for metrics, ring buffer events, subscribers, and redaction. +- Added read-only admin routes and static dashboard under `/admin/`. +- Instrumented `/v1/responses` request start, passthrough/fold start, round decision, continuation open, finish, and failure events. +- Added `[admin].max_log_events = 800` default config. +- Updated README and README_zh with dashboard usage and project layout. + +### Local Validation + +- `.venv\Scripts\python.exe -m py_compile middleware\app.py middleware\admin.py middleware\diagnostics.py middleware\proxy.py middleware\config.py tests\test_middleware.py`: passed. +- `.venv\Scripts\python.exe tests\test_middleware.py`: `123/123 checks passed`. +- `git diff --check`: passed. +- Playwright via Node REPL + system Edge validated `http://127.0.0.1:8797/admin/`: + - desktop 1440px: title `CodexCont Dashboard`, SSE `Live`, status cards `3`, metric cards `4`, horizontal overflow `false`. + - mobile 390px: horizontal overflow `false`, log table rendered, local invalid `/v1/responses` event appeared in the log table. + +### Server Rollout + +- Backup path: `/root/codexcont-dashboard-backups/20260701T080223Z`. +- Disk before rollout: `/dev/sda1` around `8.7G used / 832-833M available / 92%`. +- Uploaded updated CodexCont middleware files to `/opt/codex-stacks/codexcont/app/middleware/`. +- Added production `[admin] max_log_events = 800` to `/opt/codex-stacks/codexcont/config.toml`. +- Updated `/opt/codex-stacks/cpa-admin-tunnel/Caddyfile` so: + - `/codexcont/` routes to `codexcont:8787/admin/`. + - all other paths continue to `cpa:8317`. +- `docker exec cpa-admin-proxy caddy validate --config /etc/caddy/Caddyfile`: valid. +- Rebuilt/restarted only `codexcont`; restarted only `cpa-admin-proxy`. +- No Docker prune or bulk filesystem deletion was used. + +### Server Validation + +- Local admin proxy: + - `http://127.0.0.1:8327/codexcont/`: HTTP `200`. + - `http://127.0.0.1:8327/codexcont/status`: HTTP `200`; CPA upstream health `200`, `log_retention=800`. + - `http://127.0.0.1:8327/management.html`: HTTP `200`, preserving CPA management access. +- Cloudflare Access: + - `https://cpa-admin.konbakuyomu.us/codexcont/`: HTTP `302` to Cloudflare Access login when unauthenticated. + - `https://cpa-admin.konbakuyomu.us/management.html`: HTTP `302` to Cloudflare Access login when unauthenticated. +- Public API domain: + - `https://cpa.konbakuyomu.us/healthz`: HTTP `200`. + - `https://cpa.konbakuyomu.us/admin/`: HTTP `404`. + - `https://cpa.konbakuyomu.us/codexcont/`: HTTP `404`. + - `https://cpa.konbakuyomu.us/management.html`: HTTP `404`. +- Live-log proof: + - Current real Codex conversation traffic appeared as `fold_start`, `round_decision`, and `request_finished` for `model=gpt-5.5`. + - After the redaction fix, `round_decision` preserved numeric `reasoning_tokens` such as `281`. + - A controlled public invalid JSON request to `/v1/responses` returned HTTP `400` and appeared as `request_failed reason=invalid_json_body`. diff --git a/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/prd.md b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/prd.md new file mode 100644 index 0000000..7feb858 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/prd.md @@ -0,0 +1,73 @@ +# CodexCont Status Dashboard + +## Goal + +Add a lightweight, server-side CodexCont dashboard that shows current service health, request/continuation metrics, and real-time logs for the production CodexCont sidecar. The page must help verify the live CPA chain used by this Codex conversation without modifying CPA or storing persistent logs on the small SJC disk. + +## Confirmed Facts + +- CodexCont is a Python Starlette proxy currently serving `/v1/responses`. +- Production traffic is routed as `cpa.konbakuyomu.us/v1/responses -> CodexCont -> CPA -> OpenAI`. +- CPA remains the official image; CodexCont owns the 516/518n-2 continuation mitigation. +- Admin access already works through `cpa-admin.konbakuyomu.us` via Cloudflare Tunnel + Cloudflare Access + CPA management key. +- SJC disk is small, so first version must avoid persistent logs, databases, Docker prune, and broad cleanup. +- This Codex conversation is expected to use the same production path, so the dashboard should be able to show live logs from real ongoing chat traffic. +- Current dashboard v1 is English-first and log-first. It exposes raw events such as `fold_start`, + `round_decision`, `continuation_opened`, and `request_finished`, but a beginner cannot quickly tell + whether a request was protected, clean, automatically continued, risky, or failed. + +## UX Refinement Request + +- Convert the dashboard to Chinese-first copy. +- Make the first screen explain operational state in beginner-readable terms, without requiring the user + to understand internal event names or raw fields. +- Promote request-level protection status above raw logs: + - protected and clean: the request passed through CodexCont folding/protection and did not hit the + 516/518n-2 truncation fingerprint. + - auto-continued: CodexCont detected the 516/518n-2 fingerprint and opened a hidden continuation round. + - risky/unhandled: the truncation fingerprint was seen but continuation could not be opened because a + guard stopped it. + - failed: request or upstream handling failed. +- Keep raw logs available as an advanced detail view for debugging. + +## Requirements + +- R1: Add read-only admin routes inside CodexCont: + - `GET /admin/healthz` + - `GET /admin/status` + - `GET /admin/logs` + - `GET /admin/logs/stream` + - `GET /admin/` +- R2: Track in-process metrics: uptime, active requests, total requests, continuation count, 516/518n-2 truncation hits, failure count, last request/continuation/error timestamps, and upstream CPA health. +- R3: Add an in-memory ring buffer for structured, redacted operational events. Do not record request bodies, authorization headers, API keys, OAuth tokens, or encrypted reasoning content. +- R4: Stream logs to the browser with SSE so the page updates without manual refresh. +- R5: Provide a compact operational frontend with status cards, metrics, live log table, filters, pause/autoscroll controls, and mobile-safe layout. +- R6: Expose the page only through `https://cpa-admin.konbakuyomu.us/codexcont/`; keep public `https://cpa.konbakuyomu.us` from exposing `/admin` or `/codexcont`. +- R7: Keep logs memory-only by default, with a bounded retention size around 500-1000 entries. +- R8: Preserve existing `/v1/responses` behavior and current continuation semantics. +- R9: Add Chinese dashboard labels and beginner-readable status explanations. +- R10: Add a request-centric view that groups events by request id and surfaces protection state, model, + rounds, reasoning token counts, continuation count, and final status. +- R11: Clearly distinguish "经过 CodexCont 保护但无需续写" from "检测到 516/518n-2 并已自动续写". + +## Acceptance Criteria + +- [x] Trellis artifacts record PRD, design, implementation steps, deployment evidence, and residual risks. +- [x] Local tests pass for ring buffer retention, subscriber broadcast, redaction, admin route smoke, and existing middleware behavior. +- [x] Local frontend check confirms the dashboard renders without overlapping text on desktop and mobile viewports. +- [x] `GET /admin/status` reports live metrics and redacted config summary. +- [x] `GET /admin/logs/stream` emits live SSE events when requests flow through CodexCont. +- [x] On SJC, `https://cpa-admin.konbakuyomu.us/codexcont/` opens through the existing Cloudflare Access path. +- [x] This active Codex conversation or a real `/v1/responses` request appears in the dashboard live logs. +- [x] `https://cpa.konbakuyomu.us/admin/` and `https://cpa.konbakuyomu.us/codexcont/` are not publicly exposed. +- [x] No secrets are printed or committed, no persistent log store is added, and no Docker prune or bulk deletion is used. +- [x] Dashboard first screen is Chinese-first and readable for non-technical users. +- [x] Recent requests show beginner-readable protection status without opening raw logs. +- [x] Requests that triggered automatic continuation are visually distinct from clean protected requests. + +## Out Of Scope + +- CPA plugin implementation for v1. +- Long-term historical analytics, log database, login system, or multi-user RBAC. +- Changing CPA auth/account scheduling, OpenAI OAuth tokens, or 516 continuation logic beyond instrumentation hooks. +- Deleting old stack data or cleaning server disk outside explicitly named files. diff --git a/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/task.json b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/task.json new file mode 100644 index 0000000..72ad0ca --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-codexcont-status-dashboard/task.json @@ -0,0 +1,26 @@ +{ + "id": "codexcont-status-dashboard", + "name": "codexcont-status-dashboard", + "title": "CodexCont status dashboard", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-01", + "completedAt": "2026-07-01", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "CodexCont dashboard v2 is implemented and deployed. Local tests passed 136/136; Playwright/Edge verified Chinese request-first UI, desktop/mobile no-overflow layout, all protection chips, and SSE request/log behavior. SJC cpa-admin path /codexcont/ routes through Cloudflare Access to CodexCont; public cpa.konbakuyomu.us admin/codexcont/management paths return 404. Live requests showed current gpt-5.5 traffic as protected_clean and a controlled invalid_json_body event as failed.", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/check.jsonl b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/design.md b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/design.md new file mode 100644 index 0000000..52d030b --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/design.md @@ -0,0 +1,56 @@ +# Design + +## Architecture + +Production API data path: + +`Codex client -> cpa.konbakuyomu.us -> caddy-edge -> codexcont:8787 -> cpa:8317 -> CPA Codex executor -> socks5://172.19.0.1:1082 -> OpenAI` + +Direct CPA paths remain: + +`client -> cpa.konbakuyomu.us -> caddy-edge -> cpa:8317` + +Management path: + +`browser -> Cloudflare Access -> cpa-admin.konbakuyomu.us -> Cloudflare Tunnel -> 127.0.0.1:8327 -> cpa-admin-proxy -> cpa:8317 -> CPA management panel` + +## Runtime Layout + +- CodexCont stack: `/opt/codex-stacks/codexcont` +- CodexCont config: `/opt/codex-stacks/codexcont/config.toml` +- CodexCont app source: copied from this repository into `/opt/codex-stacks/codexcont/app` +- CodexCont container: `codexcont` +- CodexCont Docker network: `cpa_net` +- CPA admin tunnel stack: `/opt/codex-stacks/cpa-admin-tunnel` +- CPA admin loopback proxy: `cpa-admin-proxy`, host bind `127.0.0.1:8327` +- CF token file: `/opt/codex-stacks/cpa-admin-tunnel/.env`, root-only, never committed +- Backup root: `/root/cpa-codexcont-admin-backups/<timestamp>/` + +## Caddy Contract + +- `cpa.konbakuyomu.us /v1/responses` reverse proxies to `codexcont:8787`. +- `cpa.konbakuyomu.us /management.html`, `/v0/management*`, and `/v0/resource/plugins/*` return a blocking status. +- All other CPA traffic reverse proxies to `cpa:8317`. +- Rollback is restoring the backed-up Caddyfile and reloading Caddy. + +## CPA Management Contract + +- `remote-management.secret-key` is set to a generated high-entropy secret. +- `remote-management.allow-remote` remains `false`. +- The `config.yaml` bind mount becomes writable because CPA hashes and persists plaintext management keys at startup. +- The plaintext management key is stored in a root-only credential file for the user to retrieve over SSH; task artifacts only record the path. +- Docker-published `127.0.0.1:8317` reaches CPA from a bridge address, so CPA does not treat it as a local client under `allow-remote: false`. The admin proxy keeps `allow-remote: false` by setting local-only forwarding headers before proxying to CPA. + +## Cloudflare Tunnel Contract + +- The user supplies a Cloudflare Dashboard Tunnel token out of band. +- The cloudflared container runs with host networking and targets `http://127.0.0.1:8327`. +- Cloudflare Access must protect `cpa-admin.konbakuyomu.us`; CPA management key remains the second layer. +- If the token is not available during implementation, leave the tunnel stack prepared but not running, and record the exact start command. + +## Local Evaluation Contract + +- Do not modify `C:\Users\dxt98\.codex`. +- Create a temporary local test `CODEX_HOME` with only the minimal `config.toml` and auth material needed for the eval. +- Point the test provider at `https://cpa.konbakuyomu.us/v1` with `wire_api = "responses"`. +- Run `D:\Dev\50_Scripts\52_Python\codex-candy-eval\codex_candy_eval.py` from its own directory. diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/implement.jsonl b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/implement.md b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/implement.md new file mode 100644 index 0000000..b27f6cd --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/implement.md @@ -0,0 +1,150 @@ +# Implementation Plan + +## 1. Preflight + +- Confirm Git status and active task. +- Confirm SJC disk, Docker state, CPA/Caddy containers, networks, and current management endpoint status. +- Confirm no secrets are printed in command output. + +## 2. Backups + +- Create `/root/cpa-codexcont-admin-backups/<timestamp>/`. +- Back up `/opt/codex-stacks/cpa/docker-compose.yaml`, `/opt/codex-stacks/cpa/config.yaml`, Caddyfile, and any new stack manifests. +- Record only backup path and redacted evidence. + +## 3. CodexCont Sidecar + +- Create `/opt/codex-stacks/codexcont/app` and copy only runtime files needed by CodexCont. +- Create `config.toml` with: + - `server.host = "0.0.0.0"` + - `server.port = 8787` + - `server.listen_paths = ["/v1/responses"]` + - `upstream.url = "http://cpa:8317/v1/responses"` + - `upstream.mode = "fixed"` + - `auth.mode = "passthrough"` + - continuation enabled with existing defaults +- Create Dockerfile using Python 3.12 and install project dependencies. +- Start `codexcont` on `cpa_net`. +- Validate from container/network that CodexCont can reach CPA. + +## 4. Caddy Cutover + +- Update Caddy for `cpa.konbakuyomu.us`: + - Block management paths. + - Route `/v1/responses` to `codexcont:8787`. + - Route all other traffic to `cpa:8317`. +- Reload Caddy. +- Verify health, models, real responses, and CodexCont logs. +- Roll back Caddy if responses fail. + +## 5. CPA Management API + +- Generate a strong CPA management key on the server. +- Store plaintext key in a root-only credentials file and write only the path in task evidence. +- Update CPA `config.yaml` with `remote-management.secret-key` and `allow-remote: false`. +- Change compose mount for config to writable. +- Restart CPA and verify local `/management.html` and authenticated `/v0/management/config`. +- Verify public API domain management paths remain blocked. + +## 6. Cloudflare Tunnel + +- Create `/opt/codex-stacks/cpa-admin-tunnel/docker-compose.yaml`. +- Run `cpa-admin-proxy` on `127.0.0.1:8327` to preserve CPA `allow-remote: false`. +- Create root-only `.env` placeholder or use the user-provided `TUNNEL_TOKEN`. +- Start `cloudflared` only after token is available. +- Verify `cpa-admin.konbakuyomu.us/management.html` reaches Cloudflare Access / management panel. + +## 7. Eval + +- Create an isolated local test `CODEX_HOME` under a temp directory. +- Configure a test provider for `https://cpa.konbakuyomu.us/v1`. +- Run `python codex_candy_eval.py -m gpt-5.5 -r high -n 5`. +- Keep or delete the temp test home only after reporting the path; do not touch the user's normal Codex config. + +## 8. Closeout + +- Update task evidence with validation results. +- Run local CodexCont tests if repository code changed. +- Commit Trellis task artifacts and any repo changes. +- Record remaining manual Cloudflare Access/token action if blocked. + +## Execution Evidence + +### Preflight And Backups + +- SJC access: `ssh sjc-guard`. +- Backup path: `/root/cpa-codexcont-admin-backups/20260701T044651Z`. +- Disk before CodexCont build: `/dev/sda1` around `8.5G used / 845-846M free` after build, `92%`. +- No Docker prune or bulk filesystem deletion was used. + +### CodexCont Sidecar + +- Stack path: `/opt/codex-stacks/codexcont`. +- Container: `codexcont`. +- Network: `cpa_net`. +- Runtime image build used `python:3.12-slim`. +- Fixed a UTF-8 BOM in server `config.toml`; `tomllib` rejected the BOM at line 1 before the fix. +- Container health evidence: + - `codexcont` can reach `http://cpa:8317/healthz` with HTTP `200`. + - `caddy-edge` resolves `codexcont` on `cpa_net`. + +### Caddy Cutover + +- `cpa.konbakuyomu.us` Caddy block now: + - blocks `/management.html`, `/v0/management*`, `/v0/resource/plugins/*` with `404`. + - sends `/v1/responses` and `/v1/responses/*` to `codexcont:8787`. + - sends other paths to `cpa:8317`. +- `caddy validate --config /etc/caddy/Caddyfile`: valid. +- `caddy reload --config /etc/caddy/Caddyfile`: succeeded. + +### Public API Validation + +- `https://cpa.konbakuyomu.us/healthz`: HTTP `200`. +- `https://cpa.konbakuyomu.us/v1/models` with existing API key: HTTP `200`. +- `https://cpa.konbakuyomu.us/v1/responses` streaming smoke with `gpt-5.5`: HTTP `200`, expected marker text returned. +- Public management blocking: + - `https://cpa.konbakuyomu.us/management.html`: HTTP `404`. + - `https://cpa.konbakuyomu.us/v0/management/config`: HTTP `404`. +- CodexCont log evidence for live response: + - `fold start: model=gpt-5.5 path=/v1/responses url=http://cpa:8317/v1/responses`. + - continuation decisions were logged for eval rounds. +- Egress evidence: + - `sub2api-egress-att` logs show CPA traffic to `chatgpt.com:443` through `sub2api-att-residential`. + +### CPA Management + +- CPA management key generated on server only. +- Plaintext management key path: `/root/cpa-codexcont-admin-backups/20260701T044651Z/cpa-management-key.txt`. +- CPA `config.yaml` bind mount changed from read-only to writable. +- CPA restarted with local image and `--pull never`. +- CPA hashed and persisted `remote-management.secret-key`; plaintext was not left in `config.yaml`. +- `allow-remote` remains `false`. +- Direct host request to `127.0.0.1:8317/v0/management/config` returned `403 remote management disabled` because Docker publish reaches CPA as a bridge peer, not loopback. +- Admin proxy added to preserve `allow-remote: false`: + - stack path: `/opt/codex-stacks/cpa-admin-tunnel`. + - container: `cpa-admin-proxy`. + - host bind: `127.0.0.1:8327`. + - proxy sets `X-Forwarded-For`, `X-Real-IP`, and `CF-Connecting-IP` to `127.0.0.1`. +- Admin proxy validation: + - `http://127.0.0.1:8327/management.html`: HTTP `200`. + - `http://127.0.0.1:8327/v0/management/config` without key: HTTP `401`. + - `http://127.0.0.1:8327/v0/management/config` with management key: HTTP `200`. + +### Cloudflare Tunnel Status + +- `/opt/codex-stacks/cpa-admin-tunnel/docker-compose.yaml` prepared with: + - `cpa-admin-proxy` running now. + - `cpa-admin-tunnel` using `cloudflare/cloudflared:latest`, host networking, and `.env`. +- `/opt/codex-stacks/cpa-admin-tunnel/.env` is root-only and secrets were not recorded in Git, Obsidian, or Trellis artifacts. +- User provided/configured the Cloudflare Tunnel token out of band and reported the Docker Cloudflare connector is connected. +- Cloudflare Dashboard public hostname targets `http://127.0.0.1:8327`, not `8317`, because of the CPA local-client behavior above. +- Final user acceptance: `https://cpa-admin.konbakuyomu.us/management.html` reaches the CPA management panel through Cloudflare Tunnel + Cloudflare Access + CPA management key. + +### Isolated Codex Candy Eval + +- Normal `C:\Users\dxt98\.codex` was not modified. +- Temporary `CODEX_HOME`: `C:\Users\dxt98\AppData\Local\Temp\codex-cpa-eval-home-20260701130319`. +- Test provider: `https://cpa.konbakuyomu.us/v1`, `wire_api = "responses"`, API key supplied via process environment only. +- Command: `python D:\Dev\50_Scripts\52_Python\codex-candy-eval\codex_candy_eval.py -m gpt-5.5 -r high -n 5`. +- Result: `4/5` correct, `80.0%`. +- Important caveat: CodexCont did catch and continue multiple `518n-2` rounds, but one eval run still ended wrong after a first-round `reasoning_tokens=516`. Current sidecar is a mitigation and traffic-path fix, not yet a proof of complete 516-class elimination. diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/prd.md b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/prd.md new file mode 100644 index 0000000..d65d209 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/prd.md @@ -0,0 +1,48 @@ +# CPA CodexCont Sidecar and Admin Tunnel + +## Goal + +Run CodexCont as a server-side continuation middleware in front of CPA for `/v1/responses`, keep CPA on the upstream official image, and expose CPA's management panel only through a Cloudflare Tunnel protected by Cloudflare Access and a CPA management key. + +## Confirmed Facts + +- Production CPA currently runs on SJC at `/opt/codex-stacks/cpa`, container `cpa`, image `eceasy/cli-proxy-api:latest`, bound as `127.0.0.1:8317:8317`. +- `cpa` is on `cpa_net` and `sub2api_canary_net`; the `sub2api-egress-att` route remains required for `socks5://172.19.0.1:1082`. +- CPA management endpoints and `/management.html` currently return `404`, because `remote-management.secret-key` is not enabled. +- SJC root disk is small: about `9.6G` total, `8.5G` used, `1.1G` available during planning. +- The host Python is `3.10`, while CodexCont requires Python `>=3.12`; the production sidecar should therefore run in a Python 3.12 container. +- No existing `cloudflared` binary or tunnel container was found on SJC. +- Local `codex-candy-eval` uses the `codex` CLI. To avoid touching the user's daily local Codex config, evaluation must use an isolated `CODEX_HOME` or command-line config overrides. + +## Requirements + +- R1: Create `/opt/codex-stacks/codexcont` and run CodexCont as a restartable service/container on SJC. +- R2: Route only `cpa.konbakuyomu.us/v1/responses` through CodexCont; keep other CPA API routes direct to CPA. +- R3: Configure CodexCont upstream as `http://cpa:8317/v1/responses`, auth mode `passthrough`, continuation enabled. +- R4: Explicitly block public access on `cpa.konbakuyomu.us` to `/management.html`, `/v0/management*`, and `/v0/resource/plugins/*`. +- R5: Enable CPA management with a strong generated management key while keeping `remote-management.allow-remote: false`. +- R6: Prepare a Cloudflare Tunnel stack for `cpa-admin.konbakuyomu.us` using a Dashboard Token, stored only in a root-only server path. +- R7: The Cloudflare Access policy for `cpa-admin.konbakuyomu.us` must restrict access to the user's Cloudflare identity before the endpoint is considered production-ready. +- R8: Do not print OAuth tokens, API keys, Cloudflare Tunnel tokens, or management keys into terminal output, Git, Obsidian, or task artifacts. +- R9: Do not run Docker prune or recursive/bulk deletion; if disk space becomes insufficient, pause before cleanup. +- R10: Record candy-eval results honestly; do not claim the 516 class is completely eliminated unless live evidence supports it. + +## Acceptance Criteria + +- [x] Trellis artifacts record requirements, design, implementation steps, backups, evidence, and residual risks. +- [x] CodexCont is running on SJC and can reach CPA through `cpa_net`. +- [x] `https://cpa.konbakuyomu.us/healthz` succeeds. +- [x] Authenticated `https://cpa.konbakuyomu.us/v1/models` succeeds through CPA. +- [x] Authenticated `https://cpa.konbakuyomu.us/v1/responses` succeeds and CodexCont logs show it handled the request. +- [x] Public `https://cpa.konbakuyomu.us/management.html` and `/v0/management/config` are blocked. +- [x] `https://cpa-admin.konbakuyomu.us/management.html` is reachable only through Cloudflare Access and then CPA management key authentication. +- [x] A `codex-candy-eval` run using isolated local test config points at `https://cpa.konbakuyomu.us/v1` without changing `C:\Users\dxt98\.codex`. +- [x] Candy-eval result and CodexCont fold logs are recorded, including any residual 516-class failure. + +## Out of Scope + +- Forking or patching CPA itself for 516 continuation. +- Enabling a heavy CPA management stack, Postgres, Redis, or usage keeper. +- Deleting old sub2api data directories or running Docker prune. +- Storing Cloudflare or CPA management secrets in the repository. +- Claiming benchmark-level 516 correctness from route smoke tests alone. diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/task.json b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/task.json new file mode 100644 index 0000000..bb3840c --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-codexcont-sidecar-admin-tunnel/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-codexcont-sidecar-admin-tunnel", + "name": "cpa-codexcont-sidecar-admin-tunnel", + "title": "CPA CodexCont sidecar and admin tunnel", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-01", + "completedAt": "2026-07-01", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "Server-side CodexCont, CPA admin proxy, and Cloudflare Tunnel/Access admin path are deployed. User validated that cpa-admin.konbakuyomu.us/management.html reaches the CPA management panel through the intended chain. Isolated candy eval ran 5 tests via CPA and scored 4/5; sidecar caught 518n-2 rounds but does not yet prove complete 516-class elimination.", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/check.jsonl b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/check.jsonl new file mode 100644 index 0000000..f1f8092 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/check.jsonl @@ -0,0 +1,2 @@ +{"file": ".trellis/spec/backend/codex-continuation-contracts.md", "reason": "Check hash separation, no-secret projections, route blocking, and small-disk rollout constraints"} +{"file": ".trellis/spec/backend/quality-guidelines.md", "reason": "General backend quality gate placeholder for local tests and deployment verification"} diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/design.md b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/design.md new file mode 100644 index 0000000..3af528c --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/design.md @@ -0,0 +1,171 @@ +# CPA Key Management And Usage Portal Design + +## Architecture + +```text +Codex / clients + -> https://cpa.konbakuyomu.us/v1/responses + -> public Caddy + -> CodexCont sidecar + -> CPA official container + -> CPA Key Policy plugin + -> OpenAI OAuth account + +CPA official container + -> usage event queue + -> CPAMP collector + -> /data/usage.sqlite + +User browser + -> https://cpa-usage.konbakuyomu.us/ + -> cpa-usage-portal sidecar + -> Key Policy state file (read-only) + -> CPAMP monitoring API (filtered by api_key_hash) +``` + +CPA remains the source of API behavior. CodexCont remains the source of 516 +continuation protection. CPAMP and the user portal observe and project safe +usage information; they do not own request forwarding. + +## Server Components + +### CPAMP + +- Stack path: `/opt/codex-stacks/cpamp`. +- Network: `cpa_net`. +- Persistent data: `/opt/codex-stacks/cpamp/data/usage.sqlite` and + `/opt/codex-stacks/cpamp/data/data.key`. +- Collector mode: `auto`. +- CPA management endpoint: `http://cpa-admin-proxy:8327`, so CPA can keep + `remote-management.allow-remote: false`. +- Admin route: `https://cpa-admin.konbakuyomu.us/cpamp/` via the existing admin + proxy and Cloudflare Access. + +### CPA Key Policy + +- Install the official Linux amd64 release for `cpa-key-policy v0.2.1`. +- Mount a persistent CPA plugin directory and a persistent plugin state + directory into the CPA container. +- CPA config shape: + +```yaml +usage-statistics-enabled: true +plugins: + enabled: true + dir: "/CLIProxyAPI/plugins" + configs: + cpa-key-policy: + enabled: true + priority: 10 + state_file: "/CLIProxyAPI/plugin-state/cpa-key-policy-state.json" + keys: [] +``` + +When the state file exists, it is the source of truth. New user keys are created +through Key Policy management APIs or UI. Existing native CPA keys remain for +compatibility/admin access and should be migrated later. + +### User Usage Portal + +- Stack path: `/opt/codex-stacks/cpa-usage-portal`. +- Domain: `cpa-usage.konbakuyomu.us`. +- Network: `cpa_net`. +- Runtime: Python 3.12 with Starlette/httpx, matching CodexCont's lightweight + stack. +- Static frontend: one Chinese HTML/CSS/JS page; no npm build. +- Portal reads Key Policy state read-only and talks to CPAMP through an internal + admin URL with a CPAMP admin key from a root-only secret file or environment. + +## Portal Data Contracts + +### Hash Normalization + +- Raw key: accepted only in `POST /api/session`. +- Raw-key hex: `sha256(trimmed_raw_key).hexdigest()`. +- Key Policy hash: `sha256:<raw-key-hex>`; this validates login. +- CPAMP usage hash for Key Policy keys: `sha256(Key Policy id)`. +- Fallback CPAMP hash for non-id records: raw-key hex. + +The portal stores only safe hash identifiers and key metadata in a signed +HttpOnly cookie. It must not store the raw `cpa_...` key. + +### Safe Key Metadata + +The portal may return: + +- key id/name/preview if present. +- enabled/disabled status. +- model allowlist or aliases. +- RPM. +- daily/weekly USD limits. +- usage counters exposed by Key Policy state. + +It must not return Key Policy internal `key_hash`, raw key material, or plugin +management credentials. + +### Safe Usage Event + +The portal projects CPAMP events into safe fields: + +- timestamp, model, provider/account labels if already non-secret. +- status, HTTP status, latency. +- input/output/reasoning/cache tokens when available. +- cost estimate when available. +- redacted error summary. + +It must not return prompt, response text, headers, cookies, raw request payload, +raw response payload, OAuth token data, encrypted reasoning content, or any +unrecognized secret-like field. + +## Portal API + +- `POST /api/session`: body `{ "api_key": "..." }`; validates against Key + Policy state and returns safe key metadata. +- `DELETE /api/session`: clears the session cookie. +- `GET /api/me`: returns safe key metadata for the current session. +- `GET /api/usage?range=24h|7d`: returns aggregated usage for the current key. +- `GET /api/events?limit=100&before=...`: returns recent safe request events. +- `GET /api/events/stream`: SSE stream for new safe request events. + +All read APIs require a valid session. Every CPAMP query includes the current +session's CPAMP-format `api_key_hash`. + +## Budget Suggestion + +The portal package includes an admin utility module/script for budget +suggestions: + +- Inputs: total daily/weekly budget, enabled Key Policy keys, model price table. +- Disabled keys are excluded. +- Missing model price data blocks USD limit suggestions. +- Output: per-key suggested daily/weekly limits and a safe patch payload that an + admin can apply through Key Policy management APIs. + +Applying limits remains an administrator action in v1. + +## Retention + +The retention job deletes CPAMP rows older than 7 days in small batches. It must +only target known CPAMP usage tables after introspection and run +`PRAGMA wal_checkpoint(TRUNCATE)` after deletion. It must not run `VACUUM`. + +## Security Boundaries + +- Public API host must continue to block `/management.html`, + `/v0/management*`, `/v0/resource/plugins/*`, `/admin/*`, `/codexcont/*`, + `/cpamp/*`, and portal-internal admin routes. +- CPAMP remains behind Cloudflare Access and CPA management key. +- User portal does not use Cloudflare Access in v1; the API key itself is the + self-service credential. This makes strict redaction and per-key filtering the + primary boundary. + +## Rollback + +- If CPAMP fails before CPA config changes, stop only CPAMP and leave production + traffic unchanged. +- If CPA plugin startup fails, restore the backed-up CPA config/compose and + restart only `cpa`. +- If the user portal fails, remove only the portal route/container; CPA, + CodexCont, and CPAMP can continue running. +- If any public admin exposure is detected, revert Caddy/admin proxy routing + before continuing functional tests. diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/implement.jsonl b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/implement.jsonl new file mode 100644 index 0000000..77f6345 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/implement.jsonl @@ -0,0 +1,2 @@ +{"file": ".trellis/spec/backend/codex-continuation-contracts.md", "reason": "CPA Key Policy, CPAMP, usage portal, public route, and 516 sidecar integration contracts"} +{"file": ".trellis/spec/guides/cross-layer-thinking-guide.md", "reason": "Key Policy state, CPAMP analytics, portal API, and frontend projections cross several data boundaries"} diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/implement.md b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/implement.md new file mode 100644 index 0000000..add0703 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/implement.md @@ -0,0 +1,141 @@ +# CPA Key Management And Usage Portal Implementation Plan + +## Phase 1: Local Implementation + +1. Add a `cpa_usage_portal` Python package with: + - Key Policy state loader and hash normalization. + - Signed session cookie helpers. + - CPAMP client abstraction. + - Redaction and safe event projection. + - Budget suggestion helper. + - Retention helper for CPAMP SQLite. + - Starlette app and static Chinese dashboard. +2. Add focused tests for: + - `sha256:` vs bare-hex normalization. + - API key validation and unknown key rejection. + - HttpOnly session cookie behavior. + - CPAMP requests always include the current key hash. + - Secret redaction and safe event projection. + - Budget suggestion edge cases. + - Retention deleting old rows in batches without `VACUUM`. +3. Update README files with short operational notes. + +## Phase 2: Server Preflight + +1. Record `df -h /`, `docker system df`, and current container list. +2. Back up root-only copies of: + - `/opt/codex-stacks/cpa/docker-compose.yaml` + - `/opt/codex-stacks/cpa/config.yaml` + - `/opt/codex-stacks/cpa-admin-tunnel/Caddyfile` + - `/opt/codex-stacks/caddy/Caddyfile` + - Key Policy state and CPAMP data paths if already present. +3. Do not use Docker prune or broad deletion. If free space is insufficient for + image pulls, pause for an explicit per-image cleanup decision. + +## Phase 3: CPAMP And Key Policy + +1. Create `/opt/codex-stacks/cpamp` and start CPAMP on `cpa_net`. +2. Enable CPA usage statistics and persistent plugin mounts. +3. Install `cpa-key-policy v0.2.1` from the official release and verify checksum. +4. Restart only `cpa`; verify `/healthz`, `/v1/models`, and CodexCont + `/v1/responses`. +5. Route `cpa-admin.konbakuyomu.us/cpamp/` through the admin proxy. + +## Phase 4: User Portal + +1. Create `/opt/codex-stacks/cpa-usage-portal`. +2. Deploy the portal container on `cpa_net`. +3. Add `cpa-usage.konbakuyomu.us` routing. +4. Verify login with a test Key Policy key and confirm only that key's events + are visible. + +## Phase 5: Acceptance And Closeout + +1. Verify Key Policy: + - Allowed model succeeds. + - Disallowed model fails. + - Low-limit/RPM test key is constrained. +2. Verify CPAMP receives real requests. +3. Verify CodexCont dashboard and 516 protection still show current requests. +4. Verify public API domain still blocks management/plugin/admin paths. +5. Record disk/database/log sizes. +6. Update this task with deployment evidence, then commit. + +## Current Evidence + +- Existing production CPA public base: `https://cpa.konbakuyomu.us/`. +- Existing admin base: `https://cpa-admin.konbakuyomu.us/`. +- Existing CodexCont admin path: `/codexcont/`. +- Existing SJC disk is known to be tight; live preflight is required before any + server-side image pull. + +## Deployment Evidence 2026-07-01 + +- Root-only backup path: `/root/cpa-key-management-backups/20260701-193204`. +- Server disk after rollout: `/dev/sda1` size `9.6G`, used `8.8G`, available + `771M`, use `93%`. +- `docker system df` before the final portal rebuild showed images `2.227GB`, + local volumes `53.64MB`, build cache `234.8MB`; no Docker prune was used. +- CPAMP is deployed at `/opt/codex-stacks/cpamp`, container `cpamp`, and + `http://127.0.0.1:8327/health` returned `200`. +- CPA config has `usage-statistics-enabled: true`, `plugins.enabled: true`, and + `cpa-key-policy.enabled: true` with state file + `/CLIProxyAPI/plugin-state/cpa-key-policy-state.json`. +- Key Policy state path on the host: + `/opt/codex-stacks/cpa/plugin-state/cpa-key-policy-state.json`; current safe + status is `key_count=1`. +- A Key Policy smoke key successfully called + `https://cpa.konbakuyomu.us/v1/responses`: HTTP `200`, status `completed`. +- A disallowed model using the smoke key was rejected with HTTP `401`. +- Public `https://cpa.konbakuyomu.us` returned `404` for `/management.html`, + `/v0/management/config`, `/v0/resource/plugins/cpa-key-policy`, + `/admin/requests`, `/codexcont/`, and `/cpamp/`. +- CodexCont admin proxy route remains healthy: + `http://127.0.0.1:8327/codexcont/healthz` and `/codexcont/requests?limit=1` + returned `200`. +- CPAMP SQLite sizes after rollout: `usage.sqlite` `236K`, WAL `3.3M`, SHM + `32K`. +- User portal is deployed at `/opt/codex-stacks/cpa-usage-portal`, container + `cpa-usage-portal`, image `cpa-usage-portal:latest`. +- Portal route was verified through local host resolution for + `https://cpa-usage.konbakuyomu.us`: `/healthz` `200`, login `200`, + `/api/events` returned `2` own events, `/api/usage` returned `2` calls and + `0` failures. +- `/api/usage` no longer exposes either the raw-key hash or the Key Policy id + hash; it returns only safe stat fields plus a preview. +- Live CPAMP comparison confirmed the important hash contract: + `sha256("sjc-smoke-20260701")` returned `calls=1 events=1`, while + `sha256(raw smoke key)` returned `calls=0 events=0`. The portal now validates + login with raw-key hash but filters CPAMP with the Key Policy id hash. +- Local validation after the hash correction: + `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> `32/32`; + `.venv\Scripts\python.exe tests\test_middleware.py` -> `143/143`; + `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py` + completed. + +## Remaining Operational Note + +- `cpa-usage.konbakuyomu.us` DNS was later added by the user. Server-side + public health verification returned `200`. +- RPM/daily-limit enforcement was not stress-tested with many live requests to + avoid unnecessary spend. Model allowlist rejection and normal Key Policy + authentication were verified. + +## Closeout Decisions + +- Ordinary users should use Key Policy `cpa_...` keys for both Codex requests + and the user usage portal. Native CPA `sk...` keys are compatibility/admin + escape hatches, not the self-service user identity. +- The user portal intentionally rejects native `sk...` keys with + `invalid_api_key` unless they are explicitly migrated into Key Policy. +- `https://cpa-admin.konbakuyomu.us/management.html` points to CPAMP, so its + login key is the CPAMP admin key. CPA management key is a separate secret for + CPA-native management calls. +- There is no one-to-one binding between native `sk...` keys and Key Policy + `cpa_...` keys in the current design. Adding such a bridge would need a + separate mapping layer and bypass-risk review. +- CPA, CPAMP, and CPA Key Policy were not source-modified. They are deployed as + official image/plugin artifacts plus config, volumes, and Caddy routing. +- `cpa-usage-portal` and CodexCont are the custom-maintained sidecars. All + production components are separate containers/stacks so CPA/CPAMP/Key Policy + can be updated independently from custom code. diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/prd.md b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/prd.md new file mode 100644 index 0000000..3b66c7a --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/prd.md @@ -0,0 +1,85 @@ +# CPA Key Management And Usage Portal + +## Goal + +Give the CPA production stack real per-key governance without forking CPA: +administrators can monitor usage and set limits, while ordinary users can log +in with their own API key and see only their own usage and recent request +details. + +The first version combines three layers: + +- CPA Manager Plus (CPAMP) for administrator monitoring and request-level + usage collection. +- CPA Key Policy for per-key model allowlists, RPM, and daily/weekly USD + limits. +- A small independent usage portal for user self-service views. + +## Requirements + +- Keep CPA on the official image. Do not fork CPA or patch its runtime code. +- Keep CodexCont in the `/v1/responses` production path. The 516 continuation + protection must not regress. +- Deploy CPAMP as an admin-only service behind `cpa-admin.konbakuyomu.us` and + Cloudflare Access. CPAMP must consume CPA usage events and persist them in a + bounded SQLite database. +- Enable CPA usage statistics and CPA plugins. Install the official + `cpa-key-policy` plugin release and configure a persistent state file. +- New shared keys should be CPA Key Policy `cpa_...` keys. Existing native CPA + `api-keys` remain as compatibility/admin escape hatch but are not the target + mechanism for quota-managed users. +- Provide a Chinese user portal at `cpa-usage.konbakuyomu.us`. +- The user portal authenticates by accepting an API key once, validating it + against Key Policy state, and storing only a signed HttpOnly session cookie + containing safe hash identifiers and key metadata. +- The user portal must never store, log, render, or return raw API keys, OAuth + tokens, CPA management keys, CPAMP admin keys, cookies, Authorization headers, + request bodies, response bodies, or encrypted reasoning content. +- Users can only see their own key metadata, aggregate usage, and recent + request summaries. Server-side filtering by `api_key_hash` is mandatory. +- Request details may include safe operational fields: time, model, status, + latency, token usage, cost estimate, cache/reasoning counters if available, + and redacted failure summaries. +- Budget allocation is semi-automatic in v1: the administrator supplies a + trusted total budget, enabled keys are averaged, disabled keys are excluded, + and missing model prices block automatic USD limit writes. +- CPAMP request history retention is 7 days. Retention must be batched and avoid + `VACUUM`; WAL checkpoint is acceptable. +- SJC disk is small. Before server deployment, record `df -h /` and + `docker system df`; do not use Docker prune or broad directory deletion. + +## Acceptance Criteria + +- [ ] CPAMP `/health` or equivalent status endpoint is reachable through the + admin path and real CPA requests appear in monitoring data. +- [ ] CPA has `usage-statistics-enabled: true` and `plugins.enabled: true` with + Key Policy loaded from a persistent plugin directory/state file. +- [ ] A Key Policy test key can call `/v1/responses`; a disallowed model is + rejected; a low-limit/RPM test key is constrained. +- [ ] The CodexCont dashboard and `/v1/responses` folding path still work after + CPA plugin/usage changes. +- [ ] `cpa-usage.konbakuyomu.us` lets a user log in with a Key Policy key and + shows only that key's status, limits, aggregate usage, and recent events. +- [ ] User portal tests cover hash normalization, session safety, per-key + filtering, redaction, budget suggestions, and retention SQL behavior. +- [ ] Public `cpa.konbakuyomu.us` still blocks CPA management/plugin/admin + paths and does not expose CPAMP or the user portal internals. +- [ ] Deployment notes record final disk space, CPAMP SQLite/WAL size, and + backup paths without leaking secrets. + +## Non-Goals + +- No CPA fork. +- No public CPAMP access for ordinary users. +- No attempt to infer OpenAI account balance as exact dollars in v1. +- No long-term billing ledger beyond 7-day request history. +- No request/response body inspection in the user portal. + +## Notes + +- Key Policy stores the raw `cpa_...` key hash as `sha256:<hex>` for login + validation. For plugin keys, CPAMP records `api_key_hash` as + `sha256(Key Policy id)` because CPA receives the plugin key id as the + authenticated principal. The portal owns both normalizations. +- Cloudflare Access protects admin surfaces. The user portal is protected by + API-key self-check plus strict server-side filtering. diff --git a/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/task.json b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/task.json new file mode 100644 index 0000000..6111b00 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-cpa-key-management-portal/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-key-management-portal", + "name": "cpa-key-management-portal", + "title": "CPA key management and usage portal", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-01", + "completedAt": "2026-07-01", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "Completed: CPA/CPAMP/Key Policy remain official artifacts; custom code is split into CodexCont and cpa-usage-portal sidecars. Ordinary users use Key Policy cpa_... keys for Codex and self-service usage; native sk... keys are admin/compatibility only.", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/codexcont-desktop.png b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/codexcont-desktop.png new file mode 100644 index 0000000..70d8975 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/codexcont-desktop.png differ diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/codexcont-mobile.png b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/codexcont-mobile.png new file mode 100644 index 0000000..aacc784 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/codexcont-mobile.png differ diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/usage-desktop.png b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/usage-desktop.png new file mode 100644 index 0000000..9666c51 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/usage-desktop.png differ diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/usage-mobile.png b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/usage-mobile.png new file mode 100644 index 0000000..8387748 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/artifacts/usage-mobile.png differ diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/check.jsonl b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/check.jsonl new file mode 100644 index 0000000..26c4ca7 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/check.jsonl @@ -0,0 +1,2 @@ +{"file": ".trellis/spec/backend/codex-continuation-contracts.md", "reason": "Verify dashboard data contracts, route exposure, and sidecar boundaries"} +{"file": ".trellis/spec/backend/quality-guidelines.md", "reason": "General backend and frontend smoke quality gate"} diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/design.md b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/design.md new file mode 100644 index 0000000..20751fb --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/design.md @@ -0,0 +1,134 @@ +# Dashboard Visual Refresh Design + +## Visual System + +Both dashboards use a shared CPAMP-inspired dark operations aesthetic: + +- Dark page background and slightly lighter panels. +- 8px or smaller panel radius, compact spacing, no marketing hero layout. +- Blue primary buttons, green success chips, red failure chips, amber warning + chips, neutral muted chips. +- Stable card and table dimensions with explicit column widths to prevent + vertical text compression. +- Top toolbar instead of left navigation. + +The pages do not need pixel-perfect parity with CPAMP. They should feel like +they belong in the same operational family. + +## CPA Usage Portal + +The portal remains a static page served by `cpa_usage_portal`. + +Data flow is unchanged: + +```text +browser -> cpa-usage-portal -> Key Policy state + CPAMP analytics +``` + +The main event table changes from raw detail display to a scan-first view: + +- visible row fields: time, status, model, latency, tokens, reasoning, cost, + details action. +- hidden detail row: endpoint, request id, status code, service tier, + reasoning effort, quota hints, and redacted failure details. +- `failure_brief` is a short human-readable projection intended for detail + headers and table hints. + +Backend projection remains the safety boundary. The frontend formats safe +fields but must not receive secret material. + +## CodexCont Dashboard + +The CodexCont dashboard keeps the current admin APIs: + +- `GET status` +- `GET requests?limit=100` +- `GET logs?limit=200` +- `GET logs/stream` + +Only the static HTML/CSS/JS changes. The first screen becomes: + +- top toolbar: title, stream status, refresh action. +- metrics row: total requests, protected requests, auto continuations, + truncation hits, failures. +- recent requests table with protection chips and expandable protection detail. +- advanced logs as a lower-priority collapsible panel. + +## Safety And Compatibility + +- Existing API fields remain compatible. +- `failure` stays available for compatibility, but bounded and redacted. +- `failure_brief` is additive. +- No new public route is introduced. +- The admin/user split is unchanged: + `cpa-usage.konbakuyomu.us` for users, `cpa-admin.../codexcont/` for admin. + +## Rollback + +- If the usage portal page fails, redeploy the previous + `/opt/codex-stacks/cpa-usage-portal` backup and restart only that container. +- If the CodexCont dashboard fails, redeploy the previous + `/opt/codex-stacks/codexcont` backup and restart only `codexcont`. +- If public admin route checks fail, revert the touched Caddy route only after + confirming this task changed Caddy; expected implementation does not change + Caddy. + +## Follow-up Design: Local Quota Admin + +### Data Flow + +```text +browser -> cpa-usage-portal -> Key Policy state + -> CPAMP analytics + -> local portal SQLite +``` + +Key Policy remains the identity and price source for user keys. CPAMP remains +the immutable request-event source. The portal overlays local 5H/month limits +and reset watermarks, then queries CPAMP from the effective window start. + +### Local SQLite + +The portal owns a small SQLite database at `/data/portal/usage_portal.sqlite` +in production. It stores only: + +- `key_limits`: `policy_id`, `five_hour_limit_usd`, `monthly_limit_usd`. +- `reset_watermarks`: `policy_id`, `window`, `reset_at_ms`. +- `audit_log`: operator actions and before/after JSON for local metadata. + +It does not store raw API keys, request bodies, response bodies, OAuth tokens, +management keys, or encrypted reasoning content. + +### Windows And Reset + +Supported ranges are `5h`, `24h`, `7d`, and `month`. + +- `5h`: rolling five hours. +- `24h`: rolling twenty-four hours, matched to Key Policy daily limit display. +- `7d`: rolling seven days, matched to Key Policy weekly limit display. +- `month`: current Asia/Shanghai calendar month. + +A soft reset writes a watermark. For a selected range, the effective CPAMP +window starts at `max(base_window_start, reset_at_ms)`. CPAMP original event +rows are not deleted or rewritten. + +### Admin Boundary + +The admin UI is served by the same portal container but only under the admin +proxy. The app requires a proxy-injected header (`X-Usage-Admin: 1`) for all +`/admin/*` routes. The public user route does not set this header, so admin +routes return `404` there even if DNS points at the same container. + +The planned production mount is: + +```text +cpa-admin.konbakuyomu.us/usage-admin/* -> admin proxy injects header + -> cpa-usage-portal /admin/* +``` + +### Price Correction + +CPAMP Model Prices and Key Policy per-key model entries both use USD per 1M +tokens. The portal still recomputes self-service costs from Key Policy prices +because CPAMP can legitimately return zero for custom aliases until its global +price book is configured. diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/implement.jsonl b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/implement.jsonl new file mode 100644 index 0000000..a7d4f10 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/implement.jsonl @@ -0,0 +1,3 @@ +{"file": ".trellis/spec/backend/codex-continuation-contracts.md", "reason": "Dashboard, CPA usage portal, CPAMP, and admin/public route contracts"} +{"file": ".trellis/spec/guides/cross-layer-thinking-guide.md", "reason": "Safe projections travel from CPAMP/diagnostics APIs into browser tables and detail rows"} +{"file": ".trellis/spec/guides/code-reuse-thinking-guide.md", "reason": "Failure-summary projection should be centralized in the backend redaction layer"} diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/implement.md b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/implement.md new file mode 100644 index 0000000..599fdbc --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/implement.md @@ -0,0 +1,455 @@ +# Dashboard Visual Refresh Implementation Plan + +## Phase 1: Planning And Specs + +1. Record the approved product plan in `prd.md`, `design.md`, and this file. +2. Read backend specs and shared thinking guides before code edits. +3. Start the Trellis task. + +## Phase 2: Local Implementation + +1. Update `cpa_usage_portal.redaction`: + - add bounded `failure_brief`; + - keep `failure` redacted and bounded; + - strip noisy response-header blobs from table-facing output. +2. Rebuild `cpa_usage_portal/static/dashboard.html`: + - CPAMP-like dark top toolbar and metric cards; + - compact model table and recent request table; + - expandable event detail rows; + - no long summary column. +3. Rebuild `middleware/dashboard.html`: + - same dark visual language; + - recent request protection table remains first-class; + - advanced logs stay collapsible and lower priority. +4. Update tests for failure summary projection and existing admin smoke. + +## Phase 3: Local Validation + +1. Run `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py`. +2. Run `.venv\Scripts\python.exe tests\test_middleware.py`. +3. Run `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py middleware run.py`. +4. Use Playwright screenshots for desktop and 390px mobile when a local server + can be started without disturbing production. + +## Phase 4: Server Rollout + +1. Confirm SJC disk and running containers. +2. Create root-only backups of `/opt/codex-stacks/cpa-usage-portal` and + `/opt/codex-stacks/codexcont`. +3. Upload changed custom files only. +4. Rebuild/restart only `cpa-usage-portal` and `codexcont`. +5. Verify: + - `https://cpa-usage.konbakuyomu.us/` loads; + - `https://cpa-admin.konbakuyomu.us/codexcont/` loads; + - recent real requests still appear; + - public `https://cpa.konbakuyomu.us` returns `404` for admin/dashboard + paths. + +## Phase 5: Closeout + +1. Record screenshots, server evidence, and any known follow-ups. +2. Commit implementation and Trellis task artifacts. + +## Implementation Evidence + +Local changes: + +- `cpa_usage_portal.redaction.safe_event` now adds bounded `failure_brief`, + caps `failure` at 600 characters, and strips noisy response-header blobs from + success events. +- `cpa_usage_portal/static/dashboard.html` was rebuilt as a dark, no-sidebar + operations page. The main request table now shows time, status, model, + latency, tokens, reasoning, cost, and an expand action only. +- `middleware/dashboard.html` was rebuilt with the same visual language. + Recent request protection remains the first-screen focus and advanced logs + are collapsed below the request table. +- CPA, CPAMP, and CPA Key Policy official source/artifacts were not modified. + +Local validation on 2026-07-01: + +- `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> 36/36 checks + passed. +- `.venv\Scripts\python.exe tests\test_middleware.py` -> 143/143 checks + passed. +- `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py middleware run.py` + -> passed. +- Playwright screenshot review covered desktop and 390px mobile mock data for + both pages. Screenshots are in `artifacts/usage-desktop.png`, + `artifacts/usage-mobile.png`, `artifacts/codexcont-desktop.png`, and + `artifacts/codexcont-mobile.png`. The screenshots show no short-field + vertical compression or incoherent overlap; mobile tables use horizontal + scroll inside the table area instead of compressing columns. + +Server rollout on SJC: + +- Preflight: `/` was 9.6G total, 8.8G used, 759M available, 93% used. +- Root-only backup path: + `/root/codex-backups/dashboard-visual-refresh-20260701-212946/`. + Backup tarballs were created with mode `600`. +- Uploaded only changed custom files: + `cpa_usage_portal/redaction.py`, + `cpa_usage_portal/static/dashboard.html`, and + `middleware/dashboard.html`. +- Rebuilt/restarted only `cpa-usage-portal` and `codexcont`. + `cpa`, `cpamp`, `caddy-edge`, `cpa-admin-proxy`, and + `cpa-admin-tunnel` kept their prior uptime. + +Server validation: + +- `cpa-usage-portal` health via internal route returned + `{"ok":true,"key_policy_state":true,"cpamp":true}`. +- CodexCont `/admin/healthz` via internal route returned `200`. +- Admin proxy internal route `http://127.0.0.1:8327/codexcont/` returned + `200`. +- Admin proxy SSE route `/codexcont/logs/stream?once=1` returned `200` with + `ready` and `request` events. +- Public `https://cpa-usage.konbakuyomu.us/` returned `200` and the new page + contains `CPA 用量自助页`, `最近请求`, and `failure_brief`. +- Public `https://cpa-admin.konbakuyomu.us/codexcont/` returned a Cloudflare + Access login redirect, preserving the admin boundary. +- Public `https://cpa.konbakuyomu.us/admin/`, + `https://cpa.konbakuyomu.us/codexcont/`, and + `https://cpa.konbakuyomu.us/admin/requests` returned `404`. +- CodexCont request summaries returned real recent traffic including + `protected_clean` and `auto_continued` entries, proving the dashboard data + path still reflects live Codex traffic. +- Post-rollout disk remained tight but stable: 9.6G total, 8.8G used, 759M + available, 93% used. No Docker prune or broad filesystem cleanup was used. + +Known follow-up: + +- The mobile screenshots intentionally show horizontally scrollable tables. + This is preferred over compressing short columns into vertical text. + +## Follow-up: Refresh Resiliency And Live-State Feedback + +Problem reported on 2026-07-01: + +- After leaving `cpa-usage.konbakuyomu.us` or the CodexCont dashboard idle and + returning later, clicking the toolbar refresh button sometimes did not show + the newest requests. A full browser reload did recover the page. +- The refresh action had weak/no visible progress feedback. +- The realtime connection chip looked static, so it was hard to tell whether + the page was alive, reconnecting, or stale. + +Root cause: + +- The pages treated an existing `EventSource` object as usable browser state + even after a long idle/background period. Browser tab throttling, network + suspension, or Cloudflare/proxy idle behavior can leave the frontend with a + stale object or a slow in-flight fetch. +- Manual refresh reloaded some JSON snapshots, but did not force a fresh SSE + connection. It also did not guard against a late older fetch overwriting a + newer refresh result. + +Implemented fix: + +- `cpa_usage_portal/static/dashboard.html`: + - GET API calls now use `cache: "no-store"` plus a cache-busting query value. + - Dashboard refreshes use a sequence counter, so late stale responses cannot + overwrite newer data. + - Manual refresh forces a fresh `/api/events/stream` `EventSource`. + - `visibilitychange`, `pageshow`, and stale `focus` revive the dashboard by + reconnecting SSE and reloading current snapshots. + - The refresh button shows a spinner/busy label and completion pulse. + - The realtime chip pulses in connected, reconnecting, and disconnected + states. +- `middleware/dashboard.html`: + - `status`, `requests`, and `logs` snapshot fetches use no-store/cache-bust + and independent sequence guards. + - Manual refresh reloads all snapshots and force-reconnects `logs/stream`. + - Foreground resume handlers rebuild SSE and reload the dashboard. + - Refresh and realtime state animations match the usage portal. +- Regression smoke checks were added to both dashboard route tests to ensure + forced stream reconnect, foreground resume, and animation hooks stay present. + +Validation: + +- `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> 40/40 checks + passed. +- `.venv\Scripts\python.exe tests\test_middleware.py` -> 146/146 checks + passed. +- `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py middleware run.py` + -> passed. +- Node parsed the inline scripts from both HTML files successfully: + `cpa_usage_portal/static/dashboard.html: js parse ok` and + `middleware/dashboard.html: js parse ok`. + +Server rollout on SJC: + +- Preflight after the previous dashboard refresh rollout: `/` was 9.6G total, + 8.8G used, about 749M available, 93% used. +- Safe Key Policy projection confirmed real prices are stored under + `models[]` entries. `QQ专用` has 5 priced models plus daily/weekly limits + `100 / 500`; `kuma专用` has 5 priced models but no daily/weekly limits. +- Root-only backup path: + `/root/codex-backups/dashboard-pricing-refresh-20260701-225349/`. + Backed up changed server files before upload; `pricing.py` was new, so no old + file existed to back up. +- Uploaded only changed custom files: + `cpa_usage_portal/app.py`, `cpa_usage_portal/cpamp.py`, + `cpa_usage_portal/key_policy.py`, `cpa_usage_portal/pricing.py`, + `cpa_usage_portal/static/dashboard.html`, and `middleware/dashboard.html`. +- Rebuilt/restarted only `cpa-usage-portal` and `codexcont` for the main + rollout. A final one-line safety patch to all-zero price handling restarted + only `cpa-usage-portal`. +- `cpa`, `cpamp`, and `caddy-edge` kept their prior uptime; CPA, CPAMP, and + CPA Key Policy official artifacts were not modified. + +Server validation: + +- `cpa-usage-portal` health inside the container returned + `{"ok":true,"key_policy_state":true,"cpamp":true}`. +- The public usage page returned `200` and contains `CPA 用量自助页`, + `用量限额`, and `scheduleFollowUpRefreshes`. +- A temporary server-side session for `QQ专用` showed `priced_model_count=5`, + limits `daily_usd=100.0`, `weekly_usd=500.0`, and nonzero + `summary.total_cost=0.907675` with `cost_source=key_policy`. +- A temporary server-side session for `kuma专用` showed `priced_model_count=5`, + null daily/weekly limits, and nonzero Key Policy-derived usage cost. This + confirms its prior missing limits were configuration state, not a page bug. +- CodexCont internal admin page includes `scheduleRequestFollowUp`, and public + `https://cpa.konbakuyomu.us/codexcont/` remained `404`. +- Final disk state remained tight but stable: 9.6G total, 8.8G used, about + 745M available, 93% used. No Docker prune or broad deletion was used. + +## Follow-up: Key Policy Pricing, Limits, And Faster Status Convergence + +Problem reported on 2026-07-01: + +- The Key Policy admin page had per-model prices configured for user keys, but + the self-service usage portal still showed `$0.0000`. +- The self-service page did not make a user's daily and weekly USD limits + visible enough. +- Both custom dashboards could still feel stale: a request row could remain + `processing` or the realtime state could lag until a full browser refresh. + +Root cause: + +- The real Key Policy state stores model aliases and prices as structured + entries under `models[]`, with fields such as + `input_price_per_million`, `output_price_per_million`, and + `cache_read_price_per_million`. The portal only understood top-level + `model_prices`, so it neither displayed clean model names nor saw the + configured prices. +- CPAMP's own model price book did not have these custom Codex aliases, so + CPAMP analytics legitimately returned zero cost. The user portal needed to + overlay per-key prices from Key Policy instead of trusting CPAMP cost fields. +- Realtime events are not enough as durable UI state after tab idle or while a + request is still in progress; the pages needed more follow-up snapshot pulls. + +Implemented fix: + +- `cpa_usage_portal/key_policy.py`: + - parses structured `models[]` entries into clean aliases; + - parses per-model Key Policy prices from the real field names; + - returns safe pricing metadata and daily/weekly limits in `/api/me`. +- `cpa_usage_portal/pricing.py`: + - computes local per-key costs from Key Policy prices; + - separates input, output, cached input, cache read, and cache creation token + buckets; + - mirrors CPAMP's `priority` / `fast` service-tier multiplier for Codex + model families. +- `cpa_usage_portal/app.py`: + - requests CPAMP `model_stats` for `/api/usage`; + - overlays `summary.total_cost`, `model_share[].cost`, and event `cost` + with Key Policy pricing; + - marks computed rows with `cost_source: key_policy`. +- `cpa_usage_portal/static/dashboard.html`: + - shows daily and weekly USD limits directly in the metric row; + - shows pricing source / missing-price hints in the cost card; + - optimistically merges realtime event rows, then schedules delayed snapshot + refreshes so aggregate data can catch up. +- `middleware/dashboard.html`: + - refreshes status counters after request SSE updates; + - follows `processing` request rows with short delayed `/admin/requests` + reloads; + - reduces regular request snapshot polling from 15 seconds to 5 seconds. + +Validation: + +- `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> 48/48 checks + passed. +- `.venv\Scripts\python.exe tests\test_middleware.py` -> 147/147 checks + passed. +- `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py middleware run.py` + -> passed. +- Node parsed the inline scripts from both HTML files successfully: + `cpa_usage_portal/static/dashboard.html: js parse ok` and + `middleware/dashboard.html: js parse ok`. + +## Follow-up: Usage Range Visibility And Event Window + +Problem reported on 2026-07-01: + +- The user usage page's `24 小时` / `7 天` selector appeared to do nothing. +- The page did not clearly show which time window was active, and the recent + request table was still fetched from a fixed 7-day window. + +Evidence: + +- A server-side CPAMP check for `kuma专用` showed the same totals for both + windows because all retained traffic was inside the last 24 hours: + `24h calls=242`, `7d calls=242`, same tokens, cost, and model distribution. +- After the fix and redeploy, the same key still legitimately returned + identical totals (`260` calls in both windows), but the APIs now returned + explicit `usage_range` / `events_range` values for `24h` and `7d`. + +Implemented fix: + +- `GET /api/events` now accepts `range=24h|7d` and returns `range`, + `from_ms`, and `to_ms`; the compatibility default remains 7 days. +- The frontend now sends the selected range to both `/api/usage` and + `/api/events`, so top metrics, model distribution, and recent requests share + one selected window. +- The model and recent-request section subtitles now render the active range + and the resolved time window. This makes a same-number 24h/7d result visibly + understandable instead of looking like a dead dropdown. + +Validation: + +- `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> 50/50 checks + passed. +- `.venv\Scripts\python.exe tests\test_middleware.py` -> 147/147 checks + passed. +- `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py middleware run.py` + -> passed. +- Node parsed `cpa_usage_portal/static/dashboard.html` and + `middleware/dashboard.html` inline scripts successfully. + +Server rollout: + +- Preflight remained disk-tight but stable: `/` was 9.6G total, 8.8G used, + 751M available, 93% used. +- Root-only backup path: + `/root/codex-backups/usage-range-refresh-20260701-231819/`. +- Uploaded only `cpa_usage_portal/app.py` and + `cpa_usage_portal/static/dashboard.html`. +- Rebuilt/restarted only `cpa-usage-portal`; `cpa`, `cpamp`, `codexcont`, and + `caddy-edge` kept their prior uptime. +- Container-internal `/healthz` returned + `{"ok":true,"key_policy_state":true,"cpamp":true}`. +- Public `https://cpa-usage.konbakuyomu.us/` returned `200` and contains the + new selected-range UI code; public `https://cpa.konbakuyomu.us/cpa-usage/` + returned `404`. + +## Follow-up: Price Correction And Local Quota Admin + +Planned implementation: + +1. Add `cpa_usage_portal.quota_state` as the single owner of local SQLite + metadata: 5H/month limits, reset watermarks, and audit log. +2. Extend usage ranges to `5h`, `24h`, `7d`, and `month`. +3. Apply reset watermarks by narrowing CPAMP analytics windows, without + deleting or mutating CPAMP source events. +4. Expose safe local limits/reset points through user `/api/me`, `/api/usage`, + and `/api/events`. +5. Add `/admin/*` routes guarded by a proxy-injected header, plus a static + CPAMP-style admin page for per-key limits and soft reset. +6. Add a writable `/data/portal` mount to the portal deployment example. +7. Correct production Key Policy and CPAMP price data after root-only backups. +8. Route `cpa-admin.konbakuyomu.us/usage-admin/` through the existing admin + proxy, while keeping public usage/API domains from exposing admin routes. + +Local implementation evidence: + +- Added `cpa_usage_portal/quota_state.py` with SQLite tables for local limits, + reset watermarks, and audit log. SQLite connections are explicitly closed so + Windows temp-directory tests can clean up database files. +- `cpa_usage_portal/cpamp.py` now supports `5h`, `24h`, `7d`, and calendar + `month` windows. +- `cpa_usage_portal/app.py` now applies effective reset windows to CPAMP + analytics, returns safe quota projections, and exposes guarded admin APIs. +- `cpa_usage_portal/static/dashboard.html` now lets users switch 5H/day/week/ + month ranges and shows 5H/day/week/month limits plus selected-window + remaining quota. +- Added `cpa_usage_portal/static/admin.html` as a no-build, dark operations + admin page for local quota editing and soft reset. +- Updated `deploy/cpa-usage-portal/docker-compose.example.yaml` with the + writable `/data/portal` mount. +- CPA, CPAMP, and CPA Key Policy source/images remain untouched by the local + implementation. + +Local validation on 2026-07-02: + +- `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> 65/65 checks + passed. +- `.venv\Scripts\python.exe tests\test_middleware.py` -> 147/147 checks + passed. +- `.venv\Scripts\python.exe -m compileall cpa_usage_portal run_usage_portal.py middleware run.py` + -> passed. +- Node parsed inline scripts from + `cpa_usage_portal/static/dashboard.html`, + `cpa_usage_portal/static/admin.html`, and `middleware/dashboard.html` + successfully. + +Server rollout on SJC: + +- Preflight: `/` was 9.6G total, 8.8G used, about 738M available, 93% used. +- Root-only backup path: + `/root/codex-backups/usage-quota-admin-20260702-002407/`. + Backup covered the usage portal stack, Key Policy state, CPAMP SQLite, + admin-proxy Caddyfile, and edge Caddyfile. Backup files were chmod `600`. +- Uploaded only changed custom portal files into + `/opt/codex-stacks/cpa-usage-portal/app`. +- Added the usage portal writable data mount: + `/opt/codex-stacks/cpa-usage-portal/data:/data/portal`. +- Added admin proxy route: + `cpa-admin.konbakuyomu.us/usage-admin/* -> cpa-usage-portal /admin/*` + with `X-Usage-Admin: 1`. +- Added public blocks for `/usage-admin*` on both `cpa.konbakuyomu.us` and + `cpa-usage.konbakuyomu.us`. +- Corrected Key Policy model prices while `cpa` was stopped, then restarted + CPA so it did not overwrite the state file with the old in-memory prices. +- Corrected CPAMP global `model_prices` rows and restarted CPAMP once so its + analytics reloaded the price book. +- Rebuilt/restarted only `cpa-usage-portal`; restarted `cpa-admin-proxy` and + `caddy-edge` for Caddy route changes. CPA and CPAMP were restarted only for + price-state reload. + +Server validation: + +- Key Policy state now has all four enabled keys priced with: + `gpt-5.5 5/30/0.5`, `gpt-5.4 2.5/15/0.25`, + `gpt-5.4-mini 0.75/4.5/0.075`, + `gpt-5.3-codex-spark 1.75/14/0.175`, and + `codex-auto-review 5/30/0.5`. +- CPAMP `model_prices` table has the same five text aliases. After CPAMP + restart, CPAMP monitoring returned nonzero 24h cost; the probe showed + `summary.total_cost = 4.144221850000001` and model-share costs for + `gpt-5.5` and `gpt-5.4`. +- `cpa-usage-portal` health inside its container returned + `{"ok":true,"key_policy_state":true,"cpamp":true}`. +- Admin proxy `http://127.0.0.1:8327/usage-admin/` returned `200` and contains + `CPA 用量管理`. +- Admin API `http://127.0.0.1:8327/usage-admin/api/keys` returned `200`, + listed 4 keys, and exposed `5h`, `24h`, `7d`, and `month` windows with safe + previews only. +- User API with a short-lived internal test session returned safe `/api/me` + limits/reset fields and nonzero Key Policy-derived usage for both `5h` and + `month` ranges. `/api/events?range=5h` returned safe accounting metadata + listing included windows. +- Soft reset was tested on one 5H window: portal 5H usage went from + `0.9155539999999999` to `0.0` after writing the watermark. The test + watermark was then removed, and the 5H usage returned to + `0.9155539999999999`, proving CPAMP rows were not deleted. +- Public `https://cpa.konbakuyomu.us/healthz` returned `200`. +- Public `https://cpa-usage.konbakuyomu.us/` returned `200`. +- `https://cpa-admin.konbakuyomu.us/usage-admin/` returned `302`, preserving + Cloudflare Access. +- Public `https://cpa.konbakuyomu.us/admin/`, + `https://cpa.konbakuyomu.us/usage-admin/`, and + `https://cpa-usage.konbakuyomu.us/admin/` returned `404`. +- Post-rollout disk remained tight: `/` was 9.6G total, 8.9G used, about 670M + available, 94% used. No Docker prune or broad filesystem cleanup was used. + +Known notes: + +- `docker compose up -d cpa` pulled the current `eceasy/cli-proxy-api:latest` + because of the existing compose/image policy. This consumed about 69M of + root disk. No prune or bulk cleanup was performed. +- CPAMP global Model Prices affect CPAMP analytics after CPAMP reload. The + self-service portal still recomputes user-facing costs from Key Policy + prices as a safety overlay. +- The first version only displays and soft-resets local quota windows. It does + not hard-block production CPA requests. diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/prd.md b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/prd.md new file mode 100644 index 0000000..3d57b92 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/prd.md @@ -0,0 +1,104 @@ +# Dashboard Visual Refresh + +## Goal + +Make the two custom dashboards feel like a polished CPAMP-style operations +surface while keeping the ordinary-user and admin boundaries clear. + +The affected pages are: + +- `cpa-usage-portal`: ordinary users inspect their own Key Policy key usage. +- `CodexCont` admin dashboard: administrators inspect 516/518n-2 protection + status and live diagnostics. + +## Requirements + +- Use a shared dark, dense operations style inspired by CPAMP: dark shell, + compact top toolbar, status chips, metric cards, segmented controls, and + scan-friendly tables. +- Do not add a left admin sidebar. The user usage portal must not look like it + grants access to CPAMP or CPA administration. +- Keep CPA, CPAMP, and CPA Key Policy official artifacts untouched. Only custom + pages and custom safe projections may change. +- The CPA usage page must remove long failure summaries from the main table. + Main rows show only time, status, model, latency, tokens, reasoning, cost, + and an action. +- Long failure details must be available only in an expanded detail row and + must be short, redacted, and bounded. +- The CodexCont dashboard must keep the recent request protection result as the + primary first-screen signal. +- Tables must not compress short fields into vertical text on desktop or mobile. + Long fields may truncate, wrap in detail rows, or move behind expand actions. +- No React/Vue/npm build chain. Keep static HTML/CSS/JS. +- Do not expose additional public routes or weaken current public/admin + separation. + +## Acceptance Criteria + +- [x] CPA usage page uses a dark CPAMP-like layout with no side navigation. +- [x] CPA usage page main table has no long summary column and does not render + response headers or raw failure blobs in the first screen. +- [x] CPA usage events include `failure_brief`; full `failure` remains redacted + and is shown only in an expanded row. +- [x] CodexCont dashboard uses the same visual language and keeps protection + states visually distinct. +- [x] Desktop and 390px mobile screenshots show no incoherent overlap or short + fields rendered vertically. +- [x] `tests/test_cpa_usage_portal.py`, `tests/test_middleware.py`, and + compile checks pass. +- [x] Server rollout only restarts `cpa-usage-portal` and `codexcont`; CPA, + CPAMP, and CPA Key Policy remain untouched. +- [x] Public `https://cpa.konbakuyomu.us` still blocks admin/dashboard paths. + +## Out Of Scope + +- Replacing CPAMP or changing CPAMP source. +- Adding a frontend build system. +- Creating new auth flows, user management, or key migration behavior. +- Changing CodexCont `/v1/responses` folding logic. + +## Follow-up: Price Correction, Local Quotas, And Soft Reset + +### Goal + +Fix inflated/zero cost display and add operator-controlled local quota views +without forking CPA, CPAMP, or CPA Key Policy. + +### Requirements + +- Keep CPA, CPAMP, and CPA Key Policy official source/images untouched. +- Correct server configuration data so Key Policy per-key model prices and + CPAMP global Model Prices use USD per 1M tokens. +- Use the confirmed text-model prices: + `gpt-5.5 = 5 / 30 / 0.5`, `gpt-5.4 = 2.5 / 15 / 0.25`, + `gpt-5.4-mini = 0.75 / 4.5 / 0.075`, + `gpt-5.3-codex-spark = 1.75 / 14 / 0.175`, and + `codex-auto-review = 5 / 30 / 0.5`. +- Do not guess image model prices. +- Add a self-owned portal admin entry at + `cpa-admin.konbakuyomu.us/usage-admin/`, still protected by Cloudflare + Access and a proxy-injected admin header. +- Store only local portal metadata in SQLite: 5H/month limits, reset + watermarks, and admin audit entries. +- Add soft reset only: reset portal statistics from a watermark while keeping + CPAMP original events intact. +- User self-service pages must show 5H/day/week/month limits and remaining + estimated quota. +- First version does not hard-block production requests when a local limit is + exceeded. + +### Acceptance Criteria + +- [ ] CPAMP global model price table and Key Policy model entries are corrected + for the text aliases above. +- [ ] User `/api/usage` and `/api/events` support + `range=5h|24h|7d|month`. +- [ ] User `/api/me` exposes safe 5H/month local limits and reset points in + addition to Key Policy daily/weekly limits. +- [ ] Admin page lists every Key with 5H/day/week/month used/limit/remaining + estimates. +- [ ] Admin page can set 5H/month limits and soft-reset one or all windows. +- [ ] Public `cpa-usage.konbakuyomu.us` cannot access admin APIs without the + admin proxy header. +- [ ] No raw API keys, full hashes, OAuth tokens, management keys, request + bodies, response bodies, or encrypted reasoning content are returned. diff --git a/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/task.json b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/task.json new file mode 100644 index 0000000..8e3f743 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-dashboard-visual-refresh/task.json @@ -0,0 +1,26 @@ +{ + "id": "dashboard-visual-refresh", + "name": "dashboard-visual-refresh", + "title": "Dashboard visual refresh", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-01", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/check.jsonl b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/design.md b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/design.md new file mode 100644 index 0000000..9723c10 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/design.md @@ -0,0 +1,53 @@ +# Design + +## Architecture + +The migration replaces the public application relay while keeping the existing SJC egress router layer: + +`client -> cpa.konbakuyomu.us -> caddy-edge -> cpa:8317 -> CPA Codex executor -> socks5://172.19.0.1:1082 -> AT&T residential upstream -> OpenAI` + +The old sub2api stack remains available until CPA passes data-plane verification: + +`client -> sub2api.konbakuyomu.us -> caddy-edge -> sub2api-canary:8080 -> sub2api -> 1082` + +## Server Layout + +- CPA runtime directory: `/opt/codex-stacks/cpa` +- CPA config: `/opt/codex-stacks/cpa/config.yaml` +- CPA auth files: `/opt/codex-stacks/cpa/auths/*.json` +- CPA logs: `/opt/codex-stacks/cpa/logs` +- Compose service: `cpa` +- Docker network: `cpa_net` +- Published port: `127.0.0.1:8317:8317` +- Egress reachability: CPA also joins the existing `sub2api_canary_net` because the retained `sub2api-egress-att` host-network proxy listens specifically on `172.19.0.1:1082`; this keeps the migrated `proxy_url` literal and avoids reconfiguring the egress router. + +## Auth and Access Contracts + +- sub2api API keys are read from the existing database and copied into CPA `api-keys`. +- The active sub2api OAuth account is converted into CPA auth JSON with: + - `type: "codex"` + - token fields copied from sub2api credentials + - `proxy_url: "socks5://172.19.0.1:1082"` + - `disabled: false` +- Non-production or risky accounts are either skipped or written with `disabled: true`. +- Secrets stay on the server in root-only paths. Task artifacts record only counts, IDs, status, paths, and redacted key prefixes/suffixes. + +## Caddy and DNS + +The user has already created DNS for `cpa.konbakuyomu.us`. Caddy will add a new site block using the existing wildcard certificate: + +`cpa.konbakuyomu.us -> cpa:8317` + +The old `sub2api.konbakuyomu.us` route is not removed until CPA passes health, auth, model, and real response checks. + +After CPA passes verification, `sub2api.konbakuyomu.us` returns an explicit `410` response pointing clients at `https://cpa.konbakuyomu.us/`; it must not reverse proxy to the removed sub2api containers. + +## Disk Safety + +The SJC host has limited free space, so the deployment intentionally avoids extra databases and large management components. Docker prune and recursive deletion are forbidden. If space drops below the configured SJC clean threshold, only the existing capacity guard allowlist may be used. + +## Rollback + +- Before Caddy cutover: stop CPA only; leave sub2api untouched. +- After Caddy cutover: restore the previous Caddyfile from the root-only backup and reload Caddy. +- If CPA auth refresh fails: re-login the affected CPA Codex auth; do not rely on the old sub2api non-passthrough reasoning path as the long-term fallback. diff --git a/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/implement.jsonl b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/implement.md b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/implement.md new file mode 100644 index 0000000..85c3f75 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/implement.md @@ -0,0 +1,75 @@ +# Implementation Plan + +## Preflight + +- Verify SJC disk, Docker state, Caddy route, DNS for `cpa.konbakuyomu.us`, `1081`/`1082` egress ASN, and sub2api DB counts. +- Abort before pulling CPA if free disk space is below the SJC clean threshold unless the existing capacity guard allowlist recovers space. + +## Backup + +- Create a root-only backup directory under `/root/cpa-migration-backups/<timestamp>/`. +- Back up: + - sub2api compose and data config files + - Caddyfile + - egress router configs + - SQL dump of sub2api database + - redacted migration manifest with account/API key counts and routing decisions + +## CPA Deployment + +- Create `/opt/codex-stacks/cpa` with `config.yaml`, `docker-compose.yaml`, `auths/`, and `logs/`. +- Generate CPA auth JSON from sub2api DB: + - account `3` enabled with `proxy_url` set to `socks5://172.19.0.1:1082` + - account `2` disabled if exported + - error accounts skipped or disabled +- Copy existing active sub2api API keys into CPA `api-keys`. +- Start CPA and validate `http://127.0.0.1:8317/healthz`. + +## Cutover + +- Add or update Caddy `cpa.konbakuyomu.us` site block to reverse proxy CPA. +- Connect Caddy to `cpa_net` if needed. +- Reload Caddy and verify `https://cpa.konbakuyomu.us/healthz`. +- Run authenticated `/v1/models` and real `/v1/responses` smoke tests. +- Confirm response smoke coincides with `sub2api-egress-att` logs or an equivalent `1082` egress probe. + +## sub2api Retirement + +- Stop and remove only these explicit containers after CPA validation: + - `sub2api-canary` + - `sub2api-canary-postgres` + - `sub2api-canary-redis` +- Do not delete old data directories in this task. +- Verify the egress router containers remain running. + +## Final Verification + +- Check public CPA health and authenticated data-plane. +- Check `docker ps` confirms CPA and egress routers running, sub2api app/Postgres/Redis not running. +- Check disk free space after migration. +- Record verification evidence in the task and report remaining manual client base URL change. + +## Execution Evidence - 2026-07-01 + +- Root-only backup created at `/root/cpa-migration-backups/20260701T034218Z`. +- Backups include sub2api DB dump, sub2api compose/config, Caddyfile, egress router configs, and redacted migration manifests. +- CPA deployed at `/opt/codex-stacks/cpa` with container `cpa`, image `eceasy/cli-proxy-api:latest`; startup log reported `CLIProxyAPI Version: v7.2.47`, commit `00114be`, built `2026-06-29T11:00:58Z`. +- CPA auth migration: one enabled Codex auth from sub2api account `3`, `proxy_url=socks5://172.19.0.1:1082`; one disabled backup auth from account `2`; four active production API keys copied from sub2api group `openai-cpa-poc`. +- CPA initially could not reach `172.19.0.1:1082` from isolated `cpa_net`; confirmed `sub2api-egress-att` listens only on `172.19.0.1:1082`, then attached CPA to `sub2api_canary_net` as well as `cpa_net`. Final `sub2api_canary_net` members: `caddy-edge cpa`. +- Private checks passed: `http://127.0.0.1:8317/healthz` returned `200`; authenticated `/v1/models` returned seven models; authenticated `/v1/responses` returned `200` on `gpt-5.5`. +- Caddy now routes `cpa.konbakuyomu.us -> cpa:8317`; `https://cpa.konbakuyomu.us/healthz` returned `200`. +- Public authenticated data-plane passed: `/v1/models` returned seven models; `/v1/responses` returned `200`, model `gpt-5.5`, response text `cpa-public-ok`. +- Egress evidence after public response: `sub2api-egress-att` logged `172.19.0.6 -> chatgpt.com:443` matching `sub2api-att-residential[sub2api-att-whitelisted-ss]`. +- Old route retired: `https://sub2api.konbakuyomu.us/health` returns `410` with `migrated to https://cpa.konbakuyomu.us/`; Caddyfile no longer contains `reverse_proxy sub2api-canary:8080`. +- Removed only explicit containers: `sub2api-canary`, `sub2api-canary-postgres`, `sub2api-canary-redis`. Preserved `sub2api-egress-att` and `sub2api-egress-direct`. +- Final running relevant containers: `cpa`, `sub2api-egress-att`, `sub2api-egress-direct`. +- Final root disk snapshot: `/dev/sda1 9.6G`, used `8.5G`, available `1.1G`, `89%`. +- DNS note: SJC resolvers `1.1.1.1` and `8.8.8.8` resolve `cpa.konbakuyomu.us -> 38.59.246.182`; this Windows client still returned NXDOMAIN during verification, but `curl --resolve cpa.konbakuyomu.us:443:38.59.246.182` returned `{"status":"ok"}`. + +## Closeout Lessons + +- The production migration is complete for the gateway stack: CPA is the public endpoint, sub2api app/Postgres/Redis are stopped/removed, and the AT&T egress router remains in service. +- This migration does not by itself prove the 516 continuation problem is solved. It removes the old sub2api non-passthrough path and gives a cleaner base for the fix. +- The current CodexCont middleware is the executable behavior reference for 516 continuation: detect the `518 * n - 2` reasoning-token fingerprint, require replayable encrypted reasoning, discard tentative output from truncated rounds, and fold hidden upstream rounds into one downstream Responses stream. +- CPA's ordinary stream chunk plugin surface is insufficient for the full fix because it cannot own same-auth upstream continuation. A durable CPA implementation should live in the Codex executor or a new executor-level supervisor extension point. +- If avoiding a CPA fork is more important than a single-container production shape, keep CPA close to upstream and run CodexCont as a sidecar in front of CPA. If minimizing moving parts is more important, patch CPA natively and keep the patch small, tested, and regularly rebased. diff --git a/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/prd.md b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/prd.md new file mode 100644 index 0000000..dbf327b --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/prd.md @@ -0,0 +1,45 @@ +# SJC sub2api to CPA production migration + +## Goal + +Replace the SJC production sub2api endpoint with lightweight CPA while preserving Codex OAuth auth, AT&T residential egress, and rollback evidence. + +## Confirmed Facts + +- SJC is reachable with `ssh sjc-guard`; the root disk is small (`9.6G`) and had about `1.3-1.4G` free during planning. +- Existing public production endpoint `https://sub2api.konbakuyomu.us/health` returned `200`. +- The user has created the DNS record for `cpa.konbakuyomu.us`. +- Caddy currently reverse proxies `sub2api.konbakuyomu.us` to `sub2api-canary:8080`; `cli-proxy.konbakuyomu.us` is currently `410 gone`. +- Current sub2api image was created on `2026-06-06`, before the later encrypted reasoning preservation fix, and account-level `openai_passthrough` is not enabled. +- The active production OpenAI OAuth account in sub2api is account `3`, assigned to group `openai-cpa-poc`, bound to proxy `SJC-ATT-RESIDENTIAL -> socks5://172.19.0.1:1082`. +- The `1082` egress route currently exits as `AS7018 AT&T Enterprises, LLC`; `1081` remains the direct SJC route. + +## Requirements + +- R1: Deploy a lightweight CPA stack on SJC at `/opt/codex-stacks/cpa` without adding Postgres, Redis, CPA Manager, Usage Keeper, or other persistent services. +- R2: Preserve current usable client API keys so callers can migrate by changing only the base URL from `https://sub2api.konbakuyomu.us/` to `https://cpa.konbakuyomu.us/`. +- R3: Convert usable sub2api OpenAI OAuth credentials into CPA `type: codex` auth JSON files locally on the server without printing or committing tokens. +- R4: Ensure the default CPA production account uses `proxy_url: socks5://172.19.0.1:1082`; do not silently fall back to `1081` or direct SJC egress. +- R5: Add `cpa.konbakuyomu.us` to Caddy using the existing wildcard TLS material and reverse proxy it to the CPA container. +- R6: Keep rollback available until CPA is verified with health, model listing, and a real `/v1/responses` request. +- R7: After CPA verification, stop and remove only the explicitly named sub2api app/Postgres/Redis containers; keep the egress router containers because CPA depends on `1082`. +- R8: Respect the small-disk constraint: do not run `docker system prune`, `docker image prune`, recursive deletes, or bulk directory deletion. + +## Acceptance Criteria + +- [ ] Trellis artifacts exist and record the migration requirements, design, implementation order, evidence, and rollback points. +- [ ] `/opt/codex-stacks/cpa` contains root-only CPA config/auth material and a Docker Compose deployment. +- [ ] `http://127.0.0.1:8317/healthz` and `https://cpa.konbakuyomu.us/healthz` return healthy responses. +- [ ] An authenticated `/v1/models` request succeeds through CPA using a migrated API key. +- [ ] A real authenticated `/v1/responses` request succeeds through CPA and produces evidence that `1082`/AT&T egress was used. +- [ ] `sub2api-canary`, `sub2api-canary-postgres`, and `sub2api-canary-redis` are not running after cutover. +- [ ] `sub2api-egress-att` and `sub2api-egress-direct` remain running. +- [ ] No secret-bearing `.env`, auth JSON, API key, OAuth token, or database dump is written into Git or Obsidian. +- [ ] No Docker prune, recursive deletion, or bulk deletion is performed. + +## Out of Scope + +- Migrating sub2api usage history or dashboard state into CPA. +- Installing a long-term CPA management dashboard. +- Deleting old sub2api data directories without a separate explicit user confirmation. +- Retiring the egress routers or changing the upstream residential subscription architecture. diff --git a/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/task.json b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/task.json new file mode 100644 index 0000000..1cc45c4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-01-sjc-sub2api-cpa-production-migration/task.json @@ -0,0 +1,26 @@ +{ + "id": "sjc-sub2api-cpa-production-migration", + "name": "sjc-sub2api-cpa-production-migration", + "title": "SJC sub2api to CPA production migration", + "description": "Replace the SJC production sub2api endpoint with lightweight CPA while preserving Codex OAuth auth, AT&T residential egress, and rollback evidence.", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P0", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-01", + "completedAt": "2026-07-01", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/deploy/build-linux-plugin-wsl.sh b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/deploy/build-linux-plugin-wsl.sh new file mode 100644 index 0000000..3433289 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/deploy/build-linux-plugin-wsl.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash +set -euo pipefail +WORK="/tmp/codex-go-build-$(date +%s)" +mkdir -p "$WORK" +cd "$WORK" +curl -fsSL https://go.dev/dl/go1.22.6.linux-amd64.tar.gz -o go.tar.gz +tar -xzf go.tar.gz +export PATH="$WORK/go/bin:$PATH" +go version +cd /mnt/d/Dev/20_Software/23_Reference/llm-gateway/CodexCont/cpa_governor_plugin/go +mkdir -p ../dist/linux/amd64 +CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags cliproxy_plugin -buildmode=c-shared -o ../dist/linux/amd64/cpa-governor.so . +sha256sum ../dist/linux/amd64/cpa-governor.so +file ../dist/linux/amd64/cpa-governor.so diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/deploy/deploy-governor-sjc.sh b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/deploy/deploy-governor-sjc.sh new file mode 100644 index 0000000..b657a41 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/deploy/deploy-governor-sjc.sh @@ -0,0 +1,220 @@ +#!/usr/bin/env bash +set -euo pipefail + +TS="$(date +%Y%m%d-%H%M%S)" +BACKUP_DIR="/root/cpa-governor-backup-${TS}" +STAGING="/tmp/cpa-governor-deploy" +EXPECTED_SO_SHA="7f4e31cab8c214f8985c7a6a7fabbd90a559bd3b206e65bcac28f21a9a8a75d3" + +log() { printf '[deploy] %s\n' "$*"; } + +log "preflight disk" +df -h / +mkdir -p "$BACKUP_DIR" +chmod 700 "$BACKUP_DIR" + +backup_one() { + local src="$1" + local rel="${src#/}" + local dst="$BACKUP_DIR/$rel" + if [ -e "$src" ]; then + mkdir -p "$(dirname "$dst")" + cp -a "$src" "$dst" + log "backed up $src" + else + log "missing $src" + fi +} + +backup_one /opt/codex-stacks/cpa/config.yaml +backup_one /opt/codex-stacks/cpa/docker-compose.yaml +backup_one /opt/codex-stacks/cpa/plugin-state/cpa-key-policy-state.json +backup_one /opt/codex-stacks/codexcont/docker-compose.yaml +backup_one /opt/codex-stacks/codexcont/config.toml +backup_one /opt/codex-stacks/codexcont/app/middleware/app.py +backup_one /opt/codex-stacks/codexcont/app/middleware/engine.py +backup_one /opt/codex-stacks/caddy/Caddyfile +backup_one /opt/codex-stacks/cpa-admin-tunnel/Caddyfile + +log "verify artifact checksum" +ACTUAL_SO_SHA="$(sha256sum "$STAGING/cpa-governor.so" | awk '{print $1}')" +if [ "$ACTUAL_SO_SHA" != "$EXPECTED_SO_SHA" ]; then + echo "artifact sha mismatch: $ACTUAL_SO_SHA" >&2 + exit 20 +fi + +log "install codexcont engine files" +install -m 0644 "$STAGING/app.py" /opt/codex-stacks/codexcont/app/middleware/app.py +install -m 0644 "$STAGING/engine.py" /opt/codex-stacks/codexcont/app/middleware/engine.py + +log "install governor plugin artifact" +mkdir -p /opt/codex-stacks/cpa/plugins/linux/amd64 +install -m 0644 "$STAGING/cpa-governor.so" /opt/codex-stacks/cpa/plugins/linux/amd64/cpa-governor.so +mkdir -p /opt/codex-stacks/cpa/plugin-state/cpa-governor +chmod 700 /opt/codex-stacks/cpa/plugin-state/cpa-governor + +log "patch CPA config safely" +python3 - <<'PY' +from pathlib import Path +import secrets +import yaml +p = Path('/opt/codex-stacks/cpa/config.yaml') +cfg = yaml.safe_load(p.read_text()) or {} +plugins = cfg.setdefault('plugins', {}) +plugins['enabled'] = True +plugins['dir'] = '/CLIProxyAPI/plugins' +configs = plugins.setdefault('configs', {}) +existing = configs.get('cpa-governor') or {} +secret = existing.get('session_secret') or secrets.token_urlsafe(48) +existing.update({ + 'enabled': True, + 'priority': 20, + 'exclusive_auth': False, + 'state_db_path': '/CLIProxyAPI/plugin-state/cpa-governor/governor.sqlite', + 'key_policy_state_path': '/CLIProxyAPI/plugin-state/cpa-key-policy-state.json', + 'session_secret': secret, + 'codexcont_enabled': True, + 'codexcont_route': False, + 'codexcont_url': 'http://codexcont:8787', + 'fail_mode': 'fallback', +}) +configs['cpa-governor'] = existing +text = yaml.safe_dump(cfg, allow_unicode=True, sort_keys=False, default_flow_style=False) +p.write_text(text) +p.chmod(0o600) +print('CPA_CONFIG_PATCHED cpa-governor enabled passive') +PY + +log "patch admin proxy Caddy routes" +python3 - <<'PY' +from pathlib import Path +p = Path('/opt/codex-stacks/cpa-admin-tunnel/Caddyfile') +text = p.read_text() +block = ''' + # CPA Governor plugin surfaces. + handle /governor { + redir /governor/ 307 + } + + handle /governor/ { + rewrite * /v0/resource/plugins/cpa-governor/admin + reverse_proxy cpa:8317 { + import cpa_local_headers + } + } + + handle_path /governor/* { + rewrite * /v0/resource/plugins/cpa-governor/admin{uri} + reverse_proxy cpa:8317 { + import cpa_local_headers + } + } + + handle /governor-user { + redir /governor-user/ 307 + } + + handle /governor-user/ { + rewrite * /v0/resource/plugins/cpa-governor/user + reverse_proxy cpa:8317 { + import cpa_local_headers + } + } + + handle_path /governor-user/* { + rewrite * /v0/resource/plugins/cpa-governor/user{uri} + reverse_proxy cpa:8317 { + import cpa_local_headers + } + } +''' +if '/governor/*' not in text: + markers = ['\n\thandle /codexcont {', '\n handle /codexcont {'] + for marker in markers: + if marker in text: + text = text.replace(marker, '\n' + block + marker, 1) + break + else: + raise SystemExit('admin proxy insertion marker not found') +p.write_text(text) +print('ADMIN_PROXY_PATCHED governor routes present') +PY + +log "patch public cpa-usage host to Governor user surface" +python3 - <<'PY' +from pathlib import Path + +p = Path('/opt/codex-stacks/caddy/Caddyfile') +text = p.read_text() +start = text.find('cpa-usage.konbakuyomu.us {') +if start < 0: + raise SystemExit('cpa-usage block not found') +brace = 0 +end = None +for idx in range(start, len(text)): + ch = text[idx] + if ch == '{': + brace += 1 + elif ch == '}': + brace -= 1 + if brace == 0: + end = idx + 1 + break +if end is None: + raise SystemExit('cpa-usage block end not found') +old = text[start:end] +inner_lines = old.splitlines()[1:-1] +preserved = [] +for line in inner_lines: + stripped = line.strip() + if not stripped or stripped.startswith('#'): + continue + if stripped.startswith(('encode', '@', 'respond', 'reverse_proxy', 'handle')): + continue + preserved.append(line) +new_lines = ['cpa-usage.konbakuyomu.us {'] +new_lines.extend(preserved[:2]) +new_lines.extend([ + ' encode zstd gzip', + '', + ' handle /favicon.ico {', + ' respond 204', + ' }', + '', + ' handle / {', + ' rewrite * /v0/resource/plugins/cpa-governor/user', + ' reverse_proxy cpa:8317', + ' }', + '', + ' handle /v0/resource/plugins/cpa-governor/user* {', + ' reverse_proxy cpa:8317', + ' }', + '', + ' handle {', + ' respond 404', + ' }', + '}', +]) +text = text[:start] + '\n'.join(new_lines) + text[end:] +p.write_text(text) +print('PUBLIC_USAGE_PATCHED governor user surface') +PY + +log "rebuild CodexCont image" +docker compose -f /opt/codex-stacks/codexcont/docker-compose.yaml build codexcont +docker compose -f /opt/codex-stacks/codexcont/docker-compose.yaml up -d --no-deps codexcont + +log "restart CPA for plugin load" +docker restart cpa >/dev/null + +log "restart admin proxy for routes" +docker restart cpa-admin-proxy >/dev/null + +log "reload public Caddy edge" +docker exec caddy-edge caddy validate --config /etc/caddy/Caddyfile +docker exec caddy-edge caddy reload --config /etc/caddy/Caddyfile + +log "post status" +docker ps --format 'table {{.Names}}\t{{.Status}}' +sha256sum /opt/codex-stacks/cpa/plugins/linux/amd64/cpa-governor.so +printf 'BACKUP_DIR=%s\n' "$BACKUP_DIR" diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/design.md b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/design.md new file mode 100644 index 0000000..64a966b --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/design.md @@ -0,0 +1,78 @@ +# CPA Governor plugin and CodexCont engine design + +## Architecture + +Governor is a self-owned CPA plugin plus a small state database. CPA remains the public API entrypoint. Governor authenticates frontend keys, enforces policy, records usage, serves admin/user pages, and optionally invokes CodexCont Engine before upstream execution. + +CodexCont remains a separate Python container. Its new engine API is internal-only and focused on protection decisions and safe summaries. The current middleware/admin dashboard stays available during migration as rollback and evidence tooling. + +## Plugin capabilities + +- `FrontendAuthProvider`: validates Governor/Key Policy-compatible keys and returns a stable principal. It rejects disabled keys and unknown keys before CPA routing. +- `ModelRouter + Executor`: routes protected Responses requests to Governor execution. The executor can call CodexCont Engine and CPA host model callbacks. If engine is disabled or unavailable, behavior follows configured fail mode. +- `UsagePlugin`: consumes CPA `UsageRecord`, normalizes token/cache/reasoning/cost data, and stores a safe event projection. +- `ManagementAPI`: exposes authenticated admin APIs and browser resources for the CPA Admin plugin menu. +- Resource routes: expose user self-service pages and GET-only user APIs; sensitive mutations stay in Management APIs. + +## State model + +Governor SQLite stores: + +- `keys`: id, name, safe preview, key hash, enabled flag, model allowlist, rpm, concurrency, created/updated timestamps. +- `limits`: 5H/day/week/month USD limits per key. +- `prices`: per-key/per-model input, output, cache-read, cache-create prices per million tokens. +- `reset_watermarks`: per-key per-window soft reset timestamp. +- `usage_events`: safe request events, token counters, cache buckets, reasoning tokens, estimated cost, latency, status, failure summary. +- `codexcont_summaries`: request id, key id, protection result, hit round, latest/final reasoning token counters, continuation count, stop/failure reason. +- `audit_log`: admin operations, target key/config, timestamp, safe actor/source. + +No raw key, OAuth token, Authorization header, raw request/response body, encrypted reasoning content, or true CoT is stored. + +## CodexCont Engine contract + +- `GET /engine/healthz` returns service health, uptime, mode, and version. +- `POST /engine/v1/responses/analyze` accepts a sanitized request envelope plus optional chunk/usage summaries and returns a protection decision/summary. +- Full folded execution is enabled behind a feature flag once plugin host-model callback execution is validated in production. Until then, the current sidecar continuation path remains the production fallback. + +The engine returns only safe fields: request id, model, protection value, first truncation round, first truncation reasoning tokens, latest/final reasoning tokens, continuation count, folded flag, stopped reason, failure summary. + +## UI design + +Admin page inside CPA Admin: + +- Key management and bulk save/reset. +- Global usage and request detail. +- CodexCont config and health. +- Global protection status and audit log. + +User page: + +- Login with CPA/Governor key via Authorization header, not URL. +- Tabs: quota/usage, thought-chain protection, request details. +- 1-2 second polling with visible refresh/reconnect state. +- Own-key filtering enforced server-side. + +## Rollout and rollback + +1. Deploy CodexCont engine API without removing current middleware path. +2. Deploy Governor plugin disabled/passive: admin/user pages and usage ingestion first. +3. Import/mirror existing Key Policy keys and verify user/admin visibility. +4. Enable hard quota enforcement for test keys, then normal keys. +5. Enable protected executor path for test key/model only. +6. Remove Caddy `/v1/responses -> codexcont` front split only after real production smoke passes. + +Rollback keeps current working Caddy/CodexCont/CPA path. Governor can be disabled from CPA plugin config without deleting state. + +## Implementation notes + +- Governor refreshes the Key Policy state file opportunistically. UI reads, + user login, and usage ingestion re-check file mtime with a short debounce so + newly created Key Policy keys appear without restarting CPA. +- CPA frontend auth providers do not have a separate hard-deny return shape: + an unauthenticated Governor response means "not handled". Therefore hard + quota/RPM/concurrency blocking requires Governor to run as the exclusive + frontend auth provider, or the old user auth plugin must not accept the same + user keys. +- `codexcont_enabled` and `codexcont_route` are intentionally separate. + Production can show CodexCont engine health and status while keeping the + current Caddy-fronted continuation path until executor continuation is proven. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/implement.md b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/implement.md new file mode 100644 index 0000000..ec0a156 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/implement.md @@ -0,0 +1,100 @@ +# CPA Governor plugin and CodexCont engine implementation plan + +## Steps + +1. Inspect current Python middleware/usage portal and CPA plugin examples for reusable logic. +2. Add CodexCont engine API routes and safe summary projection tests. +3. Add a `cpa_governor_plugin` Go module with core packages for hashing, quotas, pricing, storage, safe projection, admin/user JSON handlers, and plugin registration skeleton. +4. Implement Governor UI resources as static HTML served by the plugin. +5. Add local tests for Python engine and Go core logic. +6. Build or document the Linux plugin artifact path; prefer server-side build if local cross-build is unavailable. +7. Prepare deployment files/scripts that do not modify official CPA/CPAMP/Key Policy images. +8. Deploy cautiously on SJC: backup, check disk, upload changed files/artifacts, restart only CodexCont/CPA as needed. +9. Validate public/admin/user routes, CodexCont engine health, key visibility, usage projection, and path blocking. +10. Record results and update README/specs with the new architecture and residual risks. + +## Validation commands + +- `python -m compileall middleware cpa_usage_portal run.py run_usage_portal.py` +- `python tests/test_middleware.py` +- `python tests/test_cpa_usage_portal.py` +- `go test ./...` inside `cpa_governor_plugin/go` +- Playwright screenshots for admin/user pages if a local server or deployed route is available. +- Server: `df -h /`, `docker ps`, `curl` health checks, authenticated test request, and public path blocking checks. + +## Risk controls + +- Do not use Docker prune or recursive deletes. +- Do not print or commit API keys/OAuth tokens/management keys. +- Keep old production path until Governor protected executor path is verified. +- Any server-side source/build artifacts must be small and backed up first. +- If plugin host callback constraints block full request execution, ship passive/admin/user/engine improvements and document the remaining cutover gate. + +## Execution record + +- Local tests passed on 2026-07-02: + - `go test ./...` in `cpa_governor_plugin/go` + - `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py` + - `.venv\Scripts\python.exe tests\test_middleware.py` (`163/163`) + - `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` (`84/84`) +- Linux plugin artifact: + - Built from WSL with a temporary Go 1.22.6 linux/amd64 toolchain. + - Local/remote SHA256: `7f4e31cab8c214f8985c7a6a7fabbd90a559bd3b206e65bcac28f21a9a8a75d3`. + - Verified as ELF 64-bit x86-64 shared object. +- SJC deployment: + - Backed up sensitive/runtime files to `/root/cpa-governor-backup-20260702-055926`. + - Additional Caddy public-edge backup before switching the user portal route: `/root/caddy-Caddyfile-before-governor-user-20260702-061604`. + - Uploaded CodexCont `middleware/app.py`, `middleware/engine.py`, and Governor plugin `.so`. + - Added `plugins.configs.cpa-governor` in CPA config with `exclusive_auth: false` and `codexcont_route: false`. + - Added admin proxy routes: + - `https://cpa-admin.konbakuyomu.us/governor/` + - `https://cpa-admin.konbakuyomu.us/governor-user/` + - Switched `https://cpa-usage.konbakuyomu.us/` from the old Python usage portal to Governor's user page, exposing only the user resource/API surface. + - Rebuilt only `codexcont`; restarted `cpa`, `cpa-admin-proxy`; reloaded `caddy-edge`. No Docker prune and no broad deletion. +- Production validation: + - `https://cpa.konbakuyomu.us/healthz` returned `200`. + - Authenticated `https://cpa.konbakuyomu.us/v1/models` returned `200`. + - Authenticated real `https://cpa.konbakuyomu.us/v1/responses` returned `200`, `status=completed`, `model=gpt-5.5`. + - CPA logs showed Governor loaded and registered from `/CLIProxyAPI/plugins/linux/amd64/cpa-governor.so`. + - Governor admin API via `127.0.0.1:8327/governor/api/keys` returned `ok=true`, `keys=3`, CodexCont `health_ok=true`, `route=false`. + - Public `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-governor/admin`, `/admin/requests`, `/codexcont/`, `/governor/` returned `404`. + - Public `https://cpa-usage.konbakuyomu.us/` returned the Governor user page; `/admin/`, `/usage-admin/`, and `/v0/resource/plugins/cpa-governor/admin` returned `404`. + - User API without session returned `401`; invalid key login returned `401 {"error":"invalid_api_key","ok":false}`. + - Final root disk state remained tight but usable: about `638M` free on `/`. +- Playwright validation: + - Access-protected admin route redirected to Cloudflare Access as expected in an unauthenticated browser. + - Local SSH tunnel to admin proxy showed Governor admin page, Key management, request details, and CodexCont tabs render. + - Mobile 390px viewport initially exposed compressed table columns; fixed by giving tables a mobile minimum width inside an overflowed panel. + - Re-validated mobile: Key table no longer collapses into vertical text; user page opens on `https://cpa-usage.konbakuyomu.us/`. +- Follow-up fix on 2026-07-02: + - User report: `cpa-usage.konbakuyomu.us` returned `invalid_api_key` for native `sk...` / newly created keys, and `cpa-admin.konbakuyomu.us/governor/` looked like a duplicate of the CPAMP sidebar `CPA Governor`. + - Evidence: Key Policy state on SJC updated and Governor saw 4 keys through `127.0.0.1:8327/governor/api/keys`, so the main issue was not missing state sync. Native CPA `sk...` keys are not valid user-portal credentials; the portal expects the full Key Policy `cpa_...` key. + - Root cause found during smoke: CPA plugin `ResourceRoute` dispatch is GET-only. `GET /v0/resource/plugins/cpa-governor/user/api/session` entered the plugin and returned the new Chinese error body, while `POST` returned `404` before the plugin. The frontend now uses GET plus `Authorization` header; keys are not placed in URLs. + - UX fix: user portal now normalizes pasted `Authorization: Bearer ...` / `Bearer ...` shapes and returns clear Chinese messages for native `sk...` keys, shortened previews, unsupported formats, disabled keys, and unmatched full `cpa_...` keys. + - UX fix: Governor admin page now states that CPAMP sidebar `CPA Governor` and `/governor/` are the same plugin page; `/governor/` is only a direct/debug entrypoint. + - Built linux/amd64 plugin SHA256 `103c63a4f151c47cda932855cee2b5d4d6fe8bbe203460be0b20cbef9cd6351d`; backed up previous plugin to `/root/cpa-governor-plugin-backup-20260702-093207`; uploaded only the `.so` and restarted only `cpa`. + - SJC verification: CPA loaded and registered Governor from `/CLIProxyAPI/plugins/linux/amd64/cpa-governor.so`; `https://cpa.konbakuyomu.us/healthz` returned `200`; authenticated `/v1/models` returned `200`; Governor API returned `ok=true`, `keys=4`, `codexcont.health_ok=true`, `route=false`; public API admin/plugin paths still returned `404`; root disk remained tight at about `629M` free. + - Playwright verification: `https://cpa-usage.konbakuyomu.us/` displayed the new full-`cpa_` helper text; clicking login with `sk-test` issued `GET /v0/resource/plugins/cpa-governor/user/api/session` and rendered `登录失败：这是 CPA 原生 sk Key，不能登录用量自助页。请使用 Key Policy 创建时弹窗里的完整 cpa_ 用户 Key。` + - Local checks after fix: `go test ./...` in `cpa_governor_plugin/go`; `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py`; scoped `git diff --check`. +- Header/caching hotfix on 2026-07-02: + - User report: one current Key Policy key could log in through `https://cpa-usage.konbakuyomu.us/`, but the CPAMP embedded `CPA Usage` page still returned the unmatched-key message; another QQ key failed on both surfaces. + - Evidence: hashing the QQ key supplied by the user did not match any current Key Policy `key_hash`; current QQ record preview was a different `cpa_...` key, so that failure is a rotated/stale-key case and requires Key Policy rotation to obtain a new full key. + - Root cause for the CPAMP embedded failure: the embedded plugin surface can carry CPAMP/admin credentials in `Authorization`, while the Governor login code was using `Authorization` as the only user-key transport. A valid user key could be ignored or overwritten on that surface. + - Fix: user portal login now sends the raw user key in `X-CPA-Governor-Key`; backend prefers `X-CPA-Governor-Key` / `X-CPA-User-Key`, falls back to `Authorization` only for direct calls, and reads headers case-insensitively. JSON and HTML plugin responses now include `Cache-Control: no-store`. + - Tests: added Go regression coverage for dedicated-header precedence over `Authorization`, case-insensitive header lookup, and no-store session responses. `go test ./...` passed in `cpa_governor_plugin/go`; scoped `git diff --check` passed. + - Built linux/amd64 plugin SHA256 `efa1133a896b0cac58777b0477f2cc6bd38ccc65a3c58b95ca9bdb9babb78ea6`; backed up previous plugin to `/root/cpa-governor-nostore-hotfix-20260702-114432`; uploaded only the `.so` and restarted only `cpa`. + - SJC verification: CPA loaded and registered Governor after restart; `https://cpa.konbakuyomu.us/healthz` returned `200`; `https://cpa-usage.konbakuyomu.us/` returned `200` with `Cache-Control: no-store`; public API plugin path and user-host admin path returned `404`; unauthenticated user API returned `401`; valid current key login via both user host and admin proxy returned `200`; stale QQ key returned `401 invalid_api_key`. Root disk remained tight at about `515M` free. +- Final closeout on 2026-07-02: + - User confirmed the previously reported Governor / CPA Usage login problems were fixed. + - Experience retained in code-spec: CPAMP embedded plugin pages must not rely on `Authorization` for user-key login because admin shells may own that header; Key Policy rotation means only the newly generated full `cpa_...` key can match the current `key_hash`, while old full keys and list previews must fail safely. + - Task accepted as complete in passive Governor mode. Remaining executor-level CodexCont cutover stays explicitly documented as a future gate, not part of this closeout. + +## Residual risk / next cutover gate + +- Production remains in safe passive mode: + - `codexcont_enabled: true` + - `codexcont_route: false` + - `exclusive_auth: false` +- Real `/v1/responses` traffic still uses the already-proven Caddy front split: + `cpa.konbakuyomu.us/v1/responses -> codexcont:8787 -> cpa:8317`. +- Governor is now the unified UI/observability surface, but it is not yet the exclusive quota enforcer or the executor-level 516 continuation owner. Full cutover requires CPA host-model callback continuation validation with a test key before changing `codexcont_route` or removing the Caddy front split. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/prd.md b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/prd.md new file mode 100644 index 0000000..bcb8715 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/prd.md @@ -0,0 +1,44 @@ +# CPA Governor plugin and CodexCont engine + +## Goal + +Move the production CPA/CodexCont integration from the current Caddy-fronted sidecar shape into a cleaner CPA-owned architecture: + +```text +Client -> CPA -> CPA Governor plugin -> optional CodexCont Engine -> CPA upstream execution path +``` + +The user-facing outcome is one coherent system: CPA stays official, CodexCont becomes a pure 516/518n-2 protection engine, and CPA Governor owns normal user keys, quotas, usage, request detail, CodexCont status, and both admin/user panels. + +## Requirements + +- Create and ship a self-owned CPA Governor plugin without modifying CPA, CPAMP, or CPA Key Policy official source/images. +- Keep existing production service usable during migration; do not remove the current Caddy/CodexCont sidecar path until the Governor path passes validation. +- Import or mirror existing Key Policy `cpa_...` users into Governor-managed identity records so ordinary users continue using CPA-style keys. +- Governor must be able to enforce enabled/disabled state, model allowlist, RPM, concurrency, and 5H/24H/7D/month USD quota before any upstream request is sent. +- Governor must persist authoritative key metadata, limits, reset watermarks, usage events, CodexCont protection summaries, and audit entries in its own SQLite database. +- Governor admin UI must live inside the CPA Admin plugin surface and include key management, limit/save/reset, request detail, CodexCont configuration/status, and audit information. +- Governor user UI must replace the independent usage portal as the target design, with tabs for quota/usage, CodexCont protection, and request details. Users must only see their own key's data. +- CodexCont Docker must gain an engine mode/API that can return safe protection summaries and never persist raw requests, responses, Authorization, OAuth tokens, encrypted reasoning, or true CoT. +- Realtime UI v1 can use 1-2 second polling rather than SSE. +- Public `cpa.konbakuyomu.us` must not expose management, plugin resource, CodexCont admin, CPAMP, or usage-admin internals. +- SJC deployment must respect small-disk constraints: no Docker prune, no broad deletion, backup before changes, and restart only required services. + +## Acceptance Criteria + +- [ ] Trellis `prd.md`, `design.md`, and `implement.md` record the architecture, safety boundaries, and rollout plan. +- [x] Local tests cover Governor key hashing, quota windows, price calculation, reset watermarks, safe event projection, and CodexCont engine summaries. +- [x] A Linux-compatible CPA Governor plugin artifact or build path exists and is documented. +- [x] CodexCont exposes `GET /engine/healthz` and a safe engine endpoint for protection summaries. +- [x] Admin UI shows all keys, limits, usage, request details, CodexCont config/status, and can save/reset through authenticated management APIs. +- [x] User UI can show own quota/usage, own request details, and own CodexCont status without exposing other users or secrets. +- [x] Server deployment keeps CPA official image and CPAMP/Key Policy official artifacts unchanged. +- [x] Production smoke verifies `/healthz`, authenticated model/request path, Governor user/admin pages, CodexCont engine health, and public admin-path blocking. +- [ ] If full executor cutover is not safe in one pass, the deployed state must still improve operations safely and leave a documented toggle/rollback to the current working path. + +## Out of Scope + +- Forking CPA, CPAMP, or CPA Key Policy. +- Claiming 516 is mathematically impossible after the change. +- Deleting old data directories or pruning Docker images. +- Exposing true reasoning content or encrypted reasoning payloads in any UI. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/task.json b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/task.json new file mode 100644 index 0000000..68fb99b --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-governor-codexcont-engine/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-governor-codexcont-engine", + "name": "cpa-governor-codexcont-engine", + "title": "CPA Governor plugin and CodexCont engine", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/design.md b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/design.md new file mode 100644 index 0000000..6f314bc --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/design.md @@ -0,0 +1,102 @@ +# Design + +## Boundaries + +This task modifies only the self-owned `cpa_key_policy_plus_plugin` and any deployment/admin-proxy route notes required for `/key-policy-plus/api/*`. CPA, CPAMP, and old Key Policy upstream code remain untouched. + +The Plus admin HTML remains a CPA plugin resource because CPAMP can render plugin resources from the left menu. Resource routes are safe for HTML and read-only GETs only. Mutations use CPA plugin management routes. + +## Admin API Transport + +The browser uses a single API base resolver: + +1. Prefer `/key-policy-plus/api` on `cpa-admin.konbakuyomu.us`. +2. In local preview/test, support an explicit `window.CPA_KEY_POLICY_PLUS_API_BASE` override. +3. Do not fall back to mutating `/v0/resource/plugins/cpa-key-policy-plus/admin/api`. + +The admin proxy must map: + +```text +/key-policy-plus/api/* -> /v0/management/plugins/cpa-key-policy-plus/* +``` + +The plugin already owns management routes such as `GET /plugins/cpa-key-policy-plus/keys`, `POST /plugins/cpa-key-policy-plus/keys/create`, `PUT /plugins/cpa-key-policy-plus/keys/save`, and `POST /plugins/cpa-key-policy-plus/keys/reset`. + +## Model Discovery + +Add a management endpoint: + +```text +GET /plugins/cpa-key-policy-plus/models +``` + +The endpoint returns a safe normalized shape: + +```json +{ + "models": [ + {"id": "gpt-5.5", "source": "configured", "known": true} + ], + "warnings": [] +} +``` + +Model sources are merged with stable de-duplication: + +- optional configured static/default model catalog from plugin config, +- current key allowlist union from Plus SQLite, +- preserved unknown models from existing keys. + +If CPA management model discovery is wired through the admin proxy in a future step, the frontend can merge those returned models with this endpoint without changing persistence. For this task, Plus must at least stop hard-coding a tiny model list and must not delete configured unknowns. + +## Admin UI Data Flow + +```text +Plus store -> /keys -> admin state +Plus model projection -> /models -> model selector state +admin edits -> normalized payload -> /keys/save +create dialog -> /keys/create -> one-time raw key modal -> reload keys +reset button -> /keys/reset -> reload keys +``` + +The table owns a draft copy of keys. Rendering formats the draft only; validation and normalization happen before sending to the API. + +## Model And Price Editing + +Main table columns show summary fields instead of long text: + +- model count and unknown count, +- price coverage `priced/selected`, +- `编辑模型/价格` action. + +The modal edits one selected key at a time. It contains: + +- search box, +- all visible/clear buttons, +- selected model chips, +- discovered model list with checkboxes, +- unknown configured model notice, +- structured price rows for selected models. + +Price units are fixed to USD per 1M tokens: + +- input, +- output, +- cache read, +- cache write/creation. + +Unknown selected models stay selected and writable. Saving never removes a model unless the admin explicitly unchecks/removes it. + +## Security + +Raw key material appears only in the successful-create response modal. The UI does not persist it in local storage or write it into URLs. API responses must keep existing safe key previews and must not add full hashes or secrets. + +## Compatibility + +Existing API payload fields are preserved where possible (`models`, `prices`, `limits`, `rpm`, `concurrency`, `max_active_sessions`). The UI may send both structured `model_prices` and compatibility `prices` shapes if the backend already expects one of them. + +If `/models` fails, the UI falls back to the union of models already present in loaded keys and displays a warning instead of blocking all edits. + +## Rollout And Rollback + +Rollout is plugin-only plus a small admin-proxy alias. Back up the old `.so`, Plus SQLite DB, and admin proxy route file before replacing the plugin. Roll back by restoring the previous `.so` and proxy route, then restarting/reloading only the affected services. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/implement.md b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/implement.md new file mode 100644 index 0000000..f9c1f84 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/implement.md @@ -0,0 +1,81 @@ +# Implementation Plan + +## Checklist + +1. Start the Trellis task after these artifacts are written. +2. Inspect the existing Plus store/API payloads and tests. +3. Add or refine backend management handlers: + - `/plugins/cpa-key-policy-plus/models`, + - safe model normalization and key-union fallback, + - transport tests for create/save/reset through management routes. +4. Rework `assets/admin.html`: + - remove raw model/price textarea workflow, + - add create dialog, + - add model/price modal, + - resolve API base to `/key-policy-plus/api`, + - keep no-store and secret-safe behavior. +5. Add tests: + - Go backend tests for model projection and management writes, + - HTML static tests proving no mutating ResourceRoute fallback, + - JS syntax check helper command. +6. Run local validation. +7. Build linux/amd64 plugin artifact only if local validation passes. +8. Prepare server rollout notes: + - backup old `.so`, Plus SQLite, CPA config, admin proxy config, + - add `/key-policy-plus/api/*` admin-proxy alias, + - restart/reload only CPA/admin proxy as needed, + - verify new key creation and public/admin route boundaries. + +## Validation Commands + +```powershell +go test ./... +node --check <extracted-admin-script.js> +git diff --check +``` + +If the repo-level Python tests are affected or touched, also run: + +```powershell +.venv\Scripts\python.exe tests\test_middleware.py +.venv\Scripts\python.exe tests\test_cpa_usage_portal.py +.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py +``` + +## Risk Points + +- ResourceRoute remains GET-only; any accidental `POST`/`PUT` fallback to `/v0/resource/plugins/...` will recreate the original failure. +- Existing configured models must be preserved when the current discovery source returns an empty or stale list. +- The full generated key is unrecoverable after the create modal closes; do not imply it can be re-read. +- Do not leak management keys, raw user keys, full hashes, or auth-file details while adding model discovery. + +## Rollback Points + +- Before deployment: keep the previous plugin `.so` and admin proxy route file. +- If create/save fails after deploy: restore old `.so`, remove or bypass `/key-policy-plus/api/*` alias, restart CPA/admin proxy. +- If public routes expose admin/plugin resources: roll back Caddy/admin proxy change first, then investigate plugin behavior. + +## Execution Notes + +- Implemented Plus admin model catalog, safe model option normalization, structured create modal, and structured model/price editor. +- Fixed local preview body forwarding so Playwright can exercise real create/save requests. +- Added `/key-policy-plus/api/*` management alias support and server admin-proxy route. Production route injects the CPA management key from the existing mounted secret, not from a committed Caddyfile literal. +- Built linux/amd64 plugin with WSL Go 1.22.6. Deployed SHA256: + `0cc73b598afd9ce109aa27d8bc0522c257dfe93d2f60b34aee481b5906c55b1d`. +- Server backup path: + `/opt/codex-stacks/backups/cpa-key-policy-plus-admin-fixes-20260702-211634`. + +## Validation Results + +- `go test ./...` passed in `cpa_key_policy_plus_plugin/go`. +- Admin HTML inline script `node --check` passed. +- `tests/test_middleware.py`: 163/163 passed. +- `tests/test_cpa_usage_portal.py`: 84/84 passed. +- `python -m compileall middleware cpa_usage_portal run.py run_usage_portal.py` passed. +- Playwright local preview verified create, model selection, price save, and 390px layout without page-level horizontal overflow. +- Server verified: + - CPA logs show `cpa-key-policy-plus` loaded and registered. + - `/key-policy-plus/api/keys` and `/key-policy-plus/api/models` return `200`. + - Disabled smoke key creation through `/key-policy-plus/api/keys/create` returns `200` and persists safe settings. + - Public `cpa.konbakuyomu.us` returns `404` for Plus resource, management, API alias, and `usage-admin` paths. + - Root disk ended at 368 MB free; no Docker prune or official image pull was used. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/prd.md b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/prd.md new file mode 100644 index 0000000..f3c6566 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/prd.md @@ -0,0 +1,44 @@ +# CPA Key Policy Plus Admin Fixes + +## Goal + +Make `cpa-key-policy-plus` usable as the single admin surface for creating and maintaining `cpa_` user keys. The admin page must create keys reliably, save limits/prices/resets through the correct CPA management API path, and provide a model-selection experience comparable to the old Key Policy/CPAMP flow instead of hard-coded textarea editing. + +## Confirmed Facts + +- CPA plugin `ResourceRoute` is browser-navigable and GET-only in the current CPA host, so `POST`/`PUT` create/save/reset calls sent to `/v0/resource/plugins/...` cannot reach the plugin. +- `cpa-key-policy-plus` already registers management routes for key list/create/save/limits/reset and CodexCont config. +- The current Plus admin UI exposes model allowlists and prices through raw textarea/JSON fields, which makes model selection fragile and hides the registered CPA model list. +- CPA management exposes model-related sources through registered auth-file models and static model definitions; existing key configuration can provide a safe fallback union. + +## Requirements + +- Keep CPA, CPAMP, and the old Key Policy upstream source/images unchanged. +- Fix Plus admin writes by routing all mutating admin actions through a CPA management route or admin-proxy alias, not through GET-only resource routes. +- Preserve the HTML admin resource as a GET page, but make its API base choose the admin management alias first. +- Rework new-key creation so the admin can set name, enabled state, RPM, request concurrency, active Codex window limit, and initial allowed models. +- Show the generated full `cpa_` key exactly once after creation with clear copy guidance; do not store or display raw keys later. +- Replace raw model and price textareas with structured UI: + - model count / price coverage in the main table, + - searchable model selector, + - select visible, clear, and selected-chip interactions, + - per-model input/output/cache-read/cache-write prices in USD per 1M tokens. +- Discover models from CPA management data where possible and fall back to configured key models without deleting unknown existing entries. +- Never return or render OAuth tokens, management keys, raw API keys, full key hashes, Authorization headers, cookies, request bodies, response bodies, or encrypted reasoning content. + +## Acceptance Criteria + +- [ ] Creating a key from the Plus admin page succeeds and the new key appears after reload. +- [ ] Saving limits, model allowlists, prices, and reset watermarks succeeds through the management API path. +- [ ] The admin UI no longer depends on `POST`/`PUT` calls to `/v0/resource/plugins/cpa-key-policy-plus/...`. +- [ ] The model picker can search, select all visible, clear, preserve unknown configured models, and show price coverage. +- [ ] Existing keys with manually configured or no-longer-discovered models are not silently stripped on save. +- [ ] Go tests cover management writes, model normalization/fallback, and admin HTML transport expectations. +- [ ] JS syntax check, `go test ./...`, and `git diff --check` pass locally. + +## Out Of Scope + +- Switching the production `/v1/responses` execution chain. +- Changing CPA/CPAMP/old Key Policy upstream source or official images. +- Retiring `usage-admin` or redesigning the public `cpa-usage` user page in this task. +- Recomputing historical usage or changing CodexCont/Governor behavior. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/task.json b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/task.json new file mode 100644 index 0000000..eda32b4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-admin-fixes/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-key-policy-plus-admin-fixes", + "name": "cpa-key-policy-plus-admin-fixes", + "title": "CPA Key Policy Plus admin fixes", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/design.md b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/design.md new file mode 100644 index 0000000..fa8a789 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/design.md @@ -0,0 +1,87 @@ +# CPA Key Policy Plus design + +## Architecture + +`cpa-key-policy-plus` is a self-owned Go CPA plugin. It replaces the old +Key Policy plugin as the exclusive frontend auth provider for ordinary `cpa_` +keys and owns the SQLite state for key settings, usage events, reset watermarks, +active request slots, active Codex sessions, and audit records. + +The intended final request chain is: + +`Client -> CPA -> cpa-key-policy-plus auth checks -> cpa-governor/CodexCont Engine routing -> CPA upstream execution`. + +Governor remains responsible for CodexCont Engine status and execution routing. +Plus owns key policy decisions and user/admin usage views. + +Current implementation note: CodexCont's production 516/518n-2 folding still +lives in the Python sidecar `/v1/responses` path. The existing Engine API only +returns safe status summaries. Therefore Plus can be deployed now as the user +key and quota authority, but the public `/v1/responses` Caddy route must not be +cut from the known-good CodexCont sidecar path to CPA-first until Governor has +an executor-level continuation supervisor that is verified with a real test key. + +## Data Model + +Plus stores: + +- `keys`: id, name, key hash, safe preview, enabled flag, RPM, request + concurrency, max active sessions, model allowlist, model prices, and + 5H/24H/7D/month USD limits. +- `usage_events`: safe request metadata, usage counters, cost breakdown, + failure summary, and key identity. +- `reset_watermarks`: per-key, per-window soft reset points. +- `active_requests`: per-key request-concurrency slots with stale TTL cleanup. +- `active_sessions`: per-key Codex window/session identifiers, source, first + seen, last seen, and missing-signal warnings. +- `audit_logs`: admin actions, imports, resets, and policy rejections. + +No table stores raw API keys, Authorization headers, request bodies, response +bodies, OAuth tokens, cookies, or encrypted reasoning content. + +## Enforcement + +Frontend auth checks run in this order: + +1. Extract and verify Bearer `cpa_` key by hash. +2. Reject disabled key or disallowed requested model. +3. Enforce RPM. +4. Enforce quota windows using reset watermarks. +5. Acquire request-concurrency slot. +6. Extract Codex session/window identity and enforce max active sessions. + +Request-concurrency slots are released by usage records when possible and have a +stale TTL fallback. Active sessions expire after 30 idle minutes. Missing session +identity is allowed in v1 and audited as `missing_session_identity`. + +## Interfaces + +Plugin registration exposes: + +- Admin resource: `/v0/resource/plugins/cpa-key-policy-plus/admin` +- User resource: `/v0/resource/plugins/cpa-key-policy-plus/user` +- User host compatibility for `cpa-usage.konbakuyomu.us` +- Management/API routes for key listing, saves, resets, usage, active sessions, + audits, and user session/login APIs. + +Plus should keep user-facing API shapes close to the current Governor user +surface so `cpa-usage` can be migrated without surprising users. + +## Migration And Rollback + +Migration imports old Key Policy state into Plus by key hash. Existing +usage-admin and Governor 5H/month limits and reset watermarks may be imported +from their SQLite files if configured, but Plus becomes the source of truth after +cutover. + +Deployment must back up CPA config, old Key Policy state, Governor SQLite, Plus +SQLite, and Caddy/admin proxy configs. Rollback means restoring the old plugin +binary/config, re-enabling old Key Policy, restoring previous routing, and +removing/ignoring Plus exclusive auth. + +## Route Boundaries + +`cpa.konbakuyomu.us` exposes only API paths needed by users. Admin/plugin/user +resources remain blocked there. `cpa-admin.konbakuyomu.us` exposes admin plugin +pages behind existing protection. `cpa-usage.konbakuyomu.us` exposes only the +ordinary user page and APIs. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/implement.md b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/implement.md new file mode 100644 index 0000000..13e28da --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/implement.md @@ -0,0 +1,110 @@ +# CPA Key Policy Plus implementation plan + +1. Read backend specs for CodexCont/Governor/user portal and current plugin + patterns. +2. Create `cpa_key_policy_plus_plugin/go` using the current Governor plugin + patterns but with independent metadata, config, store, policy engine, and UI. +3. Implement key import from old Key Policy-compatible JSON and optional + Governor local limits/resets. +4. Implement policy enforcement: key auth, model allowlist, RPM, quota windows, + request concurrency, active session count, and missing-session audit. +5. Implement usage recording and cost breakdown with safe event projection. +6. Implement admin and user resources with compact CPAMP-like dark UI. +7. Wire user host compatibility so `cpa-usage.konbakuyomu.us` serves Plus user + APIs instead of the old Governor/usage-admin source. +8. Keep the existing public `/v1/responses -> CodexCont sidecar -> CPA` route + until Governor has a verified executor-level continuation supervisor. Do not + switch Caddy to CPA-first in this task if that would bypass folding. +9. Adjust Governor only as needed so it no longer acts as the long-term key and + quota authority. +10. Add tests for import, auth, limits, reset watermarks, session extraction, + concurrency, user isolation, and UI resources. +11. Build linux/amd64 plugin artifact and prepare deployment notes/scripts with + backups and rollback steps. + +## Validation + +- `go test ./...` in the new plugin module. +- Existing `go test ./...` in `cpa_governor_plugin/go`. +- `python tests/test_middleware.py` +- `python tests/test_cpa_usage_portal.py` +- `python -m compileall middleware cpa_usage_portal run.py run_usage_portal.py` +- `git diff --check` +- Playwright desktop/mobile smoke for Plus admin, Plus user, and Governor status. + +## Production Checks + +- Back up CPA config, old Key Policy state, Governor DB, Caddy/admin proxy, and + old plugin binaries before cutover. +- Verify valid current `cpa_` keys import and authenticate. +- Verify an over-limit test key is rejected before CodexCont/upstream. +- Verify `usage-admin` no longer serves stale limit controls. +- Verify public API host still blocks admin/plugin/user paths. +- Verify the known-good CodexCont folding path still handles real + `/v1/responses` traffic after Plus is enabled. + +## Execution Evidence + +- Local checks passed on 2026-07-02: + - `go test ./...` in `cpa_key_policy_plus_plugin/go`. + - `go test ./...` in `cpa_governor_plugin/go`. + - `.venv\Scripts\python.exe tests\test_middleware.py` -> `163/163`. + - `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` -> `84/84`. + - `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py`. + - JS syntax check for embedded `admin.html` and `user.html` script blocks. + - `git diff --check`. +- Built artifact: + - `cpa_key_policy_plus_plugin/dist/linux/amd64/cpa-key-policy-plus.so` + - SHA256 `B46D205C9803A8A59CAC40B92AC89385D41AF1814E55CF05F76820CE14CBA730`. +- SJC backup before cutover: + - `/opt/codex-stacks/backups/cpa-key-policy-plus-20260702-192608`. +- Production deployment evidence: + - Remote plugin SHA256 matches the local artifact. + - Root disk before/after verification remained tight but usable: `/dev/sda1` + about `9.6G`, `9.2G` used, about `384M` available. No Docker prune or + image pull was used. + - `cpa` was restarted once; `caddy-edge` was reloaded; `cpa-admin-proxy` + was restarted because its Caddy admin API was not listening for reload. + - CPA logs show `plugin_id=cpa-key-policy-plus` loaded from + `/CLIProxyAPI/plugins/linux/amd64/cpa-key-policy-plus.so` at the cutover + time, along with Governor. Old `cpa-key-policy` remains on disk but is + disabled in config and was not loaded in the post-cutover log block. + - Plus SQLite exists at + `/opt/codex-stacks/cpa/plugin-state/cpa-key-policy-plus/policyplus.sqlite` + with imported rows: `keys=3`, `usage_events=4`, + `reset_watermarks=8`, `audit_log=28`. + - `https://cpa-usage.konbakuyomu.us/` returns `200` and `Cache-Control: + no-store`. + - Public blocked paths return `404`: + `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-key-policy-plus/user`, + `/usage-admin/`, and `/codexcont/`. + - Admin-proxy backend routes: `/key-policy-plus/` returns `200`, + `/key-policy-plus/api/keys` returns `3` keys, and legacy + `/usage-admin/` plus `/codexcont/` return `404`. + - Kuma test key login on `cpa-usage` returns `200`, identifies the safe + preview/name, and `/user/api/usage`, `/user/api/events`, and + `/user/api/codexcont` all return `200`. + - Authenticated `/v1/models` returns `200` and `7` models. + - A tiny authenticated `/v1/responses` request with `gpt-5.4-mini` returns + `200`, `status=completed`, proving the existing public sidecar route still + works after Plus is enabled. + - Direct CPA fake-model rejection currently surfaces as CPA's generic + `401 Missing API key` because Plus `frontendAuth` returns + unauthenticated for policy denial. This is accepted as the current CPA + wrapper, not as a user-facing ideal error shape. +- Playwright production smoke: + - `cpa-usage` opens as `CPA 用量自助页`. + - Kuma test key login renders the dashboard with realtime chip, active + count, limits, usage, tokens, cache, and reasoning metrics. + - `思维链保护` tab switches successfully and shows newest-first rows + (`2026/7/2 19:43:56` above `2026/7/2 19:43:47` in the smoke run). + - At `390px` viewport, page width remains `390`; dense table overflow is + contained inside `.table-wrap`. + +## Cutover Note + +This task intentionally did not switch public `/v1/responses` to CPA-first. +The verified production folding owner is still the Python CodexCont sidecar. +Governor/Plus can become the front-of-CodexCont hard gate only after an +executor-level continuation supervisor is implemented and tested with real +folding traffic. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/prd.md b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/prd.md new file mode 100644 index 0000000..05f86a2 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/prd.md @@ -0,0 +1,68 @@ +# CPA Key Policy Plus unified control + +## Goal + +Create a self-owned CPA plugin, `cpa-key-policy-plus`, that replaces the old +`cpa-key-policy` as the single authority for user `cpa_` keys, per-key limits, +quota resets, usage visibility, and pre-upstream request enforcement. The final +operator experience should remove the legacy `usage-admin` split and keep +ordinary user self-service only at `https://cpa-usage.konbakuyomu.us/`. + +## Requirements + +- Provide one administrator plugin page for every per-key setting: enabled + state, name, model allowlist, model prices, RPM, request concurrency, Codex + active window/session count, 5H/24H/7D/month USD limits, and soft reset. +- Preserve current `cpa_` keys by importing the existing Key Policy state by + hash/name/preview/RPM/model/price/daily/weekly data; users must not need a + new key solely because of this migration. +- Enforce limits before upstream model execution: disabled key, disallowed + model, RPM, request concurrency, active Codex window count, and quota + exhaustion must be rejected before model execution. During the migration + window, the current public CodexCont sidecar may still receive the request + first so the verified 516/518n-2 protection path is not lost; full + before-CodexCont rejection waits for Governor executor-level folding. +- Implement Codex window limiting as active session tracking, not RPM: + same key plus same extracted window/session identifier counts as one active + window, refreshes last-seen time, and expires after 30 idle minutes. +- If a request lacks a usable window/session identifier, do not reject in v1; + allow it under normal request-concurrency/quota checks and record a visible + warning/audit signal. +- Keep 5H/24H/7D rolling windows and Asia/Shanghai natural-month limits. + Reset must be soft watermarks and must not delete historical usage events. +- Keep CPA, CPAMP, and existing official/third-party images/source untouched. + Own code may add a new plugin and adjust own Governor/Caddy deployment. +- `cpa-usage.konbakuyomu.us` remains the only ordinary-user self-service + surface and must read the new plugin authority, not the legacy usage-admin + sidecar. +- Old `cpa-admin.konbakuyomu.us/usage-admin/` must be retired after migration + by returning 404 or redirecting to the new administrator plugin page. + +## Acceptance Criteria + +- [ ] `cpa-key-policy-plus` registers in CPA with admin and user resource + routes, exclusive frontend auth, model routing/execution integration, and + usage recording. +- [ ] Existing Key Policy keys import into Plus and can authenticate without + re-issuing keys. +- [ ] Administrator UI can view/edit/save all per-key settings and reset any + supported quota window for a selected key. +- [ ] User UI at `https://cpa-usage.konbakuyomu.us/` can log in with a valid + `cpa_` key and only see that key's limits, usage, events, and CodexCont + summaries. +- [ ] Disabled/disallowed/over-RPM/over-concurrency/over-window/over-quota + requests are rejected before upstream execution in the deployed-safe + migration mode. +- [ ] Session extraction follows the agreed priority: + `X-Codex-Window-Id`, `client_metadata.x-codex-window-id`, + `X-Codex-Turn-Metadata.window_id/prompt_cache_key`, body + `prompt_cache_key`, `Session_id`/`X-Session-ID`, then + `conversation_id`. +- [ ] Active Codex window counts expire after 30 idle minutes and repeated + requests in the same session do not consume extra window slots. +- [ ] Missing window/session signals are allowed but audited and visible. +- [ ] Legacy `usage-admin` is no longer the place to set 5H/month limits. +- [ ] Public `cpa.konbakuyomu.us` does not expose plugin/admin/user resources. +- [ ] Public `/v1/responses` is not cut to CPA-first until Governor owns a + verified executor-level continuation path; otherwise the current + CodexCont sidecar route remains in place. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/task.json b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/task.json new file mode 100644 index 0000000..57bfbcd --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-key-policy-plus-unified-control/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-key-policy-plus-unified-control", + "name": "cpa-key-policy-plus-unified-control", + "title": "CPA Key Policy Plus unified control", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/design.md b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/design.md new file mode 100644 index 0000000..bb3bc55 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/design.md @@ -0,0 +1,175 @@ +# Design + +## Boundaries + +This task changes only the self-owned `cpa-key-policy-plus` plugin and narrow proxy routing when needed. + +- Plugin backend: `cpa_key_policy_plus_plugin/go/main.go` +- Plugin store/models: `cpa_key_policy_plus_plugin/go/internal/policyplus/**` +- Plugin UI: `cpa_key_policy_plus_plugin/go/assets/admin.html`, `user.html`, `shared.css` +- Tests: `cpa_key_policy_plus_plugin/go/**/*_test.go` +- Optional admin/public proxy routing if a path bug is proven + +Do not modify CPA, CPAMP, old Key Policy, or their images/source. + +## Current Problems By Layer + +### Public `cpa-usage` + +The public route itself is alive. Caddy rewrites `/` to the Plus user resource and the resource API returns valid JSON status responses. The observed `连接异常 / 同步失败` state is therefore more likely a UI/session/retry issue than a dead proxy route. + +Current frontend behavior couples several fetches into one refresh: + +- `/me` +- `/usage` +- `/events` +- `/codexcont` + +Any one failure puts the topbar into `连接异常`. The UI does not visibly classify `401 not_authenticated` as session expiry, and a bad poll can leave the user with an empty table and generic sync failure. + +Design response: + +- Treat `401 not_authenticated` as `会话已过期，请重新登录`, not generic network failure. +- Keep last known good data when a non-auth poll fails. +- Split refresh state into per-domain health: + - session/account + - usage/events + - CodexCont protection +- The topbar can show an aggregate state, but the content area must show which section failed. +- Manual refresh must cancel older requests, start a new sequence, and ensure stale failed responses cannot overwrite a newer successful snapshot. +- Page visibility restore should always trigger a fresh snapshot if logged in. + +### Admin `CPA Key Policy+` + +The current admin table is doing too much in one row. It has no visual difference between "configured value" and "actual usage", and the row overflows into ellipsis artifacts. + +Design response: + +Use a master/detail layout. + +Desktop: + +- Left/main list: one row per key, compact and mostly read-only. +- Right detail drawer/panel: editable settings for the selected key. + +Mobile: + +- Key list becomes stacked rows. +- Detail editor opens as a full-width panel/modal. + +Key list columns: + +- Key: name, safe preview, status chip. +- Health: enabled/disabled/archived, active sessions. +- Limits: 5H/24H/7D/month mini progress summaries. +- Controls: RPM, request concurrency, Codex windows as compact text, not inline inputs. +- Models: count and price coverage. +- Usage: selected range cost and 24H quick hint. +- Actions: edit, reset, archive. + +Detail editor sections: + +- Basic settings: name, enabled, RPM, request concurrency, Codex active windows. +- Quota windows: 5H/24H/7D/month each shows `used / limit / remaining` and an editable limit field. +- Models and prices: reuse the existing model editor modal and structured price table. +- Usage reset: soft reset per window. +- Lifecycle: archive/restore; optional hard delete in danger zone depending on product decision. + +### Key Lifecycle + +Hard deleting a key row is simple but risky because usage events, Codex summaries, reset watermarks, active sessions, and audit logs may reference the key ID. The better default is to add soft archival. + +Recommended store extension: + +- Add nullable-ish columns to `keys`: + - `archived integer default 0` + - `archived_at integer default 0` +- `ListKeys` returns all keys for admin by default, with UI filter `显示归档`. +- Frontend auth treats `archived` like disabled: cannot authenticate/use. +- User portal cannot log in with archived keys. +- Usage/history remains visible to admin. + +Optional hard delete: + +- Implement as advanced route only after archive exists. +- Either reject if usage exists, or require deleting only the key row while preserving historical events with safe preview. The safer v1 is "reject if usage exists". + +## API Changes + +Admin management routes: + +- `GET /plugins/cpa-key-policy-plus/keys` + - return keys with `usage`, `limits`, `active_sessions`, `archived`, and `remaining` projections. +- `PUT /plugins/cpa-key-policy-plus/keys/save` + - accept `archived` if lifecycle is included. +- `POST /plugins/cpa-key-policy-plus/keys/archive` + - `{ id, archived: true|false }` +- Optional: + - `DELETE /plugins/cpa-key-policy-plus/keys` + - `{ id, confirm }`, reject when usage/history exists unless explicitly allowed by later decision. + +User resource API: + +- Existing paths stay stable: + - `/user/api/session` + - `/user/api/me` + - `/user/api/usage` + - `/user/api/events` + - `/user/api/codexcont` +- Error responses should include safe `error`, `message`, and a category usable by frontend: + - `auth` + - `network` + - `usage` + - `codexcont` + - `unknown` + +## UI Preview Plan + +Create local previews with seeded data before production deploy. + +Variant A, recommended: + +- CPAMP-like key list plus right detail drawer. +- Best for many keys and dense operator work. + +Variant B: + +- Key cards plus full-width detail panel below selected card. +- More readable for small key counts, less dense for admin use. + +Variant C: + +- Two-level table: collapsed key rows with expandable editor rows. +- Closest to current table, but still risks cramped layouts and row height jumps. + +Recommendation: implement Variant A, with mobile fallback behaving like Variant B. + +Preview mechanics: + +- Extend existing preview mode or add a small dev-only preview fixture endpoint. +- Seed several keys: + - enabled normal key with meaningful usage + - disabled key + - archived key + - key with no limit + - key with incomplete prices +- Seed usage events and CodexCont summaries so progress bars and details are visible. + +## Compatibility + +- Existing keys must remain usable. +- Existing SQLite DB must migrate forward with `alter table` column checks only. +- Existing `cpa_` sessions remain valid unless the session secret changed or the key hash/ID changed. +- Existing old records without new lifecycle fields display as active/non-archived. +- Current model discovery behavior remains: + - CPA/host model hints first + - Plus configured models as fallback + - unknown selected models preserved + +## Operational Notes + +- The old `cpa-usage-portal` container is still running but not on the current public route. After this task verifies Plus user page fully replaces it, we can plan a separate stop/retire action. Do not delete its files in this task. +- Server deploy should back up the current `.so`, Plus SQLite DB, CPA config, and proxy config. +- Do not run Docker prune. +- Do not batch-delete files/directories. + diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/implement.md b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/implement.md new file mode 100644 index 0000000..e44b7df --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/implement.md @@ -0,0 +1,203 @@ +# Implementation Plan + +## Phase 0: Planning Gate + +- [ ] Review this PRD/design/implementation plan with the user. +- [ ] Resolve the key lifecycle decision: + - recommended: archive/restore in v1, hard delete only as constrained danger action. +- [ ] Do not run `task.py start` until the user approves implementation. + +## Phase 1: Evidence And Reproduction + +- [ ] Add or run local Plus preview with seeded admin/user data. +- [ ] Use Playwright to capture current admin/user screenshots: + - desktop + - 390px mobile +- [ ] Reproduce public `cpa-usage` stuck state locally by mocking: + - `/codexcont` failure while `/usage` succeeds + - `401 not_authenticated` + - slow request followed by newer successful request +- [ ] Record findings in this task before changing code. + +## Phase 2: Backend Support + +- [ ] Add key lifecycle fields to `KeyRecord` and SQLite schema: + - `archived` + - `archived_at` +- [ ] Ensure `FrontendAuthProvider` rejects archived keys with a clear safe message. +- [ ] Add store methods: + - `SetArchived(ctx, id, archived, at)` + - optional `DeleteKeyIfUnused(ctx, id)` if hard delete is approved. +- [ ] Enrich admin key projection: + - used values for 5H/24H/7D/month + - limit values + - remaining values when limit exists + - active sessions + - price coverage + - archived state +- [ ] Add admin route(s): + - `/plugins/cpa-key-policy-plus/keys/archive` + - matching `/key-policy-plus/api/keys/archive` alias + - optional delete route only if approved. +- [ ] Keep raw key/hash/body/cookie data out of all responses. + +## Phase 3: Public User Page Stability + +- [ ] Refactor refresh pipeline: + - keep last good data + - isolate auth errors from section errors + - never let stale failed requests overwrite newer snapshots + - manual refresh always clears stuck `syncing` +- [ ] Show clear state messages: + - `实时已连接` + - `部分数据同步失败` + - `会话已过期` + - `正在同步` +- [ ] Add section-level fallback cards for usage/protection failures. +- [ ] Preserve current sorting newest-first and stale processing filtering. + +## Phase 4: Admin UI Redesign + +- [ ] Replace the main inline edit table with master/detail layout. +- [ ] Key list shows safe summary fields only. +- [ ] Detail editor shows editable settings and quota progress. +- [ ] Reuse existing model/price editor with layout polish. +- [ ] Add archive/restore action and confirmation. +- [ ] Add optional hard delete danger action if approved. +- [ ] Remove meaningless overflow/ellipsis artifacts. +- [ ] Keep CPAMP-like dark style and avoid flashy effects. + +## Phase 5: Local Preview Variants + +- [ ] Produce preview Variant A: list + right detail drawer. +- [ ] Produce preview Variant B: cards + full-width detail panel. +- [ ] If needed, produce Variant C: expandable key rows. +- [ ] Capture desktop and 390px screenshots. +- [ ] Compare with user before final server deployment if the user wants to choose visually. + +## Phase 6: Validation + +Run from `cpa_key_policy_plus_plugin/go`: + +```powershell +go test ./... +``` + +Run JS checks from repo root by extracting inline scripts or using existing helper pattern: + +```powershell +node --check <extracted-admin-script.js> +node --check <extracted-user-script.js> +``` + +Run repo checks: + +```powershell +git diff --check +``` + +Playwright checks: + +- admin desktop layout: no horizontal body overflow, no short-field vertical text +- admin 390px layout: detail editor usable +- user page: login state, refresh recovery, session-expired message +- mocked delayed/failing endpoint: page does not stick in `同步中` +- newest-first event/protection ordering remains correct + +## Phase 7: Server Deployment + +- [ ] Check disk space. +- [ ] Back up: + - current `cpa-key-policy-plus.so` + - Plus SQLite DB + - CPA config + - Caddy/admin proxy config +- [ ] Build linux/amd64 `.so` and record SHA256. +- [ ] Upload only the new `.so` and proxy config if changed. +- [ ] Restart only necessary services. +- [ ] Verify: + - `https://cpa-usage.konbakuyomu.us/` login with test key + - user page realtime polling recovers after tab hidden/visible + - `CPA Key Policy+` admin layout and lifecycle actions work + - `cpa.konbakuyomu.us` still blocks management/plugin/resource paths + - old Python `usage-admin` is not required by the public route + +## Risk And Rollback + +- Risk: DB migration adds columns incorrectly. + - Rollback: restore Plus SQLite DB backup and previous `.so`. +- Risk: frontend auth accidentally allows archived keys. + - Mitigation: backend tests for archived key rejection. +- Risk: public page session handling regresses. + - Mitigation: tests for valid session, expired session, and stale request race. +- Risk: hard delete breaks historical usage joins. + - Mitigation: prefer archive-first; reject hard delete when usage exists if implemented. + +## Execution Evidence + +- Implemented the confirmed lifecycle policy: Key Policy+ now supports + archive/restore instead of hard delete. Archived keys are hidden by default, + cannot log in to the user page, and cannot authenticate frontend requests. +- Migrated Plus SQLite schema with `keys.archived` and `keys.archived_at`, and + preserved archive state across legacy state imports so old Key Policy sync + cannot accidentally unarchive a Plus-managed key. +- Rebuilt the admin UI as a master/detail page: + - key list shows safe preview, status, active sessions, RPM, request + concurrency, Codex window limit, four quota windows, model count, and price + coverage; + - right detail panel edits basic policy, four quotas, model/price modal, + soft reset, and archive/restore; + - mobile layout keeps the dense table inside its scroll container and shows + the detail editor as a full-width section. +- Stabilized the user self-service page refresh behavior: + - auth errors return the user to login with `会话已过期`; + - usage/CodexCont partial failures keep last good data and show section-level + notices; + - old refreshes are aborted/ignored, so manual refresh and foreground resume + no longer leave the page stuck in `同步中`. +- Built linux/amd64 plugin artifact with WSL Go 1.22.6: + - `cpa_key_policy_plus_plugin/dist/linux/amd64/cpa-key-policy-plus.so` + - SHA256 `980d61ee7d2753bd9caa48036e804c02060e3a4515ed4b7af2d66ddfaa203b7c` + - `file`: ELF 64-bit x86-64 shared object. +- SJC deployment: + - backup path: + `/opt/codex-stacks/backups/cpa-key-policy-plus-ux-fixes-20260702-225009` + - uploaded only the new `.so`; + - restarted only `cpa`; + - did not pull images and did not run Docker prune. +- Production verification: + - CPA logs show `plugin_id=cpa-key-policy-plus` loaded and registered from + `/CLIProxyAPI/plugins/linux/amd64/cpa-key-policy-plus.so`; + - remote plugin SHA256 matches local artifact; + - admin API `/key-policy-plus/api/keys` returns `200`, `5` keys, and every + key projection includes `archived` and `quota`; + - admin API `/key-policy-plus/api/models` returns `200` with `7` models and + no warnings; + - archive/restore smoke on existing disabled smoke key succeeded and restored + the key to its prior unarchived state; + - Kuma test key login on `https://cpa-usage.konbakuyomu.us/` returns `200`; + `/me`, `/usage`, `/events`, and `/codexcont` all return `200`; + - public `https://cpa.konbakuyomu.us` still returns `404` for Plus resource, + Plus API alias, management, and `usage-admin` paths; + - root disk ended at about `344M` free after removing the single uploaded + `/tmp` artifact. + +## Validation Results + +- `go test ./...` in `cpa_key_policy_plus_plugin/go`: passed. +- Inline JS syntax checks for `assets/admin.html` and `assets/user.html`: + passed. +- `git diff --check`: passed, with only existing CRLF conversion warnings. +- Local Playwright preview: + - admin desktop `1600x1000`: no page overflow, master/detail usable, archive + toggle shows hidden rows, no raw `...` truncation; + - admin mobile `390x900`: no page overflow, detail editor usable, dense table + scroll contained; + - user desktop and mobile: login works, partial `/codexcont` failure keeps + app visible, recovery clears notice, mocked `401` returns to login. +- Production Playwright: + - `https://cpa-usage.konbakuyomu.us/` desktop and `390px` mobile both log in + with the Kuma test key; + - metrics render, `思维链保护` tab switches, no page-level overflow; + - manual refresh returns to `实时已连接` and does not remain stuck in + `同步中`. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/prd.md b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/prd.md new file mode 100644 index 0000000..c7aacb3 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/prd.md @@ -0,0 +1,86 @@ +# CPA Usage and Key Policy Plus UX fixes + +## Goal + +Make `CPA Key Policy+` and the public `CPA Usage` page understandable, stable, and pleasant enough for day-to-day operation. + +This task fixes the current rough edges without changing CPA, CPAMP, or any third-party plugin source/image. The source of truth stays our self-owned `cpa-key-policy-plus` plugin plus the existing admin/public proxy routing. + +## User Value + +- Admin can manage each `cpa_` key from one clear page without guessing what a cramped table cell means. +- Admin can see configured limits together with actual used/remaining quota before saving or resetting. +- Admin can remove unwanted keys through a safe lifecycle action. +- Ordinary users can open `https://cpa-usage.konbakuyomu.us/`, log in with their key, and see a stable realtime view instead of a stuck `连接异常 / 同步失败` state. +- Future UI changes can be previewed locally before production deploy. + +## Confirmed Facts + +- The current task exists at `.trellis/tasks/07-02-cpa-usage-key-policy-plus-ux-fixes/` and is still in `planning`. +- `cpa-usage.konbakuyomu.us` is routed by Caddy to CPA, not the old Python portal: `/` rewrites to `/v0/resource/plugins/cpa-key-policy-plus/user`, and `/v0/resource/plugins/cpa-key-policy-plus/user*` proxies to `cpa:8317`. +- The public user resource is reachable: `/v0/resource/plugins/cpa-key-policy-plus/user/api/me` returns `401 not_authenticated` without a session, and `/user/api/session` returns a structured invalid-key error when probed with a fake key. That points to a frontend/session/retry/data-state bug, not a dead domain. +- The old `cpa-usage-portal` container is still running on the server, but the current public Caddy route does not use it. This is a source of operational confusion. +- `CPA Key Policy+` admin UI is currently one wide inline-edit table. It mixes name, enabled state, RPM, request concurrency, Codex windows, four quota inputs, model/price summary, 24H usage, and reset buttons into each row. +- The `...` visible in the current admin screenshot is not a meaningful field; it is an overflow/truncation symptom from cramped columns. +- Admin key rows already receive backend usage windows from `usageWindows(...)`, but the UI only exposes a small `24H 用量` cell and does not clearly show 5H/24H/7D/month used/limit/remaining together. +- The store has hard deletion behavior only for stale legacy sync (`delete from keys where id=?`) but there is no user-facing admin route or UI for key deletion/archive. +- The user page already has stale `处理中` filtering logic and refresh abort logic, but the screenshot still shows a stuck `连接异常 / 同步失败` state; error handling and state recovery need to be made explicit and testable. + +## Requirements + +- Keep changes limited to self-owned code: + - `cpa_key_policy_plus_plugin/go/**` + - necessary admin/public proxy routing + - Trellis/task docs and tests +- Do not modify CPA, CPAMP, or old `cpa-key-policy` official source/images. +- Diagnose and fix the public `cpa-usage` realtime failure mode: + - user page must distinguish unauthenticated/session-expired from network/API failure + - a failed poll must not leave the page permanently stuck in `同步中` or `连接异常` + - manual refresh and page visibility restore must recover without full browser reload when the session is still valid +- Redesign `CPA Key Policy+` admin page around clear hierarchy rather than one huge inline table: + - overview cards + - key list + - selected-key detail editor/drawer/panel + - actual used/limit/remaining quota display for 5H/24H/7D/month + - active session/window count + - model/price coverage +- Preserve or improve existing create/save/model selection behavior: + - creating a key still shows the full `cpa_` key exactly once + - model list stays searchable and supports unknown model preservation + - price editing remains structured by model +- Add safe key lifecycle management: + - default should be `停用 + 归档/隐藏` so history and audit remain intact + - hard delete, if provided, must be clearly dangerous and constrained +- Keep user-facing data safe: + - never expose raw keys, full hashes, Authorization/cookie values, request/response bodies, or encrypted reasoning content +- Provide local preview variants before finalizing the UI: + - at minimum, preview the recommended layout and one alternative layout with seeded data + - screenshots should cover desktop and narrow/mobile widths + +## Acceptance Criteria + +- [ ] `CPA Key Policy+` admin no longer renders the single overcrowded inline table as the main editing surface. +- [ ] Admin can see for each key: name/preview/status, RPM, request concurrency, Codex window limit, active sessions, 5H/24H/7D/month used/limit/remaining, model count, and price coverage. +- [ ] Admin can edit a selected key without horizontal overflow or meaningless `...` cells on desktop. +- [ ] Admin can safely hide/archive or remove an unwanted key according to the chosen lifecycle decision. +- [ ] Public `cpa-usage` login and refresh states are testable and recover from failed polls without requiring a full page reload when the session remains valid. +- [ ] `cpa-usage` shows useful failure text for session-expired vs API/network failure. +- [ ] Existing model discovery, model selection, price editing, key creation, save, reset, usage events, and CodexCont summaries continue to work. +- [ ] Local preview(s) are runnable and screenshots are produced for desktop and 390px mobile. +- [ ] `go test ./...` passes in `cpa_key_policy_plus_plugin/go`. +- [ ] Inline JS syntax checks pass for changed HTML assets. +- [ ] Playwright verifies admin and user pages for layout, refresh recovery, and no short-field vertical text. +- [ ] Server deploy preserves public/admin boundaries: `cpa.konbakuyomu.us` still blocks plugin/resource/management paths; `cpa-usage.konbakuyomu.us` exposes only the user page/API. + +## Out Of Scope + +- Rewriting CPA, CPAMP, or official Key Policy. +- Reintroducing the old Python `usage-admin` as the source of truth. +- Changing the production `/v1/responses` execution architecture. +- Deleting production data or CPAMP/CPA history. +- Bulk filesystem cleanup or Docker prune. + +## Open Questions + +- Key lifecycle policy: should the first implementation expose only `停用 + 归档隐藏`, or also expose hard delete in an advanced/danger zone? +- UI shape: should the admin page use a right-side detail drawer, a full-width detail panel under the selected key, or a card/list layout? Recommendation is right-side detail drawer on desktop and stacked detail panel on mobile. diff --git a/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/task.json b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/task.json new file mode 100644 index 0000000..ec183b7 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-cpa-usage-key-policy-plus-ux-fixes/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-usage-key-policy-plus-ux-fixes", + "name": "cpa-usage-key-policy-plus-ux-fixes", + "title": "CPA Usage and Key Policy Plus UX fixes", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/design.md b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/design.md new file mode 100644 index 0000000..a74e08c --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/design.md @@ -0,0 +1,90 @@ +# Governor CPAMP style alignment design + +## Architecture + +This is a presentation and ordering fix inside the self-owned Governor plugin. +The runtime architecture stays unchanged: + +- `cpa-governor` serves admin/user HTML through CPA plugin resources. +- `cpa-usage.konbakuyomu.us` exposes only the user resource. +- CPAMP remains the admin shell and visual reference. +- CodexCont remains the production protection data source. + +The shared visual system lives in the plugin's embedded assets. Both +`admin.html` and `user.html` consume the same `shared.css`, so style changes +should be made once through shared tokens/classes rather than one-off per page +overrides. + +## Visual Contract + +- Use a flatter CPAMP-like dark background and panel stack: + - no radial page glow, + - restrained panel shadow, + - dark slate panels, + - muted grey table headers. +- Keep the product-specific `U` / `G` marks, but make the rest of the surface + feel like CPAMP's operations UI. +- Remove high-motion effects: + - `.topbar::after` sweep and `liveSweep`, + - `.sync-button::after` sweep and `syncSweep`, + - `.metrics.cards-updated .metric` bump, + - `rowFresh` broad row animation. +- Preserve only lightweight status-dot pulse (`statusBlink` / `statusPing`) so + operators still see live state without a strong light strip. +- Keep table layout fixed with horizontal overflow on mobile. Do not shrink + dense tables until short fields wrap vertically. + +## Ordering Contract + +User CodexCont summaries can come from two sources: + +- live CodexCont `/admin/requests`, filtered by key identity; +- Governor local `codexcont_summaries` fallback. + +The user API should return a deterministic newest-first list regardless of +source. Sort by the displayed request time: + +1. `started_at` +2. `updated_at` +3. `ended_at` + +If parsing fails, keep that row behind rows with valid timestamps while +preserving stable fallback order. The frontend may also apply the same sort as +a defensive display guard, but the backend should own the API contract. + +## Active Request Contract + +The `活跃` chip is an operator-facing "how many requests are still processing +right now" indicator. It must not directly render CodexCont +`status.counters.active_requests`, because a broken SSE/admin communication +period can leave that counter high long after those rows stop being visible. + +Both admin and user pages derive active count from the current request list: + +- include only rows whose explicit status or protection is `processing`; +- exclude rows whose latest visible timestamp is older than a short stale + threshold; +- keep stale processing rows in the history table if returned by the API, but + do not count them as active. + +The timestamp priority for stale detection mirrors sorting: `updated_at`, +`started_at`, then `ended_at`. This keeps "currently being updated" rows alive +while preventing old abnormal rows from pinning `活跃` forever. + +## Compatibility + +- Keep HTML response cache behavior and refresh recovery logic unchanged. +- Keep existing class names where tests or JS rely on them, but change their + visual effect to CPAMP-like styling. +- Existing screenshot artifacts are not tracked; new Playwright artifacts stay + under ignored `artifacts/`. +- Deployment rollback is replacing the previous + `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-governor.so` from backup and + restarting `cpa`. + +## Security + +- Do not add or print raw keys, cookies, Authorization headers, OAuth tokens, + or encrypted reasoning content. +- User protection rows remain scoped to the logged-in Key Policy key. +- Public API host must keep blocking plugin/admin/governor/codexcont paths. diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/implement.md b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/implement.md new file mode 100644 index 0000000..b8bd0fe --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/implement.md @@ -0,0 +1,107 @@ +# Governor CPAMP style alignment implementation plan + +## Checklist + +1. Load applicable Trellis specs before editing. +2. Start this task with `task.py start`. +3. Update shared Governor CSS: + - flatten page background, + - tune CPAMP-like colors/borders/shadows, + - remove topbar sweep, refresh sweep, metric bump, and row broad highlight, + - keep small status-dot pulse. +4. Review `admin.html` and `user.html` class usage and adjust only if the + shared CSS cannot express the desired CPAMP-like layout. +5. Add a backend ordering helper for user CodexCont summaries and apply it to + both live and fallback sources. +6. Add shared frontend helpers for protection ordering and non-stale + `processing` active counts; use them in both admin and user pages. +7. Add/adjust Go tests: + - visual CSS no longer contains sweep/bump hooks, + - status-dot animation remains, + - user CodexCont response is newest-first. + - admin/user HTML derives active count from non-stale `processing` rows. +8. Run local validation: + - `go test ./...` in `cpa_governor_plugin/go`, + - extracted inline JS syntax check for `assets/user.html` and + `assets/admin.html`, + - `.venv\Scripts\python.exe tests\test_middleware.py`, + - `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py`, + - `git diff --check`. +9. Playwright local preview: + - desktop and 390px mobile for user/admin pages, + - assert no sweep/bump classes produce visible animation hooks, + - assert protection table first visible row is newer than the second row. +10. Build linux/amd64 `cpa-governor.so` and record SHA256. +11. Deploy to SJC: + - check disk, + - backup old plugin, + - upload only `.so`, + - restart only `cpa`, + - remove temp upload. +12. Production smoke: + - `cpa-usage` login with the known test key, + - verify visual hook presence/absence via HTML and Playwright, + - verify protection newest-first, + - verify `活跃` equals current non-stale `processing` rows on admin and user + pages, + - verify CPAMP sidebar `CPA Governor`, + - verify public API route boundaries. +13. Update task evidence/specs as needed, commit, archive. + +## Risk Points + +- CPAMP visual parity is subjective. Use the provided screenshots as the target: + restrained dark panels, muted table headers, and no bright custom sweep. +- CodexCont live API may already order rows differently than local fallback. + The Governor user API should normalize order after filtering. +- SJC disk is tight. Do not pull images or rebuild containers for this plugin + asset-only change. + +## Validation Evidence + +- Started task with Trellis and loaded backend/guides specs before editing. +- Local tests: + - `go test ./...` in `cpa_governor_plugin/go` passed. + - Extracted `assets/user.html` and `assets/admin.html` scripts and ran + `node --check --input-type=commonjs -` for both; both passed. + - `.venv\Scripts\python.exe tests\test_middleware.py` passed `163/163`. + - `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` passed `84/84`. + - `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py` passed. + - `git diff --check` passed, with only expected CRLF warnings. +- Playwright local preview: + - Saved screenshots under ignored `artifacts/`: + `governor-admin-desktop.png`, `governor-admin-mobile.png`, + `governor-user-protection-desktop.png`, + `governor-user-protection-mobile.png`. + - Preview deliberately reported backend `active_requests=17` while the + visible request list had one fresh `processing` row and one stale + `processing` row; admin and user pages both displayed `活跃 1`. + - DOM checks found no `syncSweep`, `liveSweep`, `metricBump`, `rowFresh`, + `.sync-button::after`, `.topbar::after`, or `cards-updated` hooks. + - DOM checks confirmed status dots still animate with `statusBlink`. + - User `思维链保护` order was newest-first: + `req-preview-live`, `req-preview-a`, `req-preview-stale`. +- Build: + - Built `cpa-governor.so` on WSL with Go `1.22.6` for linux/amd64. + - SHA256: `57a353a5f45cf22cb3d80bf666c1f39f5557f1cdf8c3cf28f0e0f4705d7a7887`. + - `file` reported ELF 64-bit x86-64 shared object. +- Server deployment: + - SJC root disk before deploy: `450M` free; after deploy: `433M` free. + - Backed up old plugin to + `/root/cpa-governor-cpamp-style-backups/20260702-165855/cpa-governor.so` + with SHA256 `6d8970fd0168efbb691bb8e322fcea380d4d0e311025a6d50b96c3a6c4bc82f1`. + - Uploaded only the new `.so`, restarted only `cpa`, and removed only the + single temporary upload file `/tmp/cpa-governor-cpamp-style.so`. + - CPA `/healthz` returned `{"status":"ok"}`. + - CPA logs showed `plugin loaded` and `plugin registered` for + `cpa-governor` from `/CLIProxyAPI/plugins/linux/amd64/cpa-governor.so`. + - Public `https://cpa.konbakuyomu.us/governor/`, + `/v0/resource/plugins/cpa-governor/admin`, + `/v0/resource/plugins/cpa-governor/user`, and `/codexcont/` all returned + `404`. + - `https://cpa-usage.konbakuyomu.us/` API login with Kuma test key returned + `200`, `/me` returned the expected safe preview, and `/codexcont?limit=20` + returned 20 protection records with active count 0 at the time of smoke. + - Server-local plugin HTML checks found no removed animation hooks and found + `statusBlink`, `activeProcessingCount`, and `PROCESSING_STALE_MS` in both + admin and user resources. diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/prd.md b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/prd.md new file mode 100644 index 0000000..293b423 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/prd.md @@ -0,0 +1,69 @@ +# Governor CPAMP style alignment + +## Goal + +Align the self-owned CPA Governor user/admin pages with the CPAMP admin +visual language and fix the user CodexCont protection table ordering. + +The user-facing result should make `https://cpa-usage.konbakuyomu.us/` feel +like part of the same operations product as CPAMP: restrained dark theme, +compact cards, stable tables, modest status feedback, and newest requests at +the top in every request-like table. + +## Requirements + +- Only modify self-owned `cpa-governor` plugin page assets and tests. +- Do not modify CPA, CPAMP, or CPA Key Policy official source, images, or + runtime data. +- Update the shared custom-page visual tokens to be closer to CPAMP: + darker flat background, subdued borders, compact panels, muted table header, + less saturated cyan, and simpler buttons/chips. +- Remove the current custom sweep/glow effects: page topbar sweep, refresh + button sweep, metric-card bump, and broad row highlight. +- Keep subtle live feedback through small status dots only. +- Keep existing refresh safety behavior: abort stale requests, recover after + background/idle, and keep latest refresh results authoritative. +- Make `额度与明细` and `思维链保护` use the same table, detail-card, chip, and + button style. +- Make user `思维链保护` sort newest requests first. The visible time order + should match `额度与明细` and CPAMP request monitoring. +- Make the top-right `活跃` count mean the number of currently visible, + non-stale `processing` protection requests, not a backend counter that can be + left high after abnormal communication. +- Add the same `活跃 N` status chip to the user self-service page so `CPA Usage` + and `CPA Governor` expose the same realtime state vocabulary. +- Treat old `processing` rows as stale so they do not keep the active count high + forever. Stale rows may remain in history, but they must not be counted as + active work. +- Keep the user page without a CPAMP sidebar so ordinary users do not see an + admin-shaped navigation surface. +- Preserve public/admin route boundaries after deployment. + +## Acceptance Criteria + +- [ ] `CPA Usage` and `CPA Governor` custom pages use one shared CPAMP-like + dark style and no longer show the current bright cyan sweep/glow theme. +- [ ] Refresh buttons still show `同步中`, `刚刚更新`, and `同步失败`, but without + sweep animation or broad glow. +- [ ] Small status dots still pulse lightly for connected/syncing/error states. +- [ ] `额度与明细` and `思维链保护` have matching table density, chip shape, + detail panels, and button style. +- [ ] User `思维链保护` rows are newest-first by request time. +- [ ] Admin `活跃` count equals current non-stale `processing` rows instead of + stale `status.counters.active_requests`. +- [ ] User page topbar shows `活跃 N` with the same chip style as the Governor + admin page. +- [ ] Desktop and 390px mobile layouts have no overlapping text and no short + fields forced vertical. +- [ ] Local Go tests and existing Python regressions pass. +- [ ] Playwright validates desktop/mobile visual shape and the protection row + order. +- [ ] SJC deployment uploads only the new Governor `.so`, restarts only `cpa`, + and keeps public `cpa.konbakuyomu.us` admin/plugin routes blocked. + +## Out of Scope + +- Pixel-perfect copying of CPAMP. +- Adding a CPAMP left sidebar to the ordinary user page. +- Changing CPAMP, CPA, or Key Policy official artifacts. +- Changing the production `/v1/responses` execution path. diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/task.json b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/task.json new file mode 100644 index 0000000..48422e8 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-cpamp-style-alignment/task.json @@ -0,0 +1,26 @@ +{ + "id": "governor-cpamp-style-alignment", + "name": "governor-cpamp-style-alignment", + "title": "Governor CPAMP style alignment", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/design.md b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/design.md new file mode 100644 index 0000000..23bbc6d --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/design.md @@ -0,0 +1,99 @@ +# Governor UI route cleanup design + +## Architecture + +The task is a presentation and routing cleanup, not a provider-path migration. +CPA remains the public API entrypoint, CPAMP remains the admin shell, Key Policy +remains the current key source, and CodexCont remains the proven production +continuation service. + +Governor keeps two browser resources: + +- `/v0/resource/plugins/cpa-governor/admin` renders the admin CodexCont status + dashboard. +- `/v0/resource/plugins/cpa-governor/user` renders the user self-service page. + It remains a routable resource for the dedicated `cpa-usage` host, but it no + longer registers a CPAMP sidebar menu. + +Governor no longer exposes daily-use mutable controls inside the admin browser +resource. Low-level plugin configuration stays in CPA plugin metadata and the +CPA plugin configuration drawer. + +## Data flow + +Admin CodexCont status: + +1. CPAMP loads the Governor admin resource. +2. The page fetches snapshots from `/governor/codexcont/admin/status` and + `/governor/codexcont/admin/requests`. +3. The page opens `EventSource('/governor/codexcont/admin/logs/stream')`. +4. Caddy rewrites those admin-only paths to CodexCont `/admin/*`. +5. Public `cpa.konbakuyomu.us` never exposes these routes. + +User page: + +1. User enters a full Key Policy `cpa_...` key. +2. Browser sends it in `X-CPA-Governor-Key` to the GET-only session route. +3. Governor resolves the key against the Key Policy mirror and stores only safe + session state. +4. User quota/details come from Governor's SQLite usage projection. +5. User CodexCont status is filtered by safe key identity where available. + +The user page is a single-key monitor inspired by CPAMP realtime monitoring, not +a copy of CPAMP's global monitoring center. Admin/global dimensions such as +account summaries, client-key summaries, provider/account filters, and global +key display controls are intentionally omitted. + +Usage-event projection owns the durable request-detail contract. CPA +`UsageRecord` fields such as requested model, provider, executor type, +reasoning effort, service tier, TTFT, and failure status code are stored as +optional columns so old rows remain readable. The browser renders these fields +when present and displays `-` for older records. + +## UI contracts + +- Use the existing CodexCont dashboard style as the source visual language: + compact dark topbar, status chips, dot pulse, bounded metric cards, stable + table widths, and expandable detail panels. +- No refresh-button rotation, diagonal badge, or full-screen marketing layout. +- Manual refresh is modeled as a small state machine (`syncing`, `updated`, + `error`) instead of a decorative spinner. The syncing state keeps a brief + minimum visible duration so fast cached/local API responses still communicate + that a live refresh happened. +- Polling should feel alive without jank: use a thin live sweep, changed + metric/request-row highlights, and content enter transitions; do not blank + the table or rebuild the whole visual shell on each tick. +- Tab switches must be local and immediate. The page keeps cached state for + both tabs, starts an async refresh after switching, and ignores stale fetches + from older refresh sequences. +- Realtime polling compares stable per-row signatures. If no data changed, the + existing DOM stays in place; if data changed, only the visible metrics/table + content is updated and changed rows are highlighted. +- Admin Governor page is read-only. Controls may exist only for local UI state + such as pause/autoscroll/filter, not for server configuration. +- User page has exactly two main tabs: quota/request details and protection + status. + +## Compatibility and rollback + +- Keep existing Governor management APIs for compatibility unless removal is + required by tests. The admin browser page simply stops exposing those controls. +- Keep `Cache-Control: no-store` for plugin HTML and JSON. +- If embedded SSE fails in CPAMP, the page falls back to snapshot refresh and + reports reconnecting instead of breaking the page. +- Both custom pages use recoverable browser state. Snapshot fetches have an + AbortController timeout, foreground resume aborts stale work and starts a new + snapshot, and SSE connections are closed while hidden and recreated when the + page becomes visible again. +- Rollback is replacing the previous Governor plugin binary and restoring the + previous Caddy admin route block from backup. + +## Security + +- Do not include API keys, OAuth tokens, cookies, request bodies, response + bodies, encrypted reasoning, or complete key hashes in HTML, JSON, logs, or + Trellis docs. +- Public API domain blocks plugin resource and admin paths. +- User protection summaries are scoped to the logged-in key. Unknown key + identity must degrade to "no data" for the user page rather than showing + global data. diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/implement.md b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/implement.md new file mode 100644 index 0000000..e582528 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/implement.md @@ -0,0 +1,221 @@ +# Governor UI route cleanup implementation plan + +## Checklist + +1. Load backend specs for Governor, CodexCont dashboard, and usage portal + contracts. +2. Refactor Governor admin HTML into a CodexCont status dashboard: + - read-only page, + - snapshot fetches via `/governor/codexcont/admin/*`, + - SSE with reconnect/foreground resume, + - no Key/request/config tabs. +3. Refactor Governor user HTML into two tabs: + - quota and recent request detail, + - per-key protection status. +4. Hide the user resource from CPAMP sidebar menus while keeping the resource + routable for `cpa-usage.konbakuyomu.us`. +5. Extend usage-event storage/projection with optional CPAMP realtime-style + fields: requested/actual model, provider, executor type, reasoning effort, + service tier, TTFT, and failure status code. +6. Extend user CodexCont API to return safe per-key protection summaries, or a + safe empty projection when no matching summaries exist. +7. Adjust shared CSS to match CodexCont dashboard style and remove spinner / + diagonal animation. +8. Rework user refresh behavior: + - tab switches render cached state immediately, + - manual refresh keeps a visible short sync state, + - background polling pauses, + - foreground resume fetches a fresh snapshot, + - row signatures prevent no-op full table redraws. +9. Rework admin Governor refresh/SSE behavior: + - snapshot fetches are abortable and time out, + - hidden tabs close the SSE stream, + - foreground resume forces a fresh snapshot and stream reconnect, + - stale delayed processing follow-ups cannot keep the page stuck. +10. Update tests for menu hiding, schema migration, admin/user HTML shape, + removed controls, user filtering, request-detail projection, and no-store + behavior. +11. Run local checks: + - `go test ./...` in `cpa_governor_plugin/go` + - `.venv\\Scripts\\python.exe tests\\test_middleware.py` + - `.venv\\Scripts\\python.exe tests\\test_cpa_usage_portal.py` + - compileall for Python runtime files + - `git diff --check` +12. Use Playwright CLI to validate local/static or test-served admin/user pages + at desktop and 390px mobile widths, including immediate tab switching, + two no-op polling cycles without table jitter, changed-row highlighting, + and hung-refresh recovery for both admin and user pages. +13. Build linux/amd64 Governor plugin artifact and record SHA256. +14. Deploy to SJC: + - check disk, + - backup CPA plugin binary/config/Caddyfile/CodexCont route config, + - upload self-owned artifact and Caddy route patch, + - restart only necessary services. +15. Server smoke: + - CPA health and authenticated model path, + - CPAMP sidebar Governor page, + - CPAMP sidebar no longer advertises `CPA Usage`, + - dedicated user page login, + - Kuma test key user page refresh remains recoverable after idle/manual + refresh and current conversation traffic appears in realtime rows, + - `/codexcont/` returns 404, + - `/governor/codexcont/admin/*` works behind admin host, + - public API domain blocks admin/plugin paths. +16. Update specs or task notes with learned contracts, commit, and archive. + +## Risk points + +- CPA plugin ResourceRoute is GET-only. Do not add POST-only user-resource + behavior. +- CPAMP embedded pages may keep stale JS. Keep no-store and advise hard refresh + only if validation shows old HTML still loaded. +- Admin SSE data path must remain admin-host-only; public exposure fails + acceptance. +- The task must not switch `codexcont_route` or change the production execution + path. + +## Implementation Evidence + +- Local Go tests passed in `cpa_governor_plugin/go`: `go test ./...`. +- Python regression passed: `.venv\Scripts\python.exe tests\test_middleware.py` with 163/163 checks and `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` with 84/84 checks. +- Python compile smoke passed: `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py`. +- `git diff --check` passed. +- Playwright CLI verified local preview pages: + - `artifacts/governor-admin-desktop.png` + - `artifacts/governor-admin-mobile.png` + - `artifacts/governor-user-desktop.png` + - `artifacts/governor-user-mobile.png` +- Linux plugin artifact built from WSL: + - `cpa_governor_plugin/dist/linux/amd64/cpa-governor.so` + - SHA256 `257228790455b9bc345bdf6d390a01b114482e8044e8c71251b3018e3a5e4538` + - `file` reported ELF 64-bit x86-64 shared object. + +## Server Deployment Evidence + +- Pre-deploy SJC root disk was tight but usable: `9.6G` total, about `507M` free. No Docker prune, no image pull, and no container rebuild were used. +- Backup path: `/root/cpa-governor-ui-route-cleanup-backups/20260702-governor-ui-route-cleanup-130447`. +- Uploaded only the Governor `.so`; installed server SHA256 matches local artifact. +- Restarted only `cpa` and `cpa-admin-proxy`; reloaded `caddy-edge` config. Existing `codexcont`, `cpamp`, and `cpa-usage-portal` containers were not rebuilt. +- Admin proxy Caddy now returns `404` for `/codexcont/` and exposes admin-only `/governor/codexcont/admin/*` to CodexCont `/admin/*`. +- Public `cpa.konbakuyomu.us` blocker now includes `/governor*` alongside `/codexcont*` and plugin/admin paths. +- Server smoke results: + - `http://127.0.0.1:8317/healthz`: `200`. + - `http://127.0.0.1:8327/governor/`: `200`, contains `CodexCont 实时保护状态` and no old Key management controls. + - `http://127.0.0.1:8327/governor-user/`: `200`, contains only `额度与明细` and `思维链保护` tabs. + - `http://127.0.0.1:8327/codexcont/`: `404`. + - `http://127.0.0.1:8327/governor/codexcont/admin/status`: `200`. + - `http://127.0.0.1:8327/governor/codexcont/admin/requests?limit=2`: `200`. + - `http://127.0.0.1:8327/governor/codexcont/admin/logs/stream?once=1`: `200`, emitted `ready` and `request` events. + - Public `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-governor/admin`: `404`. + - Public `https://cpa.konbakuyomu.us/codexcont/`: `404`. + - Public `https://cpa.konbakuyomu.us/governor/`: `404`. + - Public `https://cpa-usage.konbakuyomu.us/`: `200`, contains the two-tab user page. +- CPA logs after restart showed `plugin loaded plugin_id=cpa-governor` and `plugin registered plugin_id=cpa-governor`. +- User API smoke used a server-signed short-lived session without printing raw keys or secrets: + - `/governor-user/api/me`: `200`, safe name/preview only. + - `/governor-user/api/codexcont?limit=80` for `QQ专用`: `200`, returned only that key's CodexCont summaries. + - `/governor-user/api/events?range=24h&limit=3`: `200`. +- Post-deploy disk remained about `495M` free. + +## Animation Refinement Evidence + +- User feedback after the first rollout: `CPA Usage` still felt stiff, and the + top-right refresh button lacked clear realtime feedback. +- Added a refresh state model to the user page: `同步中` with `aria-busy`, then + `刚刚更新` or `同步失败`; hand-triggered refresh keeps a short minimum visible + syncing state so fast responses do not appear as no-op clicks. +- Added shared CSS hooks for smooth realtime behavior without reintroducing + the old spinner or diagonal metric decoration: + `syncSweep`, live topbar sweep, metric bump, changed row highlight, and + content enter transition. +- Local validation after the refinement: + - `go test ./...` in `cpa_governor_plugin/go`. + - `.venv\Scripts\python.exe tests\test_middleware.py` with 163/163 checks. + - `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` with 84/84 checks. + - `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py run_usage_portal.py`. + - `git diff --check`. +- Playwright preview validation: + - `artifacts/governor-user-animation-desktop.png` + - `artifacts/governor-user-animation-mobile.png` + - `artifacts/governor-user-refresh-syncing.png` + - Refresh state sample confirmed `syncing` at 0 ms and 180 ms, then + `just-updated` at 600 ms. +- Linux plugin artifact rebuilt after the animation refinement: + - SHA256 `bd980ad3ea82e224ed6becfc1e1e9729eb9a7e7f539c3853494fe005362c1813`. +- SJC deployment evidence for the animation refinement: + - Pre-deploy root disk remained tight: about `489M` free; no Docker prune, + image pull, or container rebuild was used. + - Backup path: + `/root/cpa-governor-ui-route-cleanup-backups/20260702-governor-usage-animation-140408`. + - Uploaded only `cpa-governor.so`, installed server SHA256 matched the local + artifact, and restarted only `cpa`. + - CPA logs showed `plugin loaded plugin_id=cpa-governor` and + `plugin registered plugin_id=cpa-governor` after restart. + - Server smoke: `http://127.0.0.1:8317/healthz` returned `200`; + `http://127.0.0.1:8327/governor-user/` contained `sync-button`, + `content-refreshing`, and `row-fresh`; public + `https://cpa.konbakuyomu.us/governor/` returned `404`; + public `https://cpa-usage.konbakuyomu.us/` contained the new animation + hooks. + - Post-deploy disk was about `477M` free, and `/tmp/cpa-governor.so` was + removed explicitly. + +## Sync Light Follow-up Evidence + +- User feedback after the animation refinement: the right-top realtime/sync + status light also needs to follow the same state as the refresh label and + realtime chip. +- Fixed both custom Governor pages so the refresh button light keeps a live + state class (`live-ok`, `live-info`, `live-warn`, or `live-bad`) driven by + the same status function as the connection chip. When the temporary + `刚刚更新` label resets to `刷新`, the light remains in the current live + state instead of falling back to grey. +- Admin Governor page now also keeps a minimum visible `同步中` duration for + manual/foreground snapshot refreshes, matching the user page and preventing + fast local snapshots from making the click feel like a no-op. +- Local validation after the sync-light follow-up: + - `go test ./...` in `cpa_governor_plugin/go`. + - JavaScript syntax check by extracting `<script>` from `assets/user.html` + and `assets/admin.html` and running `node --check --input-type=commonjs -`. + - `.venv\Scripts\python.exe tests\test_middleware.py` with 163/163 checks. + - `.venv\Scripts\python.exe tests\test_cpa_usage_portal.py` with 84/84 + checks. + - `.venv\Scripts\python.exe -m compileall middleware cpa_usage_portal run.py + run_usage_portal.py`. + - `git diff --check` passed with only CRLF warnings. +- Playwright CLI validation: + - Local preview user page showed `syncing live-info`, then + `just-updated live-ok`, then settled on `刷新` with `live-ok`. + - Local preview admin page showed `syncing live-info`, then + `just-updated live-warn` while the preview SSE was reconnecting, then + settled on `刷新` with the realtime error/reconnect light still active. + - Mobile screenshots were captured for both user and admin pages: + `artifacts/governor-user-sync-light-mobile.png` and + `artifacts/governor-admin-sync-light-mobile.png`. +- Linux plugin artifact rebuilt after the sync-light follow-up: + - SHA256 `6d8970fd0168efbb691bb8e322fcea380d4d0e311025a6d50b96c3a6c4bc82f1`. + - `file` reported an ELF 64-bit x86-64 shared object. +- SJC deployment evidence for the sync-light follow-up: + - Pre-deploy root disk remained tight: about `470M` free; no Docker prune, + image pull, or container rebuild was used. + - Backup path: + `/root/cpa-governor-ui-route-cleanup-backups/20260702-sync-light-154445`. + - Uploaded only `cpa-governor.so`, installed server SHA256 matched the local + artifact, restarted only `cpa`, and removed the temporary upload file. + - CPA logs showed `plugin loaded plugin_id=cpa-governor` and + `plugin registered plugin_id=cpa-governor` after restart. + - Server smoke: `http://127.0.0.1:8317/healthz` returned `200`; + `http://127.0.0.1:8327/governor/` and `/governor-user/` contained the + sync-light hooks; admin data channel `/governor/codexcont/admin/status` + and `/requests?limit=2` returned `200`. + - Public boundary stayed closed: + `https://cpa.konbakuyomu.us/governor/`, + `/v0/resource/plugins/cpa-governor/admin`, and `/codexcont/` all returned + `404`; `https://cpa-usage.konbakuyomu.us/` returned `200`. + - User API smoke with the Kuma test key succeeded without printing raw + secrets: session ok, key name `kuma专用`, 24h usage returned calls and cost, + and CodexCont summary returned 20 own-key records. + - Production Playwright smoke on `https://cpa-usage.konbakuyomu.us/` logged + in with the test key and verified 100 request rows plus the same + `syncing live-info -> just-updated live-ok -> 刷新 live-ok` transition. + - Post-deploy root disk was about `458M` free. diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/prd.md b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/prd.md new file mode 100644 index 0000000..ade271d --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/prd.md @@ -0,0 +1,105 @@ +# Governor UI route cleanup + +## Goal + +Clean up the CPA Governor UI and routing model so each entrypoint has one clear +job: + +- CPAMP `配置面板` remains CPAMP-only configuration. +- CPA `插件管理 -> cpa-governor -> 编辑配置` remains low-level Governor plugin + configuration. +- CPAMP sidebar `CPA Governor` becomes a read-only CodexCont protection status + dashboard. +- `https://cpa-usage.konbakuyomu.us/` becomes the only daily user + self-service entry. The Governor user resource remains addressable for the + dedicated host, but it is no longer advertised as a CPAMP sidebar menu. + +The user-facing result is less confusion, a smoother realtime status page, and +one consistent visual language across custom Governor/usage pages. + +## Requirements + +- Create/maintain this Trellis task with `prd.md`, `design.md`, and + `implement.md` before implementation. +- Do not modify CPA, CPAMP, or CPA Key Policy upstream source or images. +- Replace the current Governor admin page content with a read-only CodexCont + protection dashboard modeled after the existing CodexCont panel. +- Remove daily-use Key management, request-detail, and CodexCont setting forms + from the Governor admin page. Persistent Governor settings belong in CPA's + plugin configuration drawer. +- Return 404 for old `https://cpa-admin.konbakuyomu.us/codexcont/` entrypoint + after migration. +- Add an admin-only data path under `/governor/codexcont/admin/*` that proxies + to CodexCont `/admin/*` so the embedded Governor page can use the existing + snapshot and SSE contracts. +- Redesign the user page into two tabs: + - `额度与明细`: key quota summary plus recent requests with expandable + request detail. + - `思维链保护`: the current key's CodexCont protection summaries. +- User views must filter by the logged-in `cpa_` key and must not show other + users' request or protection records. +- The user page should feel like a single-key, simplified version of CPAMP + `请求监控 -> 实时监控`: keep total usage stats and per-call rows, but remove + admin/global dimensions such as account summaries, client-key summaries, + provider/account filters, and global key display controls. +- The user page's `额度与明细` tab must show quota, usage, recent requests, and + rich expandable request details in one balanced layout. +- The user page's `思维链保护` tab must use the same table/detail visual system + as `额度与明细`, scoped to the current key's CodexCont protection records. +- Remove the visible refresh spinner / diagonal animation style. Use CodexCont + style realtime chips and dot pulse animation. +- User page manual refresh must provide a visible realtime rhythm: a short + `同步中` state, a completion confirmation, a subtle live sweep, and changed + row/card highlights. Fast local responses must not collapse the animation + into an imperceptible instant update. +- Tab switching must be immediate and must not wait for network responses. +- Avoid full table re-render loops where possible. Admin status uses SSE; the + user page uses cached tab state, lightweight visible-tab polling, data + signatures, and foreground resume so the table does not visibly jitter every + polling cycle. +- The admin `CPA Governor` CodexCont page and the user page must both survive + idle/background tabs: no permanent `同步中`/`正在连接` state, stale requests are + aborted or ignored, and foreground resume forces a fresh snapshot plus stream + reconnect. +- Keep public `https://cpa.konbakuyomu.us` from exposing admin, plugin resource, + or CodexCont routes. + +## Acceptance Criteria + +- [ ] `CPA Governor` sidebar menu renders a read-only CodexCont status page + with protection result, active requests, hit round, latest reasoning, + continuation count, failures, and advanced logs. +- [ ] The Governor page contains no Key management tab, request-detail tab, or + server-side CodexCont save button. +- [ ] `/governor/codexcont/admin/status`, `/requests`, and `/logs/stream` work + through the admin host, while old `/codexcont/` returns 404. +- [ ] `CPA Usage` no longer appears as a CPAMP sidebar plugin menu, while the + dedicated `https://cpa-usage.konbakuyomu.us/` entry still serves the + two-tab user UI. +- [ ] The user UI behaves like a single-key realtime monitor: quota/usage cards, + per-call rows, rich expand details, and per-key CodexCont protection view + without account/client-key summary tabs. +- [ ] User page rejects native `sk...` keys and shortened previews as before, + and valid `cpa_` users only see their own data. +- [ ] Custom pages match the CodexCont dark compact style and do not show the + old spinning refresh or diagonal badge animation. +- [ ] `CPA Usage` manual refresh visibly transitions through syncing and + updated states, while realtime polling highlights changed rows/cards + without a full-page flash. +- [ ] Switching between `额度与明细` and `思维链保护` is immediate, and two polling + cycles do not cause periodic table reflow or short-field vertical text. +- [ ] After simulating a hung refresh or idle tab, both `CPA Governor` and + `cpa-usage` recover with manual refresh or foreground resume without a + full page reload. +- [ ] Local Go and Python tests pass. +- [ ] Playwright verifies desktop and mobile layouts for Governor admin and user + pages. +- [ ] Production deployment updates only self-owned Governor/Caddy/CodexCont + routing as needed, with no Docker prune and no official image source + changes. +- [ ] Server smoke verifies public/admin route boundaries after deployment. + +## Notes + +- Existing production `/v1/responses` flow stays on the proven CodexCont path. + This task does not cut over the Governor executor path. diff --git a/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/task.json b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/task.json new file mode 100644 index 0000000..923ed72 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-governor-ui-route-cleanup/task.json @@ -0,0 +1,26 @@ +{ + "id": "governor-ui-route-cleanup", + "name": "governor-ui-route-cleanup", + "title": "Governor UI route cleanup", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/design.md b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/design.md new file mode 100644 index 0000000..1051ef9 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/design.md @@ -0,0 +1,51 @@ +# Design + +## Boundaries + +- `cpa-key-policy-plus` 是普通用户 Key 的权威配置和额度统计入口。 +- `cpa-governor` 仍只负责管理员 CodexCont 实时状态展示。 +- 官方 CPA/CPAMP 镜像保持不变；生产只替换自有插件 `.so`。 + +## Key Control Changes + +- Frontend auth 删除并发和 active session 判定，只保留 enabled、archived/deleted absence、model allowlist、RPM、quota windows。 +- Store 层保留旧字段和表以兼容已有 SQLite schema，但保存 Key 时强制 `Concurrency=0`、`MaxActiveSessions=0`。 +- Governor 侧也停止执行 request concurrency，避免两套插件里仍有隐藏拦截点。 + +## Hard Delete Contract + +- 新管理路由： + - `POST /v0/management/plugins/cpa-key-policy-plus/keys/delete` + - admin proxy alias: `POST /key-policy-plus/api/keys/delete` +- Request: `{ "id": "<key id>", "confirm": "delete" }` +- Effect: + - delete from `keys` + - delete from `reset_watermarks` + - delete from `active_sessions` + - keep `usage_events` + - keep `codexcont_summaries` + - append `audit_log` action `delete_key` +- Old archive route is removed from registration and returns HTTP 410 if stale HTML calls it. + +## UI Design + +- Key Policy+ admin list/detail: + - Show Key name/preview, enabled state, RPM, quota summaries, model/pricing summary. + - Remove concurrency/session UI. + - Add destructive delete action in detail panel with explicit confirm dialog. +- Shared visual system: + - Keep a single canonical `shared.css` shape for Plus and Governor. + - Use the same `.chip` output for protection values everywhere. + - Replace large custom protection result text in user detail cards with the same chip used by Governor rows. + +## Deployment Shape + +- Build Linux amd64 Plus plugin; build Governor only if its source changes. +- Backup Plus `.so`, Plus SQLite, CPA config, Caddy/admin proxy config. +- Upload changed plugin artifacts and restart only required services. +- After plugin deployment, call delete API for all keys where `enabled=false` or `archived=true`. + +## Rollback + +- Restore previous `.so` and Plus SQLite backup, restart CPA. +- Since hard delete removes active config rows, rollback for deleted keys requires SQLite backup restoration. diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-governor-admin-desktop.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-governor-admin-desktop.png new file mode 100644 index 0000000..4283d7e Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-governor-admin-desktop.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-governor-admin-mobile.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-governor-admin-mobile.png new file mode 100644 index 0000000..28427f9 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-governor-admin-mobile.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-usage-user-protection-desktop.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-usage-user-protection-desktop.png new file mode 100644 index 0000000..ed97462 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-usage-user-protection-desktop.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-usage-user-protection-mobile.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-usage-user-protection-mobile.png new file mode 100644 index 0000000..f34ef20 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/cpa-usage-user-protection-mobile.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/key-policy-plus-admin-desktop.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/key-policy-plus-admin-desktop.png new file mode 100644 index 0000000..0257a9c Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/key-policy-plus-admin-desktop.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/key-policy-plus-admin-mobile.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/key-policy-plus-admin-mobile.png new file mode 100644 index 0000000..2ed382c Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/key-policy-plus-admin-mobile.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/production-cpa-usage-kuma-protection.png b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/production-cpa-usage-kuma-protection.png new file mode 100644 index 0000000..89837a7 Binary files /dev/null and b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/evidence/production-cpa-usage-kuma-protection.png differ diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/implement.md b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/implement.md new file mode 100644 index 0000000..4a6f5f9 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/implement.md @@ -0,0 +1,81 @@ +# Implementation Plan + +## Local Implementation + +1. Update Plus backend: + - Remove frontend-auth calls to active session and request concurrency checks. + - Force create/save to persist `concurrency=0` and `max_active_sessions=0`. + - Add `DeleteKey` store method and admin delete route. + - Make archive route return 410 and remove it from management registration. +2. Update Plus admin UI: + - Remove create/detail/list fields for request concurrency and Codex windows. + - Remove show archived and archive/restore controls. + - Add delete button with confirmation and `/keys/delete` call. +3. Update Governor backend: + - Stop enforcing request concurrency in frontend auth. + - Keep compatibility fields untouched unless required by tests. +4. Update UI shared style: + - Align Plus and Governor `shared.css`. + - Render protection result in user detail cards via `chip(req.protection)`. +5. Update tests: + - Replace archive tests with delete tests. + - Add create/save tests for forced-zero concurrency/session fields. + - Add archive 410 compatibility test. + - Add frontend HTML smoke assertions for removed labels and delete button. + +## Validation + +- `go test ./...` in `cpa_key_policy_plus_plugin/go` +- `go test ./...` in `cpa_governor_plugin/go` +- Extract `assets/admin.html` and `assets/user.html` scripts for `node --check` +- `git diff --check` +- Playwright: + - Key Policy+ admin page desktop and 390px mobile + - CPA Usage user page desktop and 390px mobile + - CPA Governor admin page desktop + - Confirm removed labels and unified protection chips + +## Server Rollout + +1. Check SJC disk free space. +2. Backup: + - `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-key-policy-plus.so` + - `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-governor.so` if changed + - `/opt/codex-stacks/cpa/plugin-state/cpa-key-policy-plus/policyplus.sqlite` + - CPA config and Caddy/admin proxy config. +3. Upload changed `.so` files, restart CPA only. +4. Verify plugin load logs and SHA256. +5. Delete production disabled/archived keys through the new delete API. +6. Verify Kuma test key login and a lightweight authenticated call. +7. Verify public blocked paths remain 404. + +## Risks + +- Hard deletion is intentionally irreversible without SQLite backup. +- Existing stale admin HTML may call archive route; 410 response prevents accidental stale archive behavior. + +## Execution Notes + +- Local tests passed: + - `go test ./...` in `cpa_key_policy_plus_plugin/go` + - `go test ./...` in `cpa_governor_plugin/go` + - inline script syntax smoke for Plus/Governor `admin.html` and `user.html` + - `git diff --check` +- Playwright preview checks passed for desktop and 390px mobile: + - Key Policy+ admin has no retired concurrency/Codex-window/archive controls and has hard delete. + - CPA Usage and CPA Governor protection details render protection result with the same `.chip` component. + - Tables stay inside overflow panels; no page-level horizontal overflow. +- Linux amd64 artifacts deployed on SJC: + - `cpa-key-policy-plus.so` SHA256 `a7a2b3cac09af1a019b37942f65a25e177b27f55e384c0a15bc93b2c62bfac37` + - `cpa-governor.so` SHA256 `c0903c72d574fde4fa1f47185a0dbd9ec8ccc5579a533a8eeb8cf14c31d116dd` +- Production backup: + - `/opt/codex-stacks/backups/key-policy-plus-rpm-delete-ui-unify-20260703-000536` +- Production cleanup: + - Deleted disabled keys `key_5ce1c632...a19b5a` and `key_2bb8fd29...b016b1` through the new delete API. + - Remaining Plus keys: 3 enabled, 0 disabled/archived. +- Production smoke: + - CPA logs show both plugins loaded and registered after restart. + - Kuma test key logs into `https://cpa-usage.konbakuyomu.us/`. + - Kuma test key authenticates against `https://cpa.konbakuyomu.us/v1/models` and returns 7 models. + - User usage/events/codexcont APIs return `ok`. + - Public `https://cpa.konbakuyomu.us` returns 404 for plugin/admin/governor/codexcont paths. diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/prd.md b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/prd.md new file mode 100644 index 0000000..b6c5a53 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/prd.md @@ -0,0 +1,33 @@ +# CPA Key Policy+ RPM-only、硬删除 Key、Usage/Governor 风格统一 + +## Goal + +将普通用户 Key 控制收敛到 `RPM + 模型白名单 + 5H/24H/7D/月额度`，移除不好用的「请求并发」和「Codex窗口」限制；将 Key 生命周期从「禁用/归档」改为管理员可执行的硬删除；统一 `cpa-usage.konbakuyomu.us` 用户页和 CPAMP 左下角 `CPA Governor` 面板的视觉组件。 + +## Requirements + +- `cpa-key-policy-plus` 不再执行 `concurrency` 和 `max_active_sessions` 限制。 +- 新建/保存 Key 时兼容旧 payload，但 `concurrency` 和 `max_active_sessions` 统一保存/返回为 `0`。 +- 管理页移除「请求并发」「Codex窗口」「显示归档」「归档隐藏」「恢复 Key」。 +- 管理页新增硬删除按钮和确认流程，删除 Key 配置、reset watermarks、active sessions；保留脱敏历史 usage/codex summary。 +- 旧 archive API 不再注册，兼容入口返回 `410 archive_removed_use_delete`。 +- 部署后删除生产中当前所有禁用或归档 Key。 +- `CPA Usage` 和 `CPA Governor` 统一状态 chip、详情卡、表格、按钮、工具栏状态灯风格，特别是保护结果在详情中也使用同款气泡 chip。 +- 不改 CPA、CPAMP 官方源码或镜像；只改自有 Plus/Governor 插件和必要部署。 + +## Acceptance Criteria + +- [x] Key Policy+ 管理页不再出现「请求并发」「Codex窗口」「显示归档」或归档/恢复按钮。 +- [x] 新建/保存 Key 后返回的 `concurrency` 与 `max_active_sessions` 为 `0`。 +- [x] 删除按钮能删除 Key；删除后该完整 `cpa_` key 无法登录用户页或通过 CPA frontend auth。 +- [x] 删除不会删除该 Key 既有 usage/codex summary 历史记录。 +- [x] 当前生产禁用/归档 Key 被删除，启用 Key 继续可登录和调用。 +- [x] CPA Usage 与 CPA Governor 中保护状态 chip、详情卡、表格密度和按钮视觉一致。 +- [x] 本地 Go 测试、HTML 脚本语法检查、Playwright 关键页面检查通过。 +- [x] 生产公网 `cpa.konbakuyomu.us` 仍阻断管理和插件路径。 + +## Out of Scope + +- 不实现新的 Codex 窗口识别机制。 +- 不删除历史 usage/codex summary 账本。 +- 不切换 `/v1/responses` 执行链路。 diff --git a/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/task.json b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/task.json new file mode 100644 index 0000000..6ce7fc3 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-key-policy-plus-rpm-delete-ui-unify/task.json @@ -0,0 +1,26 @@ +{ + "id": "key-policy-plus-rpm-delete-ui-unify", + "name": "key-policy-plus-rpm-delete-ui-unify", + "title": "CPA Key Policy+ RPM-only deletion and UI unification", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-03", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/check.jsonl b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/design.md b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/design.md new file mode 100644 index 0000000..1168ff6 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/design.md @@ -0,0 +1,227 @@ +# Usage Admin Batch Save And Request Details Design + +## Boundary + +This task modifies only the custom `cpa-usage-portal` service: + +- backend: `cpa_usage_portal/app.py`, `pricing.py`, `redaction.py`, and local + quota state helpers if needed +- frontend: `cpa_usage_portal/static/admin.html` and + `cpa_usage_portal/static/dashboard.html` +- tests: `tests/test_cpa_usage_portal.py` + +CPA, CPAMP, CPA Key Policy, and their images/source trees remain untouched. + +## Admin Batch Save + +Current flow: + +- `GET /admin/api/keys` returns safe key projections plus local 5H/month limits. +- Each row renders a per-key `保存` button. +- Clicking it sends `PUT /admin/api/keys/{key_id}/limits`. + +New flow: + +- The table renders editable 5H/month inputs only. +- A toolbar-level `保存全部` button compares current input values with the last + loaded snapshot. +- The button is disabled when there are no dirty edits. +- Dirty state is visible in the toolbar and, optionally, on changed rows. +- Submit one payload: + +```json +{ + "limits": [ + { + "id": "alice-key", + "five_hour_usd": 1.25, + "monthly_usd": 20 + } + ] +} +``` + +Backend adds `PUT /admin/api/keys/limits`: + +- requires the existing admin guard +- validates that `limits` is a list +- validates each id resolves to a Key Policy record +- parses each numeric limit with the existing `_parse_float_limit` +- writes all valid limits to local `QuotaState` +- returns refreshed safe records, or an explicit error with the bad id/index + +The existing per-key route can remain for backwards compatibility and tests. + +## Admin Recent Requests + +`usage-admin` changes from "one selected Key only" to "all Keys by default": + +- `key_id=all` merges recent events from enabled Key Policy records. +- Each event receives a safe `key` projection with `id`, `name`, `preview`, and + `enabled`. +- The UI renders an `全部 Key` select option first, then one option per key. +- The recent-request table adds a `用户/Key` column. +- Single-key filtering remains available by passing the concrete policy id. + +## Pricing Breakdown + +Current `pricing.py` recomputes only `cost` and `cost_source`. + +New backend projection: + +- keep the current total cost fields for compatibility +- add `cost_breakdown` to event rows after `safe_events(...)` and + `apply_event_pricing(...)` +- compute breakdown in Python from the same Key Policy `ModelPrice` used for + table totals, so frontend math does not duplicate pricing rules +- align cache semantics with CPAMP's own management panel. CPAMP API + `cached_tokens` is already the compatibility cached-input bucket. The portal + displays it as `CPAMP 缓存命中`, while fine-grained cache read/create remain + separate. If a raw row includes `cache_tokens`, normalize it with CPAMP's + formula before pricing to avoid double counting. + +Suggested `cost_breakdown` shape: + +```json +{ + "source": "key_policy", + "price_model": "gpt-5.5", + "unit": "usd_per_1m_tokens", + "service_tier": "priority", + "service_tier_multiplier": 2.5, + "prices": { + "input_per_million": 5, + "output_per_million": 30, + "cache_read_per_million": 0.5, + "cache_creation_per_million": 5 + }, + "tokens": { + "input": 955, + "cached_input": 86016, + "cpamp_cached_input": 86016, + "billable_uncached_input": 955, + "cache_read": 0, + "cache_creation": 0, + "fine_grained_cache_read": 0, + "fine_grained_cache_creation": 0, + "effective_cache_read_for_hit_rate": 86016, + "cache_hit_rate": 0.989, + "cache_semantics": "cpamp_compatible_cached_tokens", + "output": 2455, + "reasoning": 2270, + "visible_output_estimate": 185, + "total": 89426 + }, + "costs": { + "input": 0.004775, + "cached_input": 0.043008, + "cache_read": 0, + "cache_creation": 0, + "output": 0.07365, + "subtotal": 0.121433, + "total": 0.3035825 + } +} +``` + +`visible_output_estimate` is a safe derived number: + +```text +max(output_tokens - reasoning_tokens, 0) +``` + +It must be clearly labeled as an estimate because upstream accounting can vary. + +## Detail UI + +The user page keeps the compact main table: + +- time/request id +- status +- model +- latency/TTFT +- total tokens with input/output hint +- reasoning tokens +- cost +- expand action + +Expanded detail should be reorganized into sections: + +- `请求信息`: request id, endpoint, status code, model/requested model, + service tier, reasoning effort +- `Token 组成`: input, cached input, cache read, cache creation, output, + reasoning, visible output estimate, total +- `费用组成`: price model, source, per-million prices, service tier multiplier, + individual cost parts, total +- `限额窗口`: included windows, current range, remaining, reset/start point +- `失败详情`: short redacted failure reason, expanded redacted detail only when + failed + +The admin events area can reuse the same rendering helpers or a simplified +variant, but it should expose the same cost/token composition for operators. + +## Sub2API Lessons Applied + +Sub2API stores and exposes separate token and cost categories instead of one +opaque amount: + +- input tokens/cost +- output tokens/cost +- cache creation tokens/cost +- cache read tokens/cost +- total cost and actual cost +- service tier and reasoning effort +- request type, stream/openai-ws mode, latency, TTFT + +Our portal cannot recover fields CPAMP never records, and should not invent +secret/raw payload fields. The useful adaptation is to present all safe fields +CPAMP already provides, plus a deterministic pricing projection using our Key +Policy prices. + +## CodexCont Protection Correlation And Key Identity + +This task does not join usage events to CodexCont request summaries. It does add +safe Key identity to CodexCont's own request summaries: + +- Add optional `[admin] key_policy_state_path`. +- On request start, parse `Authorization: Bearer ...` only long enough to hash + the key and match the Key Policy state. +- Store only a safe `key_identity` projection on diagnostics request summaries. +- If the key is missing, display `未携带 Key`; if unmatched, display + `未识别 Key` plus a safe hash preview. + +This gives the CodexCont table the same operator-friendly "who made this +request" context as `usage-admin` without introducing a cross-service join. + +## Visual Consistency + +Both custom dashboards use the same dark operations-console style: + +- remove the current `metric::after` crescent decoration +- keep 8px cards, restrained gradients, compact chips, fixed-width tables, and + dense but readable detail panels +- avoid decorative shapes that can be mistaken for broken chart widgets + +## Compatibility And Security + +- Existing `/api/events` and `/admin/api/events` fields stay compatible. +- New fields are additive. +- Redaction remains server-side. +- The frontend must not reconstruct costs from hidden raw event data. +- No raw request body, response body, authorization header, cookies, OAuth + token, full hash, management key, or encrypted reasoning content is exposed. + +## Rollout + +Local first: + +- run unit/smoke tests +- inspect the pages with desktop and narrow mobile widths if implementation + changes layout materially + +Server rollout later: + +- back up `/opt/codex-stacks/cpa-usage-portal` +- upload/rebuild/restart only `cpa-usage-portal` +- do not touch CPA, CPAMP, or Key Policy images/source +- do not use Docker prune or batch deletion diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/implement.jsonl b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/implement.md b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/implement.md new file mode 100644 index 0000000..bee609f --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/implement.md @@ -0,0 +1,259 @@ +# Usage Admin Batch Save And Request Details Implementation Plan + +## Evidence Already Collected + +- `cpa_usage_portal/static/admin.html` currently renders `button data-save` + per key and calls `saveLimits(keyId)`. +- `cpa_usage_portal/app.py` currently exposes only + `PUT /admin/api/keys/{key_id}/limits` for limit updates. +- `cpa_usage_portal/static/dashboard.html` currently has a thin expanded + request row without itemized pricing. +- `cpa_usage_portal/pricing.py` currently computes only total cost and the + matched price model. +- Sub2API's usage types and billing service separate token and cost categories + into input, output, cache creation, cache read, total, actual, service tier, + reasoning effort, latency, and TTFT. + +## Implementation Checklist + +1. Add backend batch save route. + - Add `admin_update_limits_batch`. + - Route: `PUT /admin/api/keys/limits`. + - Validate list shape, ids, numeric limits. + - Save through `QuotaState.set_limits`. + - Return refreshed safe key projections. + +2. Refactor `usage-admin` frontend. + - Remove per-row `保存`. + - Add toolbar `保存全部`. + - Track loaded snapshot and current input values. + - Mark dirty state and disable save when clean. + - Keep reset buttons per row. + - Add saving/saved/error feedback. + +3. Add admin all-key events. + - Support `key_id=all` in `/admin/api/events`. + - Fetch each enabled key's CPAMP events, attach safe key summary, merge, + sort by timestamp descending, and cap by requested limit. + - Update admin UI default select option to `全部 Key`. + +4. Add pricing breakdown projection. + - Add a function in `pricing.py` that returns both total cost and itemized + breakdown from `ModelTokens` and `ModelPrice`. + - Reuse the existing `service_tier_multiplier`. + - Preserve existing `cost`, `cost_source`, and `price_model`. + - Add `cost_breakdown` to priced events. + +5. Extend safe event projection if needed. + - Keep only safe scalar fields. + - Include fields needed by breakdown that are already safe: + `cached_tokens`, `cache_read_tokens`, `cache_creation_tokens`, + `reasoning_tokens`, `total_tokens`, `service_tier`, `reasoning_effort`. + - Do not expose raw CPAMP event payloads. + +6. Add CodexCont key identity. + - Add optional `AdminCfg.key_policy_state_path`. + - Add a small read-only identity resolver that parses Key Policy state and + hashes `Authorization` bearer values without retaining the raw key. + - Pass safe identity into `Diagnostics.request_started`. + - Render a `用户/Key` column in the CodexCont request table. + +7. Redesign expanded details. + - Update `dashboard.html` `detailRow`. + - Add token composition, cost composition, and quota/accounting sections. + - Keep failure detail compact and redacted. + - Update `admin.html` event rendering to expose comparable detail or a + compact operator version. + +8. Unify visual style. + - Remove `metric::after` crescent decorations from custom dashboard CSS. + - Keep refresh/loading/live-state animations consistent. + +9. Tests. + - Batch save succeeds and persists all edited keys. + - Batch save rejects malformed values/unknown ids. + - Existing per-key route still works. + - Admin all-key events include key summaries and never leak raw hashes. + - CodexCont request summaries include safe key identity when configured. + - Event pricing breakdown sums to displayed total. + - Redaction still removes secret-like fields. + - HTML smoke checks for `保存全部`, dirty state markers, and cost breakdown + labels. + +10. Validation. + - `python -m compileall cpa_usage_portal run_usage_portal.py` + - `python -m compileall middleware run.py` + - `python tests/test_cpa_usage_portal.py` + - `python tests/test_middleware.py` + - `git status --short --branch` + +## Files Likely To Change + +- `cpa_usage_portal/app.py` +- `cpa_usage_portal/pricing.py` +- `cpa_usage_portal/redaction.py` +- `cpa_usage_portal/static/admin.html` +- `cpa_usage_portal/static/dashboard.html` +- `middleware/config.py` +- `middleware/diagnostics.py` +- `middleware/app.py` +- `middleware/dashboard.html` +- `tests/test_cpa_usage_portal.py` +- `tests/test_middleware.py` + +## Risks And Rollback + +- Pricing math risk: table total and breakdown must use the same backend + calculation. Do not let the frontend recalculate totals independently. +- Partial save risk: batch route should fail visibly on invalid input before + the UI claims success. +- Security risk: expanded details should not become a raw CPAMP event viewer. +- Rollback is simple because this affects only `cpa-usage-portal`; revert these + files or redeploy the previous portal container. + +## Implementation Notes 2026-07-02 + +- Added `PUT /admin/api/keys/limits` for all-key local 5H/month limit saves. + Validation runs for the full payload before any `QuotaState` write. +- Added `key_id=all` to `/admin/api/events`; it queries enabled Key Policy + records, attaches safe `key` summaries, merges newest-first, and keeps each + row's own accounting/reset context. +- Added backend-owned `cost_breakdown` on priced events. The table `cost` is + the same value as `cost_breakdown.costs.total`; frontends only render it. +- Added CodexCont `middleware/key_identity.py` plus optional + `[admin] key_policy_state_path`. It hashes the bearer key for matching and + stores only `known/name/id/preview/source/enabled`. +- Reworked `cpa_usage_portal/static/admin.html` to use a single global + `保存全部`, all-key default request view, safe `用户/Key` column, and detailed + token/cost/reasoning/accounting expansion. +- Reworked `cpa_usage_portal/static/dashboard.html` expansion to show the same + useful breakdown sections for ordinary users. +- Added a CodexCont request table `用户/Key` column and removed the metric-card + crescent decoration from both custom dashboards. +- Documented the optional `key_policy_state_path` in `config.toml`, + `README.md`, and `README_zh.md`. + +## Verification 2026-07-02 + +- `.venv/Scripts/python.exe -m compileall cpa_usage_portal run_usage_portal.py` + passed. +- `.venv/Scripts/python.exe -m compileall middleware run.py` passed. +- `.venv/Scripts/python.exe tests/test_cpa_usage_portal.py` passed: + 79/79 checks. +- `.venv/Scripts/python.exe tests/test_middleware.py` passed: + 152/152 checks. +- Node inline-script syntax check passed for: + `cpa_usage_portal/static/admin.html`, + `cpa_usage_portal/static/dashboard.html`, and `middleware/dashboard.html`. +- `playwright-cli --version` confirmed the global Mise-managed CLI is available + (`0.1.14`). +- `playwright-cli -s=codex run-code --filename=.../research/playwright-layout-check.js` + passed for desktop and 390px mobile views of `usage-admin`, + `usage-dashboard`, and `codexcont-dashboard`. +- Playwright initially caught `usage-admin/mobile` horizontal body overflow. + The fix was to make the topbar brand flex child shrinkable (`min-width: 0`) + and full-width on narrow screens; the same guard now covers all three custom + pages. + +## Deployment Notes 2026-07-02 + +- Server backup created at + `/root/codex-backups/20260702-usage-admin-batch-save/self-owned-services-before-deploy.tgz`. +- Uploaded and rebuilt only `codexcont` and `cpa-usage-portal`; CPA, CPAMP, and + CPA Key Policy official images/source were not modified. +- Server validation caught one deployment-only issue: CodexCont's + `key_policy_state_path` must point to a container-visible mount, not the host + `/opt/...` path. Fixed by mounting + `/opt/codex-stacks/cpa/plugin-state:/data/plugin-state:ro` into `codexcont` + and setting + `key_policy_state_path = "/data/plugin-state/cpa-key-policy-state.json"`. +- Post-deploy validation: + - `codexcont` and `cpa-usage-portal` containers are running. + - `cpa-usage-portal /healthz` reports `key_policy_state=true` and + `cpamp=true`. + - CodexCont `/admin/requests` shows current requests with known + Key Policy identity. + - `cpa-admin` internal proxy renders `usage-admin` with `保存全部`, + `全部 Key`, and `用户/Key`, and renders the CodexCont dashboard with + `用户/Key`. + - Public `cpa.konbakuyomu.us/admin/requests` and `/codexcont/` return `404`. + - Public `cpa-usage.konbakuyomu.us/admin/api/keys` and `/usage-admin/` + return `404`. + - Root filesystem remained at about `667M` free after rebuild; no Docker + prune or broad deletion was used. + +## Cache Semantics Follow-up 2026-07-02 + +- User reported that the custom usage detail displayed `Cache Read = 0` and + `Cache Write = 0` even though the main panel showed high cache hit behavior. +- CPAMP source confirms that its analytics API projects `cached_tokens` with a + compatibility expression: + `max(max(cached_tokens, cache_tokens) - cache_read_tokens - + cache_creation_tokens, 0)`. +- CPAMP's monitoring UI uses `cached_tokens + cache_read_tokens` as cache-hit + tokens for hit-rate display. Therefore OpenAI/Codex rows can correctly have + large `cached_tokens` and zero fine-grained cache read/write fields. +- The portal fix keeps CPA, CPAMP, and CPA Key Policy untouched. It updates only + our `cpa-usage-portal` pricing projection and static pages: + - backend `cost_breakdown.tokens` now exposes `cpamp_cached_input`, + `fine_grained_cache_read`, `fine_grained_cache_creation`, + `effective_cache_read_for_hit_rate`, `cache_hit_rate`, and + `cache_semantics`; + - frontend labels now show `CPAMP 缓存命中` separately from + `细粒度 Cache Read/Write`, with an inline note explaining the OpenAI/Codex + zero-read case; + - regression tests cover CPAMP-compatible cached rows and raw + `cache_tokens` normalization without double counting. + +## Verification Follow-up 2026-07-02 + +- `.venv/Scripts/python.exe -m compileall cpa_usage_portal run_usage_portal.py` + passed. +- `.venv/Scripts/python.exe tests/test_cpa_usage_portal.py` passed: + 84/84 checks. +- `.venv/Scripts/python.exe -m compileall middleware run.py` passed. +- `.venv/Scripts/python.exe tests/test_middleware.py` passed: + 152/152 checks. +- HTML script extraction syntax check passed for `cpa_usage_portal` admin/user + pages and `middleware/dashboard.html`. +- Playwright layout check passed for `usage-admin`, `usage-dashboard`, and + `codexcont-dashboard` on desktop and 390px mobile. Playwright first caught a + mobile clipped-chip risk in the CodexCont realtime status chip; it was fixed + by giving chips a stable 32px min-height and explicit line-height. + +## Deployment Follow-up 2026-07-02 + +- Pre-deploy SJC root disk: `9.6G` total, `8.9G` used, about `661M` free. + `docker system df` showed build cache available, but no Docker prune or broad + deletion was used. +- Backup created at + `/root/codex-backups/20260702-cache-semantics-fix/self-owned-files-before-cache-fix.tgz`. +- Uploaded only self-owned files: + - `/opt/codex-stacks/cpa-usage-portal/app/cpa_usage_portal/pricing.py` + - `/opt/codex-stacks/cpa-usage-portal/app/cpa_usage_portal/static/admin.html` + - `/opt/codex-stacks/cpa-usage-portal/app/cpa_usage_portal/static/dashboard.html` + - `/opt/codex-stacks/codexcont/app/middleware/dashboard.html` +- Rebuilt/restarted only `cpa-usage-portal` and `codexcont`; CPA, CPAMP, and + CPA Key Policy official images/source were not modified. +- Post-deploy validation: + - `cpa-usage-portal` container health returned + `{"ok": true, "key_policy_state": true, "cpamp": true}`. + - `http://127.0.0.1:8327/usage-admin/` contained `CPAMP 缓存命中` and + `细粒度 Cache Read`. + - `http://127.0.0.1:8327/usage-admin/api/events?key_id=all&range=24h&limit=5` + returned 5 events; one recent event projected + `cpamp_cached_input=201216`, `fine_grained_cache_read=0`, + `effective_cache_read_for_hit_rate=201216`, + `cache_hit_rate≈0.9935`, and + `cache_semantics=cpamp_compatible_cached_tokens`. + - `codexcont` `/admin/healthz` returned `200`, and + `/codexcont/requests?limit=5` returned request summaries with + `key_identity`. + - `https://cpa-usage.konbakuyomu.us/` returned `200` and contains the new + cache labels; `https://cpa-admin.konbakuyomu.us/usage-admin/` returned + Cloudflare Access `302`. + - Public `https://cpa.konbakuyomu.us/admin/requests`, + `/codexcont/`, and `/usage-admin/` stayed `404`; public + `https://cpa-usage.konbakuyomu.us/admin/api/keys` and `/usage-admin/` + stayed `404`. + - Post-deploy root disk remained about `659M` free. diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/prd.md b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/prd.md new file mode 100644 index 0000000..f04d946 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/prd.md @@ -0,0 +1,100 @@ +# Usage admin batch save and request details + +## Goal + +Improve the custom CPA usage portal so it is easier to operate multiple keys +and easier to understand what each request actually cost. + +The task stays inside our own `cpa-usage-portal` service. It must not fork or +modify CPA, CPAMP, or CPA Key Policy official source/images. + +## Requirements + +- On `usage-admin`, replace per-row limit save buttons with one global + `保存全部` action. +- Keep per-key soft reset actions (`清零当前` / `清零全部`) as explicit, + destructive actions with confirmation. +- Track unsaved 5H/month limit edits in the admin UI so the operator can see + whether the page is clean, dirty, saving, saved, or failed. +- Add a batch limits API in `cpa-usage-portal` so all edited key limits are + validated and saved in one request. +- Improve request expansion details in the user usage page and the admin + request view by learning from Sub2API's usage display model: + - token breakdown: input, output, cached input, cache read, cache creation, + total, reasoning tokens + - cost breakdown: input cost, cached input cost, cache read cost, cache + creation cost, output cost, subtotal/total, price model, pricing source, + service tier multiplier + - request metadata: request id, endpoint, status code, latency, TTFT, + requested/resolved model, service tier, reasoning effort + - quota/accounting context: which windows the request counts into, selected + window remaining amount, reset watermark/start point + - failure details: short visible reason by default, longer redacted detail + only inside the expanded panel +- Preserve the existing security boundary: + - never return request body, response body, raw API key, full key hash, + Authorization, cookies, OAuth token, CPA/CPAMP management key, or encrypted + reasoning content + - never display real chain-of-thought; only display safe metrics such as + reasoning token counts and reasoning effort +- Cache display must follow CPAMP main-panel semantics: `cached_tokens` is the + CPAMP-compatible cache-hit bucket for OpenAI/Codex, while + `cache_read_tokens` / `cache_creation_tokens` are fine-grained fields that + may legitimately be zero. +- Keep the existing dark, compact dashboard style. The new details should feel + like an operational breakdown, not a log dump. +- `usage-admin` 最近请求默认显示全部 Key 的请求，表格必须显示 + 用户/Key 摘要，并保留单 Key 筛选。 +- CodexCont 保护状态面板也要显示安全的用户/Key 归属；来源为可选 + Key Policy state，只做哈希匹配和安全预览，不保存原始 key。 +- 两个自定义页面必须统一视觉语言，并移除指标卡中的月牙形装饰。 + +## Acceptance Criteria + +- [x] `usage-admin` shows one global `保存全部` button and no per-row `保存` + buttons. +- [x] Editing any 5H/month limit marks the admin page dirty; saving persists + all edited keys and then refreshes the displayed quota windows. +- [x] Batch save validates malformed numeric limits and unknown key ids without + partially hiding errors. +- [x] Per-row soft reset actions still work and still warn that they only write + our local reset watermark. +- [x] `usage-admin` 最近请求默认是全部 Key；表格能分清每条请求属于哪个 + 用户/Key，并可筛选到单个 Key。 +- [x] `/api/events` and `/admin/api/events` continue to return existing + compatible fields and additionally include a safe `cost_breakdown` + projection when Key Policy pricing is available. +- [x] `/admin/api/events?key_id=all` returns merged events across enabled keys, + sorted newest first and capped by `limit`. +- [x] CodexCont `/admin/requests` and request SSE include safe `key_identity` + when Authorization can be identified. +- [x] Expanded request detail shows useful cost/token/reasoning/accounting + sections and no longer spends most of the space on low-value raw failure + blobs. +- [x] The cost shown in the table equals the sum of the cost breakdown parts + after service-tier multiplier. +- [x] The metric cards in both custom pages no longer show the crescent-shaped + decorative arc. +- [x] Tests cover batch save, pricing breakdown, redaction safety, and request + detail HTML markers. +- [x] `.venv/Scripts/python.exe -m compileall cpa_usage_portal run_usage_portal.py` passes. +- [x] `.venv/Scripts/python.exe tests/test_cpa_usage_portal.py` passes. + +## Notes + +- Confirmed from current code: `cpa_usage_portal/static/admin.html` has + per-row `data-save` buttons calling `saveLimits(keyId)`, and + `cpa_usage_portal/app.py` only has a per-key + `PUT /admin/api/keys/{key_id}/limits` route. +- Confirmed from current code: `cpa_usage_portal/static/dashboard.html` + expansion currently shows request id, endpoint, status code, service tier, + reasoning effort, provider quota, accounting windows, and redacted failure + text, but does not show itemized pricing. +- Confirmed from Sub2API reference: + `frontend/src/types/index.ts` models usage with input/output/cache creation/ + cache read token and cost fields, plus `total_cost`, `actual_cost`, + `rate_multiplier`, `service_tier`, `reasoning_effort`, request type, stream, + latency, and TTFT. Its backend `CostBreakdown` separates input/output/image/ + cache creation/cache read costs before summing total/actual cost. +- Product decision resolved on 2026-07-02: `usage-admin` 最近请求采用 + "全部 Key 默认，单 Key 可筛选"。 diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/research/playwright-layout-check.js b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/research/playwright-layout-check.js new file mode 100644 index 0000000..95670a9 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/research/playwright-layout-check.js @@ -0,0 +1,322 @@ +async page => { + const results = []; + const consoleErrors = []; + page.on("console", msg => { + if (msg.type() === "error") consoleErrors.push(msg.text()); + }); + page.on("pageerror", err => consoleErrors.push(err.message)); + + await page.addInitScript(() => { + const key = { + id: "alice-key", + name: "Alice", + enabled: true, + preview: "cpa_...live", + rpm: 12, + limits: { five_hour_usd: 2.5, daily_usd: 5, weekly_usd: 30, monthly_usd: 25 }, + reset_points: { "5h": 0, "24h": 0, "7d": 0, month: 0 }, + usage_windows: { + "5h": { range: "5h", used_usd: 0.22, limit_usd: 2.5, used_percent: 0.088, reset_at_ms: null }, + "24h": { range: "24h", used_usd: 0.46, limit_usd: 5, used_percent: 0.092, reset_at_ms: null }, + "7d": { range: "7d", used_usd: 1.8, limit_usd: 30, used_percent: 0.06, reset_at_ms: null }, + month: { range: "month", used_usd: 6.2, limit_usd: 25, used_percent: 0.248, reset_at_ms: null }, + }, + pricing: { + priced_model_count: 1, + models: [{ + model: "gpt-5.5", + input_per_million: 5, + output_per_million: 30, + cache_read_per_million: 0.5, + cache_creation_per_million: 5, + }], + }, + }; + const event = { + request_id: "req_visual_a", + event_hash: "evt_visual_a", + timestamp_ms: Date.now() - 10000, + model: "gpt-5.5", + requested_model: "gpt-5.5", + endpoint: "/v1/responses", + status: "success", + failed: false, + status_code: 200, + latency_ms: 1280, + ttft_ms: 320, + input_tokens: 1000, + output_tokens: 500, + cached_tokens: 800, + cache_read_tokens: 0, + cache_creation_tokens: 0, + reasoning_tokens: 320, + total_tokens: 1500, + cost: 0.016, + cost_source: "key_policy", + price_model: "gpt-5.5", + service_tier: "priority", + reasoning_effort: "high", + api_key_preview: "94c1ab2d...51ee22", + key: { id: "alice-key", name: "Alice", preview: "cpa_...live", enabled: true }, + cost_breakdown: { + source: "key_policy", + price_model: "gpt-5.5", + unit: "usd_per_1m_tokens", + service_tier: "priority", + service_tier_multiplier: 2.5, + prices: { + input_per_million: 5, + output_per_million: 30, + cache_read_per_million: 0.5, + cache_creation_per_million: 5, + }, + tokens: { + input: 1000, + cached_input: 800, + cpamp_cached_input: 800, + billable_uncached_input: 200, + cache_read: 0, + cache_creation: 0, + fine_grained_cache_read: 0, + fine_grained_cache_creation: 0, + effective_cache_read_for_hit_rate: 800, + cache_hit_rate: 0.8, + cache_semantics: "cpamp_compatible_cached_tokens", + total_cache_activity: 800, + output: 500, + reasoning: 320, + visible_output_estimate: 180, + total: 1500, + }, + costs: { + input: 0.0025, + cached_input: 0.001, + cache_read: 0, + cache_creation: 0, + output: 0.0375, + subtotal: 0.0164, + total: 0.041, + }, + }, + accounting: { + selected_range: "24h", + included_windows: ["5h", "24h", "7d", "month"], + window_from_ms: Date.now() - 86400000, + window_to_ms: Date.now(), + reset_at_ms: null, + current_window_limit_usd: 5, + current_window_remaining_usd: 4.54, + }, + quota: { used_percent: 8.2, plan: "pro" }, + failure_brief: "", + failure: "", + }; + const usage = { + range: "24h", + from_ms: Date.now() - 86400000, + to_ms: Date.now(), + quota: { limit_usd: 5, used_usd: 0.46, remaining_usd: 4.54, used_percent: 0.092 }, + summary: { + total_calls: 8, + success_calls: 8, + failure_calls: 0, + success_rate: 1, + total_cost: 0.46, + total_tokens: 18000, + cached_tokens: 13000, + output_tokens: 2600, + reasoning_tokens: 1800, + cost_source: "key_policy", + }, + timeline: [], + model_share: [{ model: "gpt-5.5", calls: 8, tokens: 18000, cost: 0.46 }], + model_stats: [], + api_key_stats: [], + }; + const codexRequest = { + request_id: "cc_visual_a", + model: "gpt-5.5", + path: "/v1/responses", + started_at: new Date().toISOString(), + updated_at: new Date().toISOString(), + ended_at: new Date().toISOString(), + duration_ms: 2100, + status: "completed", + protection: "protected_clean", + folded: true, + passthrough: false, + passthrough_reason: null, + key_identity: { known: true, source: "key_policy_state", id: "alice-key", name: "Alice", preview: "cpa_...live", enabled: true }, + rounds: [{ round: 1, reasoning_tokens: 320, n: null, decision: "clean", buffered: ["message"], truncation_match: false }], + latest_round: 1, + latest_reasoning_tokens: 320, + first_truncation_round: null, + first_truncation_reasoning_tokens: null, + first_truncation_n: null, + first_truncation_decision: null, + continuation_count: 0, + truncation_match: false, + final_status: "completed", + stopped_reason: "natural", + failure_reason: null, + failure_detail: null, + }; + window.EventSource = class { + constructor(url) { + this.url = url; + this.readyState = 1; + this.listeners = {}; + setTimeout(() => { + if (this.onopen) this.onopen({}); + this.dispatch("ready", { ok: true }); + }, 30); + } + addEventListener(name, fn) { + this.listeners[name] = this.listeners[name] || []; + this.listeners[name].push(fn); + } + dispatch(name, data) { + for (const fn of this.listeners[name] || []) fn({ data: JSON.stringify(data) }); + } + close() { + this.readyState = 2; + } + }; + window.fetch = async input => { + const url = String(input); + const ok = data => new Response(JSON.stringify(data), { + status: 200, + headers: { "Content-Type": "application/json" }, + }); + if (url.includes("/admin/api/keys/limits")) return ok({ ok: true, keys: [key] }); + if (url.includes("/admin/api/keys")) return ok({ keys: [key] }); + if (url.includes("/admin/api/events")) { + return ok({ + key_id: url.includes("key_id=all") ? "all" : "alice-key", + range: "24h", + from_ms: Date.now() - 86400000, + to_ms: Date.now(), + reset_at_ms: null, + quota: null, + events: [event], + }); + } + if (url.includes("/api/me")) return ok({ me: key }); + if (url.includes("/api/usage")) return ok(usage); + if (url.includes("/api/events")) return ok({ + range: "24h", + from_ms: usage.from_ms, + to_ms: usage.to_ms, + reset_at_ms: null, + quota: usage.quota, + events: [event], + has_more: false, + }); + if (url.includes("status")) { + return ok({ + ok: true, + uptime_seconds: 120, + counters: { + total_requests: 8, + active_requests: 0, + folded_requests: 8, + continuations: 1, + truncation_hits: 1, + failures: 0, + }, + upstream: { ok: true }, + config: { upstream_host: "cpa:8317" }, + last_request_at: new Date().toISOString(), + last_continuation_at: new Date().toISOString(), + last_error_at: null, + }); + } + if (url.includes("requests")) return ok({ requests: [codexRequest], max_requests: 200 }); + if (url.includes("logs")) return ok({ events: [], max_events: 800 }); + return ok({}); + }; + }); + + const pages = [ + { + name: "usage-admin", + url: "file:///D:/Dev/20_Software/23_Reference/llm-gateway/CodexCont/cpa_usage_portal/static/admin.html", + must: ["保存全部", "全部 Key", "用户/Key", "Alice"], + detail: ["Token 组成", "费用组成", "CPAMP 缓存命中"], + }, + { + name: "usage-dashboard", + url: "file:///D:/Dev/20_Software/23_Reference/llm-gateway/CodexCont/cpa_usage_portal/static/dashboard.html", + must: ["CPA 用量自助页", "模型分布", "最近请求", "Alice"], + detail: ["Token 组成", "费用组成", "CPAMP 缓存命中"], + }, + { + name: "codexcont-dashboard", + url: "file:///D:/Dev/20_Software/23_Reference/llm-gateway/CodexCont/middleware/dashboard.html", + must: ["CodexCont 保护状态面板", "用户/Key", "Alice", "cc_visual_a"], + detail: ["思维链保护判断", "身份来源"], + }, + ]; + const viewports = [ + { label: "desktop", width: 1440, height: 900 }, + { label: "mobile", width: 390, height: 844 }, + ]; + + for (const viewport of viewports) { + await page.setViewportSize({ width: viewport.width, height: viewport.height }); + for (const target of pages) { + await page.goto(target.url); + await page.waitForTimeout(650); + for (const text of target.must) { + await page.getByText(text, { exact: false }).first().waitFor({ timeout: 5000 }); + } + const firstExpand = page.getByRole("button", { name: /展开/ }).first(); + if (await firstExpand.count()) { + await firstExpand.click(); + await page.waitForTimeout(120); + for (const text of target.detail) { + await page.getByText(text, { exact: false }).first().waitFor({ timeout: 5000 }); + } + } + if (target.name === "usage-admin") { + await page.locator('input[data-key="alice-key"][data-limit="5h"]').fill("3"); + await page.getByRole("button", { name: "保存全部" }).click(); + await page.waitForTimeout(250); + await page.waitForFunction(() => { + const button = document.querySelector("#saveAllBtn"); + const chip = document.querySelector("#saveChip"); + return button && button.disabled && chip && chip.textContent.includes("无改动"); + }, null, { timeout: 5000 }); + } + const metricsHaveArc = await page.evaluate(() => getComputedStyle(document.querySelector(".metric"), "::after").content !== "none"); + const layout = await page.evaluate(() => { + const root = document.scrollingElement || document.documentElement; + const badButtons = [...document.querySelectorAll("button, .chip, th")] + .map(el => { + const rect = el.getBoundingClientRect(); + return { + text: el.textContent.trim(), + width: rect.width, + height: rect.height, + scrollWidth: el.scrollWidth, + scrollHeight: el.scrollHeight, + }; + }) + .filter(item => item.text.length > 0 && item.width > 0 && item.height > 0) + .filter(item => item.scrollWidth > Math.ceil(item.width) + 2 || item.scrollHeight > Math.ceil(item.height) + 4); + return { + pageOverflow: root.scrollWidth > root.clientWidth + 2, + badButtons, + }; + }); + if (metricsHaveArc) throw new Error(`${target.name}/${viewport.label}: metric arc is still visible`); + if (layout.pageOverflow) throw new Error(`${target.name}/${viewport.label}: body has horizontal overflow`); + if (layout.badButtons.length) throw new Error(`${target.name}/${viewport.label}: clipped control text ${JSON.stringify(layout.badButtons.slice(0, 3))}`); + results.push(`${target.name}/${viewport.label}: ok`); + } + } + if (consoleErrors.length) { + throw new Error(`console errors: ${consoleErrors.join(" | ")}`); + } + return results; +} diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/research/sub2api-usage-detail-patterns.md b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/research/sub2api-usage-detail-patterns.md new file mode 100644 index 0000000..4b36123 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/research/sub2api-usage-detail-patterns.md @@ -0,0 +1,70 @@ +# Sub2API Usage Detail Patterns + +## Files Inspected + +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/frontend/src/views/KeyUsageView.vue` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/frontend/src/api/usage.ts` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/frontend/src/types/index.ts` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/frontend/src/utils/usagePricing.ts` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/frontend/src/utils/usageServiceTier.ts` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/backend/internal/pkg/usagestats/usage_log_types.go` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/backend/internal/service/billing_service.go` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/backend/internal/service/usage_log.go` +- `D:/Dev/20_Software/23_Reference/llm-gateway/sub2api/backend/internal/handler/dto/mappers.go` + +## Useful Concepts To Borrow + +Sub2API's useful design is not a specific table layout. The useful part is the +data model: + +- request identity: request id, model, requested/upstream model, endpoint +- performance: duration, first token latency, stream/request type +- reasoning metadata: reasoning effort +- token categories: input, output, cache creation, cache read, total +- cost categories: input cost, output cost, cache creation cost, cache read + cost, total cost, actual cost +- billing context: service tier, rate multiplier, billing mode/type + +The user-facing DTO intentionally hides admin-only fields such as account rate +multiplier and account details. That matches our portal's security model: +show useful cost/token composition, but do not expose internal secrets or raw +payloads. + +## Cost Logic Shape + +Sub2API's backend `CostBreakdown` keeps separate fields: + +- `InputCost` +- `OutputCost` +- `ImageOutputCost` +- `CacheCreationCost` +- `CacheReadCost` +- `TotalCost` +- `ActualCost` +- `BillingMode` + +Its token calculation applies service-tier / long-context / cache multipliers +before summing total cost. For our portal, the closest safe adaptation is: + +- use Key Policy prices already available to the portal +- calculate itemized costs server-side +- expose the breakdown as an additive safe projection on each event +- keep the main table's cost equal to the breakdown total + +## Differences In Our Portal + +Our portal receives CPAMP monitoring events, not Sub2API's native usage log +schema. Therefore: + +- we can only show fields CPAMP records and Key Policy prices can price +- we should not invent `actual_cost` semantics unless we add a rate multiplier + model later +- we should label the cost as an estimate based on Key Policy prices +- we should not expose raw CPAMP event JSON + +## Recommended MVP + +Implement itemized token/cost breakdown from CPAMP safe fields plus Key Policy +prices. Defer cross-service CodexCont protection correlation unless the user +explicitly wants that extra scope in this task. + diff --git a/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/task.json b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/task.json new file mode 100644 index 0000000..20ef9d8 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-02-usage-admin-batch-save-request-details/task.json @@ -0,0 +1,26 @@ +{ + "id": "usage-admin-batch-save-request-details", + "name": "usage-admin-batch-save-request-details", + "title": "Usage admin batch save and request details", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-02", + "completedAt": "2026-07-02", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "Completed: usage-admin now uses one batch save, all-key recent requests, richer request cost/token/reasoning details, and safe key identity in CodexCont. CPAMP-compatible cached_tokens semantics are documented and deployed: OpenAI/Codex cache hits can appear as large cached_tokens with zero fine-grained cache_read/cache_creation. Deployed only self-owned cpa-usage-portal and codexcont sidecars; CPA/CPAMP/Key Policy official artifacts were not modified.", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/check.jsonl b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/design.md b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/design.md new file mode 100644 index 0000000..de24745 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/design.md @@ -0,0 +1,60 @@ +# Design + +## Ownership Boundary + +CPA/CPAMP remains the source of truth for native `sk-...` keys and aliases. +Plus should not represent old self-managed `cpa_...` rows as active strategy +rows. Plus can retain old rows in SQLite for audit/history, but UI/API default +projections must be native-present rows. + +## Native Key Sync + +The sync routine reads: + +```text +/CLIProxyAPI/config.yaml top-level api-keys +/CLIProxyAPI/plugin-state/cpamp-usage.sqlite api_key_aliases +``` + +Admin key list and save/reset endpoints that operate on visible keys must call +sync first, not only plugin configure. This makes the Plus page refresh reflect +official CPAMP changes. + +Alias lookup must normalize both CPAMP shapes: + +```text +<sha256 hex> +sha256:<sha256 hex> +``` + +The normalized map key is always bare lowercase SHA256 hex. + +## Admin Projection + +Store may retain historical rows, but admin default response should expose only +active native policy rows: + +- `source == native_cpa` +- `source_present == true` +- `hidden == false` + +Removed native rows remain available only behind an explicit diagnostic/show +removed mode if needed. Legacy Plus rows are not part of the default current-key +view. + +## Auth Availability + +The executor owns model aliasing before host callbacks. Current production only +mapped `gpt-5.4`; user traffic now asks for `gpt-5.5`. Add `gpt-5.5` to the +upstream alias map in code/tests and production config, mapping to the same +provider-registered Codex upstream model already proven to work: +`gpt-5.3-codex-spark`. + +Downstream SSE and Plus usage projection should keep client-visible +`gpt-5.5`; host callback metadata/body uses the upstream model. + +## Rollback + +- Plugin rollback: restore prior `.so` backups from `/opt/codex-stacks/cpa/plugins/linux/amd64/` backups if new tests fail. +- Config rollback: remove the added `gpt-5.5` alias if it causes unexpected upstream behavior. +- DB rollback should not be required; rows are hidden/projected rather than deleted. diff --git a/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/implement.jsonl b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/implement.md b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/implement.md new file mode 100644 index 0000000..0184165 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/implement.md @@ -0,0 +1,28 @@ +# Implementation Plan + +1. Read applicable backend specs before editing. +2. Add/adjust Plus sync tests: + - CPAMP alias loader accepts bare SHA256 and `sha256:<hex>`. + - admin key listing triggers native sync and reflects alias changes. + - default admin list excludes `legacy_plus` and removed/hidden native rows. +3. Patch Plus implementation: + - normalize CPAMP alias hashes robustly. + - sync before admin key list and relevant strategy operations. + - filter default admin projection to current native rows. +4. Add/adjust executor test for `gpt-5.5` upstream alias preserving visible + model. +5. Patch executor alias/config behavior if needed. +6. Run local checks: + - `go test ./...` in `cpa_key_policy_plus_plugin/go`. + - `go test ./...` in `cpa_codexcont_executor_plugin/go`. + - `git diff --check`. + - `python ./.trellis/scripts/task.py validate .trellis/tasks/07-03-07-03-native-key-sync-auth-availability`. +7. Build linux/amd64 Plus and executor plugins with existing WSL Go; record + SHA256. +8. Deploy only changed `.so` files and minimal CPA config alias change. +9. Restart only `cpa`, then verify: + - plugin logs loaded/registered. + - Plus admin/default API has official active native key count and aliases. + - public `/v1/responses` with `model=gpt-5.5` succeeds. + - public usage portal and public admin blocks remain correct. +10. Commit, push branch, update PR. diff --git a/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/prd.md b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/prd.md new file mode 100644 index 0000000..3219a1a --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/prd.md @@ -0,0 +1,101 @@ +# Fix native key sync and auth availability + +## Goal + +Restore production usability for the CPA-first plugin chain: + +- `CPA Key Policy+` must mirror the official CPA/CPAMP native API key list. +- The Plus admin page must show only currently-present official native keys by default. +- Alias changes and deletion from the official CPAMP key panel must be reflected promptly. +- Requests using all enabled official keys must stop failing with + `host_call_failed: auth_unavailable: no auth available (providers=codex, model=gpt-5.5)`. + +## User Value + +The operator should manage API keys in the official CPAMP panel only. Plus is a +policy overlay and should not show stale old `cpa_...` records or stale aliases. +Users should be able to call supported Codex models with any enabled official +key. + +## Confirmed Evidence + +- Current production Plus DB contains 7 rows: 3 `legacy_plus` rows and 4 + `native_cpa` rows. +- CPAMP official alias DB contains 5 alias rows, including the new names + `alice` and `alicea`. +- Plus admin UI currently shows stale legacy rows and one native row with hash + preview instead of the CPAMP alias. +- Current production executor config has only one upstream alias: + `gpt-5.4 -> gpt-5.3-codex-spark`. +- User-reported failure is for `model=gpt-5.5`, which is not mapped to a + provider-registered upstream model in current executor config. + +## Requirements + +- Plus admin list must default to current official native keys only. +- Old `legacy_plus` / `cpa_...` rows must not appear in the ordinary Plus admin + key strategy table. +- Official native key alias updates must be visible after refresh/re-entering + the Plus page. +- Official native key deletion must remove the row from the ordinary table on + the next sync/read. Historical usage may remain in DB, but stale key rows must + not be part of the active strategy list. +- Sync must accept CPAMP `api_key_aliases.api_key_hash` whether stored as bare + SHA256 hex or `sha256:<hex>`. +- Admin APIs that read keys should trigger native sync before listing so the UI + does not wait for CPA restart or a user login. +- Executor model routing must map client-visible `gpt-5.5` to the currently + available Codex upstream model, preserving downstream/user-visible model name. +- Public admin/resource paths must remain blocked. +- Do not modify official CPA/CPAMP source. +- Do not store or print raw `sk-...` keys, bearer tokens, cookies, or encrypted + reasoning. + +## Acceptance Criteria + +- [x] Production Plus admin table shows exactly the official active native keys + by default, matching the official panel count and aliases. +- [x] Creating or aliasing a native key in CPAMP is reflected in Plus after + refresh without restarting CPA. +- [x] Deleting a native key in CPAMP hides/removes it from the Plus default + table after refresh without showing stale old `cpa_...` rows. +- [x] Plus tests cover admin list native-sync refresh, legacy rows hidden from + default strategy list, and bare-hash CPAMP alias reads. +- [x] Executor tests cover `gpt-5.5` aliasing to the configured upstream Codex + model while preserving downstream model `gpt-5.5`. +- [x] Production `/v1/responses` smoke with `model=gpt-5.5` succeeds with an + enabled official key. +- [x] Production `cpa-usage.konbakuyomu.us` still works. + +## Final Evidence + +- Local code branch pushed: `837b48a fix(cpa): sync native keys and restore Codex model routing`. +- Production Plus admin API returned exactly 4 current native CPA rows by default: + one disabled new native row plus `kuma的官key`, `QQ的官key`, and + `阿伟的官key`. +- Production CPA config contains 4 native API keys and the executor `gpt-5.5` + upstream alias. +- Local tests passed before deploy: + `go test ./...` in `cpa_key_policy_plus_plugin/go`, + `go test ./...` in `cpa_codexcont_executor_plugin/go`, `git diff --check`, + and Trellis task validation. +- Initial production smoke with `model=gpt-5.5` reached CPA host callback and + returned `401 authentication_error / auth_unavailable` with an invalidated + Codex OAuth token message, proving the remaining all-key failure was upstream + OAuth account state rather than native key sync or Plus policy. +- Replaced the active production Codex OAuth auth JSON with a fresh + `chatgpt`/Codex login-derived auth file from the same account, preserved the + invalidated auth file under `auths-disabled`, and restarted only `cpa`. +- After restart, CPA logs reported `1 auth entries`; both + `cpa-codexcont-executor` and `cpa-key-policy-plus` loaded and registered. +- Non-streaming production smoke: + `POST http://127.0.0.1:8317/v1/responses` with `model=gpt-5.5` returned + HTTP `200`, `status=completed`, `model=gpt-5.5`, text `OK`. +- Streaming production smoke: + `model=gpt-5.5` returned HTTP `200`, included `response.completed`, did not + include `response.incomplete`, and contained the expected `STREAM_OK` text. +- Public route checks after restart: + `https://cpa-usage.konbakuyomu.us/` returned `200`; public + `/v0/resource/plugins/cpa-key-policy-plus/admin`, + `/v0/resource/plugins/cpa-codexcont-executor/admin`, `/codexcont/`, + `/governor/`, and `/management.html` on `cpa.konbakuyomu.us` returned `404`. diff --git a/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/task.json b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/task.json new file mode 100644 index 0000000..6d0fd9d --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-07-03-native-key-sync-auth-availability/task.json @@ -0,0 +1,26 @@ +{ + "id": "07-03-native-key-sync-auth-availability", + "name": "07-03-native-key-sync-auth-availability", + "title": "Fix native key sync and auth availability", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P1", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-03", + "completedAt": "2026-07-04", + "branch": null, + "base_branch": "codex/codexcont-executor-migration", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/check.jsonl b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/design.md b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/design.md new file mode 100644 index 0000000..c7eadce --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/design.md @@ -0,0 +1,99 @@ +# Design + +## Architecture Boundary + +`cpa-key-policy-plus` remains the Key Policy and user-portal authority. The new +executor plugin owns only CodexCont continuation execution for streaming +Responses requests. This avoids conflating `cpa-usage.konbakuyomu.us` with the +sidecar replacement. + +The intended cutover path is: + +```text +Codex -> Caddy -> CPA + -> cpa-key-policy-plus frontend auth / RPM / quota + -> cpa-codexcont-executor model route + executor + -> CPA host model stream + -> upstream Responses API +``` + +## Executor Flow + +- `model.route` handles only enabled, streaming Responses-style requests. +- `executor.execute_stream` opens the first upstream stream through the CPA host + callback, reads upstream SSE chunks, and emits one folded downstream SSE + stream through `host.stream.emit`. +- The folding state machine mirrors the Python sidecar: forward first + lifecycle events, stream reasoning items, buffer tentative message/function + output, inspect terminal usage, continue on `518*n-2` when encrypted reasoning + is replayable, and emit one reconstructed terminal response. +- Continuation rounds rebuild the request body from the original input plus + replayed reasoning and a hidden commentary marker. `previous_response_id` is + dropped because state is carried explicitly. + +## Upstream Model Alias + +The executor may rewrite the internal upstream model while preserving the +client-visible model downstream. This is plugin-owned routing state, not a CPA +or CPAMP fork. The first production alias is: + +```yaml +upstream_model_aliases: + gpt-5.4: gpt-5.3-codex-spark +``` + +When a request enters as `gpt-5.4`, the host model callback receives +`gpt-5.3-codex-spark` in both the callback metadata and request body. The +folded downstream SSE and stored safe summary keep `gpt-5.4`, so the client and +Plus usage view do not learn or depend on the internal provider alias. + +## Host Stream Compatibility + +CPA host callbacks may return SSE as logical line chunks without trailing +newlines, for example one read containing only `event: response.created`. The +executor parser treats `event:`, `data:`, comment, and blank line chunks as +complete SSE lines. This prevents false `response.incomplete` results when the +host stream transport splits events differently than the old Python sidecar. + +## Summary Bridge + +The executor plugin writes safe request summaries to its own SQLite store. +Plus may optionally read that store for `/user/api/codexcont`, filtered by the +current Key Policy key id. Plus never writes executor state, and executor never +owns Plus user sessions or quota state. + +If the summary bridge is unavailable, Plus returns an empty/degraded protection +summary while keeping `/user/api/me`, `/user/api/usage`, and `/user/api/events` +healthy. + +## CPAMP Monitor + +The executor plugin also owns the read-only CPAMP-side CodexCont monitor that +lets operators replace Governor's protection dashboard. It registers an admin +resource/menu for executor observability only, backed by safe status and +summary endpoints. The monitor polls executor summaries for rolling updates; it +does not expose `/user`, `/user/api/*`, key editing, quota controls, request +bodies, response bodies, or encrypted reasoning. + +The current CPA/CPAMP resource-menu path expects plugin resources to come from +plugins that declare the same resource-adjacent capabilities used by Plus and +Governor. The executor therefore declares `frontend_auth_provider=true` and +`usage_plugin=true` only as resource-registration compatibility shims. +`frontend_auth.authenticate` always returns unauthenticated, and `usage.handle` +is an explicit no-op. These shims must not authenticate requests, store usage, +price costs, mutate quotas, or become a billing source. + +## Safety + +The executor plugin must not store or return request bodies, response bodies, +Authorization headers, raw keys, cookies, OAuth tokens, or encrypted reasoning. +Only safe counters and status fields are persisted: request id, key id if known, +model, protection status, rounds, reasoning counters, continuation count, +stopped/failure reason, timestamps, duration, and safe diagnostics such as +upstream model alias evidence, stream id presence, and byte counts. + +## Compatibility + +`cpa-key-policy-plus` keeps `codexcont_route` deprecated/off. The executor +plugin owns the new route switch, so disabling continuation protection is a +one-setting operation without changing the user portal. diff --git a/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/implement.jsonl b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/implement.md b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/implement.md new file mode 100644 index 0000000..a689640 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/implement.md @@ -0,0 +1,390 @@ +# Implementation Plan + +1. Reuse the existing Governor plugin scaffolding as the base for a dedicated + executor-only plugin, but remove user-portal ownership from the executor + concept and naming. +2. Add a Go continuation package with the old Python sidecar behavior: + truncation math, SSE parse/serialize helpers, request payload rebuild, + usage aggregation, terminal reconstruction, and summary projection. +3. Wire `model.route` and `executor.execute_stream` so the route switch handles + only streaming Responses requests and falls back cleanly when disabled. +4. Persist safe summaries in the executor store and add a read-only Plus + summary bridge from `cpa-key-policy-plus` to the executor store path. +5. Add the executor CPAMP admin monitor resource for read-only rolling + CodexCont protection summaries, replacing Governor's monitoring role + without registering any user portal. +6. Update docs/specs to state that `cpa-usage.konbakuyomu.us` is Plus-owned and + executor-only plugins replace only the Docker CodexCont sidecar. +7. Add focused Go tests for executor folding, executor monitor registration, + and Plus summary degradation. +8. Run: + - `go test ./...` in the executor plugin package. + - `go test ./...` in `cpa_key_policy_plus_plugin/go`. + - `git diff --check`. + +## Risk Points + +- Stream folding must emit exactly one terminal event and keep downstream + sequence numbers monotonic. +- Tentative output from truncated rounds must never leak before a continuation + decision. +- Summary bridging must fail soft so user quota and usage APIs are unaffected. +- The plugin must not accidentally register `cpa-usage` user resources. +- The CPAMP monitor must be read-only observability; key/quota management stays + in Plus and ordinary user usage stays on `cpa-usage`. +- CPA/CPAMP currently requires resource-adjacent capabilities for plugin + resource menu registration. Executor uses non-exclusive + `frontend_auth_provider=true` and `usage_plugin=true` only as compatibility + shims; `frontend_auth.authenticate` returns unauthenticated and + `usage.handle` is no-op. + +## Execution Evidence + +- Added `cpa_codexcont_executor_plugin` as a separate executor-only CPA plugin. +- Added Go folding coverage for: + - two-round auto continuation, + - missing encrypted reasoning, + - upstream EOF, + - continuation open error, + - `max_continue`, + - monotonic sequence numbers and reconstructed metadata. +- Added plugin registration/route-switch tests confirming the executor plugin + does not expose user resources or usage-portal ownership. +- Added a reconfigure regression: CPA calls `plugin.reconfigure` after the + first load and still expects full plugin metadata/capabilities. Returning + only `{"configured": true}` makes CPA mark the plugin unregistered and drops + its CPAMP resource routes. +- Added upstream model alias support so a client-visible model can be mapped to + a provider-registered internal model without changing downstream SSE or safe + summaries. Current production alias: + `gpt-5.4 -> gpt-5.3-codex-spark`. +- Added a host-stream parser regression for CPA line-sized SSE chunks without + trailing newlines. This fixed the false `response.incomplete` result seen + when the host callback returned chunks such as `event: response.created`. +- Added safe diagnostics to executor summaries: requested model, upstream + model, rewritten body model, stream id presence, read counts, first/last read + byte counts, and chunk status. Diagnostics do not include raw request bodies, + raw response bodies, keys, or encrypted reasoning. +- Added Plus `codex_summary_db_path` read-only bridge and tests proving + executor summaries are filtered by the current key. +- Validation: + - `go test ./...` in `cpa_codexcont_executor_plugin/go`: passed. + - `go test ./...` in `cpa_key_policy_plus_plugin/go`: passed. + - `git diff --check`: passed with only CRLF conversion warnings. + +## Local Build Toolchain Evidence + +- Do not download Go again for WSL/Linux plugin builds unless both reusable + toolchains are missing and the user explicitly approves a new install. +- Current WSL check: + - plain `go` is not present in WSL `PATH`. + - preferred reusable Go works: + `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go` + -> `go version go1.22.6 linux/amd64`. + - project-scoped fallback also works: + `/mnt/d/Dev/20_Software/_LocalRuntime/CodexCont/go-sdk-1.22.6/bin/go` + -> `go version go1.22.6 linux/amd64`. + - cached tarball exists: + `/mnt/d/Dev/20_Software/_LocalRuntime/go/downloads/go1.22.6.linux-amd64.tar.gz`. + - `/tmp` currently has no `codex-go*` temporary Go directories. +- Future build/test commands should call the preferred Go path explicitly, for + example: + `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go test ./...`. +- Follow-up validation used the preferred reusable Go path, not a WSL download: + - `go test ./...` in `cpa_codexcont_executor_plugin/go`: passed. + - `go test ./...` in `cpa_key_policy_plus_plugin/go`: passed. + - `git diff --check`: passed with CRLF conversion warnings only. + - `python ./.trellis/scripts/task.py validate .trellis/tasks/07-03-codexcont-executor-plugin`: + passed. + +## Rollout Checklist + +1. Build linux/amd64 plugin artifacts for `cpa-codexcont-executor` and, when + deploying the summary bridge, `cpa-key-policy-plus`; record SHA256 hashes. +2. Install the executor plugin into CPA with `route_enabled: false` first. + This verifies plugin loading without changing live `/v1/responses` routing. +3. Confirm existing public contracts still work: + - `https://cpa-usage.konbakuyomu.us/` login and usage APIs. + - authenticated `https://cpa.konbakuyomu.us/v1/models`. + - existing Codex request path still succeeds through the known-good route. +4. Enable executor routing only for a controlled smoke window, then test a real + streaming `/v1/responses` request and inspect safe executor summaries. +5. After the CPA-first executor path is stable, remove the old Caddy special + route to `codexcont:8787` and stop the Docker sidecar. + +## Server Staging Evidence + +- Built linux/amd64 artifacts with Go 1.22.6 in WSL and verified both are ELF + x86-64 shared objects. +- Artifact hashes: + - `cpa-codexcont-executor.so`: + `041f06bc0c7c5ea7edc3082b86fd484364f7ee83e6411a8993663ab67b417951` + - `cpa-key-policy-plus.so`: + `9fc597a67b852e3ec212e6d3f6e4dc08d0ef6ee728df25fb4ac4b99e5aac75e8` +- SJC backup before plugin replacement: + `/opt/codex-stacks/backups/cpa-codexcont-executor-20260703-105924`. +- Installed both artifacts into + `/opt/codex-stacks/cpa/plugins/linux/amd64/`. +- Added executor config with `route_enabled: false` and Plus + `codex_summary_db_path` bridge to the executor SQLite store. +- Restarted only the `cpa` container. `caddy-edge`, `codexcont`, `cpamp`, + `cpa-admin-proxy`, and `cpa-usage-portal` kept running. +- Server validation: + - local CPA `http://127.0.0.1:8317/healthz`: HTTP `200`. + - public `https://cpa.konbakuyomu.us/healthz`: HTTP `200`. + - CPA logs show `cpa-codexcont-executor` and `cpa-key-policy-plus` loaded + and registered. + - public executor resource path: + `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-codexcont-executor/status` + returned HTTP `404`. + - public Plus admin resource path: + `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-key-policy-plus/admin` + returned HTTP `404`. + - public old CodexCont dashboard path: + `https://cpa.konbakuyomu.us/codexcont/` returned HTTP `404`. + - public `https://cpa-usage.konbakuyomu.us/` returned HTTP `200`; HTML no + longer contains `rangeSelect` or `<select>`. + +## Server Monitor Staging Evidence + +- User clarified executor should also replace Governor's CPAMP-side realtime + rolling CodexCont monitor. Plan/docs/spec were updated so executor owns only + the read-only monitor; Plus still owns `cpa-usage`, keys, quota, RPM, and + user usage APIs. +- Added executor admin resource: + `/v0/resource/plugins/cpa-codexcont-executor/admin`, with safe status and + summaries APIs under `/admin/api/status` and `/admin/api/summaries`. +- Root cause found during staging: executor `plugin.reconfigure` returned only + `{"configured": true}`. CPA v7.2.48 reuses the registration decoder for + `plugin.reconfigure`, so later config refreshes logged + `returned invalid metadata or no capabilities` and removed executor from the + active plugin snapshot. Fix: both `plugin.register` and + `plugin.reconfigure` now return the same full registration object. +- Final deployed executor artifact: + `edd8dad11a3843672f802aee8412ae02a21421e7807dc569c509a7529875255a`. +- SJC backups before monitor replacements: + `/opt/codex-stacks/backups/cpa-codexcont-executor-monitor-20260703-125630`, + `/opt/codex-stacks/backups/cpa-codexcont-executor-monitor-20260703-131110`, + and + `/opt/codex-stacks/backups/cpa-codexcont-executor-monitor-20260703-132133`. +- Final staging validation: + - local CPA `http://127.0.0.1:8317/healthz`: HTTP `200`. + - local executor resource `/v0/resource/plugins/cpa-codexcont-executor/admin`: + HTTP `200`, `Cache-Control: no-store`, HTML contains `实时滚动监控`. + - local executor status API: + `/v0/resource/plugins/cpa-codexcont-executor/admin/api/status`: HTTP `200` + with `route_enabled:false`. + - local executor summaries API: + `/v0/resource/plugins/cpa-codexcont-executor/admin/api/summaries?limit=5`: + HTTP `200`, currently empty summaries. + - public executor resource: + `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-codexcont-executor/admin`: + HTTP `404`. + - public CPA health: `https://cpa.konbakuyomu.us/healthz`: HTTP `200`. + - public Plus usage page: `https://cpa-usage.konbakuyomu.us/`: HTTP `200`; + HTML contains no `rangeSelect` and no `<select>`. + +## Server Executor Smoke Evidence + +- Root cause found during controlled executor smoke: the public/client model + `gpt-5.4` was not the provider-registered upstream model on SJC. The executor + now owns a plugin-only alias from `gpt-5.4` to `gpt-5.3-codex-spark`; official + CPA/CPAMP code and Plus ownership were not changed. +- Root cause found after the first alias test: CPA host callbacks can deliver + SSE as line-sized chunks without trailing newlines. The parser now accepts + standalone `event:`, `data:`, comment, and blank line chunks, so a valid + upstream stream no longer collapses into `response.incomplete`. +- Final deployed executor artifact on SJC: + `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-codexcont-executor.so` + with SHA256 + `499538733a37c77ff11b39bbb1818eaa1b748b87110988a4c97b1ed8997c1b81`. +- Current SJC executor config after smoke: + - `enabled: true`. + - `route_enabled: false`. + - `upstream_model_aliases.gpt-5.4: gpt-5.3-codex-spark`. + - status API reported `alias_count: 1`. +- Controlled internal smoke temporarily set `route_enabled: true` and sent a + local streaming request to `http://127.0.0.1:8317/v1/responses`. + Result: + - HTTP `200`. + - terminal event `response.completed`. + - summary `protection=protected_clean`. + - downstream stream and summary preserved `model=gpt-5.4`. + - diagnostics showed upstream callback `model/body_model=gpt-5.3-codex-spark`. +- After smoke, `route_enabled` was restored to `false`. Public Caddy + `/v1/responses` has not been switched away from the old Docker sidecar yet. +- Temporary Plus smoke key was deleted after validation; follow-up check showed + the raw key temp file was absent. +- Final public-safety checks: + - local CPA health: HTTP `200`. + - public CPA health: HTTP `200`. + - `https://cpa-usage.konbakuyomu.us/`: HTTP `200`. + - usage page has no `rangeSelect`, no `<select>`, and contains `24 小时`. + - public executor admin path: + `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-codexcont-executor/admin` + returned HTTP `404`. + - old Docker `codexcont` sidecar is still running until public cutover is + explicitly approved and verified. + +## Current Remote State Recheck + +- Read-only SJC check used SSH alias `sjc-snap`; the plain `sjc` alias is not + configured in this Windows SSH profile. +- Current deployed artifact hashes: + - `cpa-key-policy-plus.so`: + `e261cfbe8777c43ec2d45ce76f4a094ae5656bdbb3ad33513c9ba2c0322b2369`. + This is the AuthID usage-mapping fix artifact. + - `cpa-codexcont-executor.so`: + `499538733a37c77ff11b39bbb1818eaa1b748b87110988a4c97b1ed8997c1b81`. +- Current public Caddy route is still the safety rollback route: + `reverse_proxy @responses codexcont:8787`, then ordinary fallback + `reverse_proxy cpa:8317`. Therefore public `/v1/responses` still goes + through the old Docker CodexCont sidecar first. +- Current CPA executor config has `route_enabled: true`, but public traffic is + still bypassing it because Caddy special-cases `/v1/responses` to the + sidecar. +- Current container state: + - `cpa` running. + - `caddy-edge` running. + - `cpamp` healthy. + - `codexcont` old sidecar still running. + - `cpa-usage-portal` running. +- Current health/portal checks: + - SJC root disk remains tight: about `1.1G` free, `90%` used. + - local CPA health `http://127.0.0.1:8317/healthz`: HTTP `200`. + - public `https://cpa-usage.konbakuyomu.us/`: HTTP `200`. + - usage page HTML has no `rangeSelect`, no `<select>`, and contains + `24 小时`. + +## Production Cutover Evidence + +- Fixed the duplicate CPAMP executor menu: + - Root cause: the executor management route + `/plugins/cpa-codexcont-executor/status` set + `Menu: CodexCont Executor`, so CPAMP rendered it as a second sidebar page + that displayed raw JSON. + - Fix: only `/admin` resource sets the menu label; status/summaries remain + internal management routes with no `Menu`. + - Deployed executor artifact: + `a4469e59dc2c6255ef7891711e8410d257615c9f5dae17e057c8c688c6096b86`. + - Validation: local CPA + `/v0/resource/plugins/cpa-codexcont-executor/admin` returned HTTP `200` + and contained `实时滚动监控`; local resource + `/v0/resource/plugins/cpa-codexcont-executor/status` returned HTTP `404`. +- Disabled the old `cpa-governor` plugin in + `/opt/codex-stacks/cpa/config.yaml`: + - `cpa-governor.enabled: false`. + - After CPA restart, logs showed `cpa-codexcont-executor` and + `cpa-key-policy-plus` registered, with no `cpa-governor` registration. + - Validation: local + `/v0/resource/plugins/cpa-governor/admin` returned HTTP `404`; public + `https://cpa.konbakuyomu.us/v0/resource/plugins/cpa-governor/admin` + returned HTTP `404`. +- Retired the old Governor plugin from the CPAMP installed-plugin surface: + - Moved the single old artifact + `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-governor.so` into + `/opt/codex-stacks/backups/cpa-governor-retired-20260703-193559/`. + - Removed only the `plugins.configs.cpa-governor` block from + `/opt/codex-stacks/cpa/config.yaml`; the legacy Governor state DB remains + available for Plus read-only import/audit paths. + - Restarted `cpa` and `cpamp`. + - Validation: CPAMP management plugin list now returns only + `cpa-codexcont-executor` with `menus=['CodexCont Executor']` and + `cpa-key-policy-plus` with `menus=['CPA Key Policy+']`; `cpa-governor` + is absent from the installed list. + - Validation: local executor admin resource stayed HTTP `200`, executor + status resource stayed HTTP `404`, Governor admin stayed HTTP `404`, + public CPA health stayed HTTP `200`, public usage portal stayed HTTP + `200`, and public executor/Governor admin resources stayed HTTP `404`. +- Cut public Caddy `/v1/responses` to CPA-first: + - Removed the special route + `reverse_proxy @responses codexcont:8787`. + - Current `cpa.konbakuyomu.us` site block falls through to + `reverse_proxy cpa:8317`. + - `docker exec caddy-edge caddy validate --config /etc/caddy/Caddyfile`: + passed. + - `docker exec caddy-edge caddy reload --config /etc/caddy/Caddyfile`: + succeeded. +- Fixed Plus usage projection for executor aliases: + - First CPA-first smoke proved usage rows were recorded, but model projected + as internal `gpt-5.3-codex-spark`. + - Plus now resolves records by `AuthID` first and maps known executor usage + alias `gpt-5.3-codex-spark -> gpt-5.4` into `model` and + `requested_model`, while preserving `actual_model`. + - Final deployed Plus artifact: + `fefa58c4a8bc6c99de55ba590807cbb2489af3b5e1b15cbfeb5c9df94883d381`. +- Production smoke before sidecar stop: + - Public `https://cpa.konbakuyomu.us/v1/responses`: HTTP `200`. + - Stream contained `response.completed`. + - Stream did not contain `response.incomplete`. + - Plus `usage_events` recorded the request for the temporary key. + - Final corrected usage projection: + `model=gpt-5.4`, `requested_model=gpt-5.4`, + `actual_model=gpt-5.3-codex-spark`. +- User portal validation: + - `https://cpa-usage.konbakuyomu.us/`: HTTP `200`. + - User API with temporary key: + `/session`, `/me`, `/usage?range=24h`, and + `/events?range=24h&limit=100` all returned HTTP `200`. + - `/events` showed the latest smoke row as `gpt-5.4` with internal model + preserved separately. +- Old Docker sidecar stopped: + - Ran `docker compose stop codexcont` in `/opt/codex-stacks/codexcont`. + - `docker ps` no longer showed a running `codexcont` container. + - `cpa`, `caddy-edge`, `cpamp`, and `cpa-usage-portal` stayed running. +- Old chain cleanup after executor cutover: + - Created backup directory + `/opt/codex-stacks/backups/old-codexcont-chain-cleanup-20260703-195045/` + with the prior CPA config, admin Caddyfile, and old CodexCont compose + file. + - Removed the stopped Docker container `codexcont` with explicit + `docker rm codexcont`. + - Removed the old sidecar image `codexcont-codexcont:latest` with explicit + `docker image rm codexcont-codexcont`; no Docker prune or bulk cleanup was + used. + - Renamed + `/opt/codex-stacks/codexcont/docker-compose.yaml` to + `docker-compose.yaml.retired-20260703-195045`, so a default + `docker compose up` in that directory can no longer recreate the old + sidecar. + - Updated `/opt/codex-stacks/cpa/config.yaml` so Plus has + `codexcont_enabled: false` and no longer carries the old + `codexcont_url: http://codexcont:8787` line. The executor SQLite bridge + remains the source for safe protection summaries. + - Updated `/opt/codex-stacks/cpa-admin-tunnel/Caddyfile` so retired + `/governor*`, `/governor-user*`, and `/codexcont*` admin paths return + `404`; no route now reverse-proxies to `codexcont:8787` or + `cpa-governor`. + - `docker exec cpa-admin-proxy caddy validate --config + /etc/caddy/Caddyfile`: passed. + - Restarted `cpa`, `cpa-admin-proxy`, and `cpamp`. + - Validation: + - `docker ps -a` and `docker image ls` no longer show `codexcont`. + - Live CPA config contains `codexcont_enabled: false` and no + `codexcont_url`. + - Live admin Caddyfile contains no `codexcont:8787`, `cpa-governor`, or + `/governor/codexcont/admin` references. + - Local/public CPA health and `https://cpa-usage.konbakuyomu.us/` all + returned HTTP `200`. + - Admin proxy `/governor/` and `/governor/codexcont/admin/status` returned + HTTP `404`. + - Executor admin resource remained HTTP `200`. + - Public `/codexcont/`, `/governor/`, executor resource, and Governor + resource all returned HTTP `404`. + - CPAMP plugin list still shows only `cpa-codexcont-executor` with + `menus=['CodexCont Executor']` and `cpa-key-policy-plus` with + `menus=['CPA Key Policy+']`. +- Production smoke after sidecar stop: + - Public `https://cpa.konbakuyomu.us/v1/responses`: HTTP `200`. + - Stream contained `response.completed`. + - Stream did not contain `response.incomplete`. + - New Plus usage row: + `model=gpt-5.4`, `requested_model=gpt-5.4`, + `actual_model=gpt-5.3-codex-spark`. +- Temporary cutover key cleanup: + - Deleted the temporary key row from Plus `keys`, plus related + `reset_watermarks` and `active_sessions`. + - Preserved `usage_events` history rows for audit evidence. + - Removed explicit sensitive temp files: + `/tmp/cpa-codexcont-cutover-key.txt` and + `/tmp/cpa-codexcont-cutover-key-id.txt`. diff --git a/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/prd.md b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/prd.md new file mode 100644 index 0000000..721bc29 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/prd.md @@ -0,0 +1,65 @@ +# CodexCont executor plugin migration + +## Goal + +Replace the Docker-hosted Python CodexCont sidecar with an executor-only CPA +plugin while keeping Key Policy Plus as the owner of the ordinary user portal, +keys, quotas, RPM, and usage APIs. + +## Requirements + +- `cpa-key-policy-plus` continues to own `https://cpa-usage.konbakuyomu.us/`, + user sessions, `/user/api/*`, quota windows, RPM, and usage details. +- The new executor plugin must not register or expose an ordinary user usage + portal. It replaces only the old CodexCont Docker sidecar behavior for + streaming `/v1/responses` continuation protection. +- The public execution chain after cutover is: + `Codex -> Caddy -> CPA -> Key Policy Plus auth/quota -> CodexCont Executor + plugin -> upstream model`. +- The executor plugin must implement the old CodexCont stream-folding behavior: + `518*n-2` reasoning-token detection, encrypted reasoning replay, hidden + commentary continuation marker, max continuation cap, terminal event + reconstruction, and safe metadata. +- The executor plugin must have a single on/off config switch. When disabled, + CPA should continue using the normal upstream path without continuation + protection. +- The executor plugin should replace Governor's CPAMP-side CodexCont realtime + monitoring role with a read-only admin monitor resource. This monitor may + show safe rolling request summaries and executor health, but must not become + a user portal or key/quota control plane. +- Safe protection summaries may be shown through Plus, but summary transport + must not change ownership of `cpa-usage` and must not expose request bodies, + response bodies, keys, cookies, OAuth tokens, or encrypted reasoning. + +## Acceptance Criteria + +- [x] A CPA plugin named for CodexCont executor behavior exists separately from + `cpa-key-policy-plus` and does not register user/admin portal resources + beyond internal/management observability needed for executor health. +- [x] The executor plugin exposes a CPAMP admin menu/resource for read-only + realtime rolling CodexCont monitoring, replacing Governor's daily + protection-monitoring role without exposing `/user` or `/user/api/*`. +- [x] With the executor switch disabled, model routing returns unhandled and + existing CPA behavior remains available. +- [x] With the executor switch enabled, streaming Responses requests are routed + to the plugin executor and folded into one downstream stream. +- [x] Unit tests cover clean passthrough, auto-continued two-round folding, + `max_continue`, missing encrypted reasoning, upstream EOF, upstream + error, sequence-number monotonicity, and reconstructed metadata. +- [x] Plus user page/API contracts remain unchanged: + `/user/api/session`, `/user/api/me`, `/user/api/usage?range=24h`, + `/user/api/events?range=24h&limit=100`, and `/user/api/codexcont`. +- [x] Protection summary failures degrade only the summary display; user login, + quota, usage, and events still work. +- [x] `go test ./...` passes in both the executor plugin and Key Policy Plus + plugin packages. +- [x] Public Caddy `/v1/responses` is cut over to CPA-first executor routing, + then the old Docker CodexCont sidecar is stopped. This is intentionally + a separate production approval gate. + +## Notes + +- Do not modify official CPA or CPAMP. +- Do not move `cpa-usage.konbakuyomu.us` to the executor plugin. +- Old Docker/Caddy sidecar removal is a production rollout step after local and + server validation, not part of the first local code change. diff --git a/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/task.json b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/task.json new file mode 100644 index 0000000..88ab5d0 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-codexcont-executor-plugin/task.json @@ -0,0 +1,26 @@ +{ + "id": "codexcont-executor-plugin", + "name": "codexcont-executor-plugin", + "title": "CodexCont executor plugin migration", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-03", + "completedAt": "2026-07-03", + "branch": null, + "base_branch": "codex/codexcont-executor-migration", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/check.jsonl b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/check.jsonl new file mode 100644 index 0000000..8c3bb7c --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/check.jsonl @@ -0,0 +1,3 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} +{"file": "cpa_key_policy_plus_plugin/go/main_test.go", "reason": "policy denial, admin UI, user portal tests"} +{"file": "cpa_key_policy_plus_plugin/go/internal/policyplus/policyplus_test.go", "reason": "native sync and key hint tests"} diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/design.md b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/design.md new file mode 100644 index 0000000..58130df --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/design.md @@ -0,0 +1,81 @@ +# Design + +## Ownership Boundary + +CPA/CPAMP remains the key lifecycle authority. Plus becomes an overlay policy +database keyed by the native API key hash. Alias is display/template metadata, +not a ledger identity. + +```text +CPA config api-keys + CPAMP aliases + -> Plus native key sync + -> Plus policy rows keyed by native_<hash preview> + -> frontend auth / usage / user portal +``` + +## Native Key Identity + +- Normalize native key with the existing submitted-key normalizer. +- Compute SHA256 and store it as `sha256:<hex>`. +- Generate ID as `native_<preview>`, where preview is the existing safe + `HashPreview` without raw key material. +- Read alias from CPAMP `api_key_aliases.api_key_hash`; fallback display is the + safe preview. +- `Name` follows the alias for read-only display. Admin edits do not change it. + +## Sync Rules + +When a native key exists in CPA config: + +- Existing row: update hash, preview, alias/name, `source=native_cpa`, + `source_present=true`, `hidden=false`; preserve policy fields. +- New row with no unique same-alias removed template: insert disabled with + empty/default policy. +- New row with exactly one removed same-alias template: copy policy fields and + enabled state into the new ID, set `inherited_from`, and leave usage/reset + rows under the old ID. +- Multiple removed same-alias templates: insert disabled with + `inherit_conflict=true` so UI can ask for manual policy selection later. + +When a stored native row is absent from CPA config: + +- Set `source_present=false`, `enabled=false`, `hidden=true`. +- Keep usage events, reset watermarks, and Codex protection summaries. + +## Policy Decision + +A single `PolicyDecision` projection owns auth allow/deny state: + +- `Allowed`, `StatusCode`, `Type`, `Code`, `Message`, `Window`, `UsedUSD`, + `LimitUSD`, `KeyID`, and safe `KeyName`. +- Missing/disabled/source-removed/model denials are deterministic before rate + counters. +- RPM consumes one in-memory bucket entry only when allowed by earlier checks. +- Fee quota checks read current usage sums and deny when `used >= limit`. + +Frontend auth still returns `Authenticated=false` for denied requests because +the CPA frontend-auth ABI has no custom response body. To make the user see the +real reason, Plus also exposes a deny-only model route/executor. The router +handles only requests whose bearer key maps to a denied Plus policy decision; +the executor returns an OpenAI-compatible 429 body or streaming SSE error. + +## UI Contract + +Admin page is renamed to "Key 策略": + +- Shows native alias/name, preview, source status, inherited/conflict state, + enabled, RPM, models, prices, and quota windows. +- Removed rows are hidden by default and can be shown with a toggle. +- No Plus-side create/delete/rotate/copy-full-key/rename lifecycle controls. + +User page: + +- Accepts the existing header transport but text/hints refer to native + `sk-...` keys. +- `cpa_...` inputs return migrated/retired guidance. + +## Compatibility + +Old `key_policy_state_path` import remains as legacy migration support, but +native sync is the preferred source. Existing usage APIs and executor summary +bridge stay unchanged. diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/implement.jsonl b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/implement.jsonl new file mode 100644 index 0000000..9bc652b --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/implement.jsonl @@ -0,0 +1,6 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} +{"file": "cpa_key_policy_plus_plugin/go/internal/policyplus/models.go", "reason": "native key fields and safe projections"} +{"file": "cpa_key_policy_plus_plugin/go/main.go", "reason": "structured policy decisions and deny-only 429 route"} +{"file": "cpa_key_policy_plus_plugin/go/internal/policyplus/store.go", "reason": "native CPA config sync, alias DB reader, lifecycle state"} +{"file": "cpa_key_policy_plus_plugin/go/assets/admin.html", "reason": "policy-only admin UI"} +{"file": "cpa_key_policy_plus_plugin/go/assets/user.html", "reason": "native sk key login guidance"} diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/implement.md b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/implement.md new file mode 100644 index 0000000..66d2fd7 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/implement.md @@ -0,0 +1,104 @@ +# Implementation Plan + +1. Add native-key fields, schema migration, and safe projections. +2. Add native CPA config parser, CPAMP alias SQLite reader, and sync routine. +3. Wire config fields and configure-time sync. +4. Replace boolean RPM/quota checks with structured `PolicyDecision`. +5. Enable deny-only model routing/executor responses for explicit 429 payloads. +6. Change user login hints from `cpa_...` to native `sk-...`. +7. Simplify admin UI to policy editing only and retire create/delete endpoints. +8. Add/update tests for sync, policy decisions, UI strings, and denial bodies. +9. Validate with: + - `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go test ./...` + - `git diff --check` + - `python ./.trellis/scripts/task.py validate .trellis/tasks/07-03-cpa-key-policy-native-keys` + +## Rollout Notes + +- Deploy Plus first with native sync paths configured but review disabled new + rows before enabling user traffic. +- After deployment, verify cpa-usage login/usage/protection and a deliberately + over-limit `/v1/responses` call. +- No official CPA/CPAMP binary changes are part of this task. + +## Implementation Evidence + +- Implemented native CPA key sync in `cpa_key_policy_plus_plugin/go`: + top-level CPA `api-keys` are read from `native_keys_config_path`, CPAMP + aliases are read from `cpamp_alias_db_path`, and Plus stores only safe + hash/preview/alias/source metadata. +- Reworked Plus admin UI from key lifecycle management to key strategy editing: + no create/delete/rotate/full-key-copy controls remain in the page. +- Added structured policy denial decisions for missing policy, removed source, + disabled key, model allowlist, RPM, and 5H/24H/7D/month quota windows. +- Added deny-only model routing/executor responses so over-limit model calls + return OpenAI-compatible JSON/SSE payloads with explicit + `rate_limit_exceeded` details. Production validation showed the official CPA + executor ABI does not let a plugin set the final HTTP status/header on + `/v1/responses`, so the public response body is the reliable client-facing + contract unless CPA core is changed. +- Updated user login guidance from legacy `cpa_...` keys to CPA native + `sk-...` keys while keeping `/user/api/*` usage portal ownership in Plus. + +## Rollout Evidence + +- Built the production Plus plugin with the existing WSL Go toolchain: + `go version go1.22.6 linux/amd64`. +- Artifact check: `file` reported an ELF 64-bit x86-64 shared object and + `sha256sum` reported + `ea4c84348545826548fe1b449afde9803ae1b5eebcc2b1391389b07426055b87`. +- Deployed to SJC at + `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-key-policy-plus.so`; container + path `/CLIProxyAPI/plugins/linux/amd64/cpa-key-policy-plus.so` reports the + same SHA after `docker restart cpa`. +- CPA logs after restart show both `cpa-codexcont-executor` and + `cpa-key-policy-plus` loaded and registered. +- Production config keeps Plus as the user portal owner and disables the old + sidecar route: + `codexcont_enabled: false`, `codexcont_route: false`, + `native_keys_config_path: /CLIProxyAPI/config.yaml`, + `cpamp_alias_db_path: /CLIProxyAPI/plugin-state/cpamp-usage.sqlite`. +- CPAMP alias DB contains 3 aliases. Plus DB contains 3 enabled + `native_cpa` rows (`QQ的官key`, `kuma的官key`, `阿伟的官key`) and 1 new + native row that remains disabled by default. +- Public health/boundary checks: + `https://cpa.konbakuyomu.us/healthz` -> 200, + `https://cpa-usage.konbakuyomu.us/` -> 200, + public plugin/admin paths on `cpa.konbakuyomu.us` -> 404. +- User portal acceptance with an enabled native CPA key: + `/v0/resource/plugins/cpa-key-policy-plus/user/api/session`, + `/me`, `/usage?range=24h`, `/events?range=24h&limit=3`, and + `/codexcont?limit=3` all returned 200; `/me` reported `source=native_cpa`. +- Normal API smoke with the same key: + `/v1/models` -> 200 with 7 models; + `/v1/responses` -> 200 and response JSON reported `status=completed`. +- Quota denial smoke temporarily set that key's 5H limit to `$0.00`, called + `/v1/responses`, then restored the original `$60.00` limit. The client saw + an OpenAI-compatible JSON error body: + `type=rate_limit_exceeded`, `code=five_hour_quota_exceeded`, `param=5h`, and + Chinese message `CPA Key Policy+ 已拦截：QQ的官key 触发 5小时费用限额，已用 + $0.00 / 上限 $0.00。`. + +## Known ABI Limitation + +- The original target asked for HTTP `429` plus `X-CPA-Policy-*` headers. + Current official CPA executor response types expose only + `Payload`, `Headers`, and `Metadata`, and the public `/v1/responses` path + does not let a plugin set the final HTTP status. Production therefore returns + HTTP 200 with a clear OpenAI-compatible error body. Achieving true HTTP 429 + would require a CPA core/ABI change, which is outside this task's constraint + of not modifying official CPA/CPAMP. + +## Validation Evidence + +- `wsl -e /mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go version` + -> `go version go1.22.6 linux/amd64`. +- `wsl -e bash -lc 'cd /mnt/d/Dev/20_Software/23_Reference/llm-gateway/CodexCont/cpa_key_policy_plus_plugin/go && /mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go test ./...'` + -> pass for `codexcont/cpa-key-policy-plus-plugin` and + `codexcont/cpa-key-policy-plus-plugin/internal/policyplus`. +- `git diff --check` -> pass. +- `python ./.trellis/scripts/task.py validate .trellis/tasks/07-03-cpa-key-policy-native-keys` + -> pass. +- Production acceptance after redeploy: + user portal APIs -> pass, normal `/v1/responses` -> pass, quota denial error + body -> pass, true HTTP 429/header -> blocked by current CPA executor ABI. diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/prd.md b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/prd.md new file mode 100644 index 0000000..fbf66a8 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/prd.md @@ -0,0 +1,56 @@ +# CPA Key Policy+ native key policy layer + +## Goal + +Make `CPA Key Policy+` a passive policy layer over CPA native `sk-...` API +keys. CPA/CPAMP owns key creation, deletion, full-key copy, and alias editing. +Plus owns policy, quota, usage projection, and the user usage portal. + +## Requirements + +- Plus must continue to own `https://cpa-usage.konbakuyomu.us/`, + `/user/api/*`, usage windows, RPM, pricing, and protection summary display. +- CPA/CPAMP native API keys are the source of truth. Plus syncs them from CPA + config and reads CPAMP aliases; it must not store raw keys. +- Plus stores only safe key identity: native hash, preview, source flags, and + read-only alias/name. +- New native keys default to disabled. +- A new native key with the same alias as exactly one removed historical policy + inherits that old policy and enabled state, but starts with a new ledger. +- Removed native keys are marked `source_present=false`, disabled, hidden by + default, and retained for usage/protection history. +- Plus admin UI changes from "Key management" to "Key policy": no create, + delete, rotate, full-key copy, or alias edit controls. +- User page login must use native `sk-...` keys; old `cpa_...` keys should + fail with clear migrated/retired guidance. +- RPM and quota denials must produce explicit OpenAI-compatible `429` errors + with Chinese window/key/used/limit details, not a generic CPA auth failure. +- Cost limits use post-accounting blocking: when current window usage is + already `>= limit`, the next request is denied. +- Denial priority is: missing policy, removed/disabled, model allowlist, RPM, + then quota windows `5h`, `24h`, `7d`, `month`. + +## Acceptance Criteria + +- [ ] Native key sync reads top-level CPA `api-keys` and CPAMP + `api_key_aliases`. +- [ ] Store migration adds safe native-key fields and preserves existing rows. +- [ ] New native keys are disabled unless they inherit from one unique removed + same-alias policy. +- [ ] Removed native keys are disabled and hidden by default but keep history. +- [ ] Plus frontend auth accepts native `sk-...` keys and rejects retired + `cpa_...` self-service keys. +- [ ] Structured policy decisions cover missing, disabled, source-removed, + model, RPM, and every fee window denial. +- [ ] Deny route/executor returns OpenAI-compatible JSON and SSE error payloads + with `429` semantics and safe diagnostic headers. +- [ ] Admin HTML no longer contains create/delete/raw-key lifecycle controls. +- [ ] User HTML points users at native `sk-...` keys. +- [ ] `go test ./...` passes in `cpa_key_policy_plus_plugin/go`. + +## Constraints + +- Do not modify official CPA or CPAMP code. +- Do not store or expose raw keys, full hashes, Authorization headers, cookies, + request bodies, response bodies, or encrypted reasoning. +- Keep executor plugin ownership separate; this task only changes Plus. diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/task.json b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/task.json new file mode 100644 index 0000000..d4b1feb --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-key-policy-native-keys/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-key-policy-native-keys", + "name": "cpa-key-policy-native-keys", + "title": "CPA Key Policy+ native key policy layer", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-03", + "completedAt": "2026-07-03", + "branch": null, + "base_branch": "codex/codexcont-executor-migration", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/check.jsonl b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/design.md b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/design.md new file mode 100644 index 0000000..e265bea --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/design.md @@ -0,0 +1,31 @@ +# Design + +## User Page Contract + +The Key Policy Plus user dashboard uses a fixed primary observation window: +`24h`. The top toolbar no longer has a time-range selector. The first-page +summary cards still show side-by-side quota windows from `/me`, while the live +usage summary and request table are backed by `/usage?range=24h` and +`/events?range=24h&limit=100`. + +## Data Flow + +- `/user/api/me` remains the source for key identity, configured limits, and + four-window quota usage. +- `/user/api/usage?range=24h` remains the source for current live usage metrics. +- `/user/api/events?range=24h&limit=100` remains the source for the request table. +- Existing backend range handling stays intact for admin surfaces and future API + consumers. + +## Error Handling + +The refresh code already cancels active fetches when a forced refresh supersedes +an in-flight one or when the page is hidden. Abort errors from that cancellation +path are control flow, not user-visible failures. Refresh jobs must treat aborts +as ignored stale work and avoid writing `state.errors.usage`, +`state.errors.protection`, or connection-failure UI. + +## Compatibility + +This is a user-page UX change only. It does not remove API range parameters, +database columns, quota windows, reset watermarks, or admin controls. diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/implement.jsonl b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/implement.md b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/implement.md new file mode 100644 index 0000000..ff8e8ab --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/implement.md @@ -0,0 +1,45 @@ +# Implementation Plan + +1. Read the applicable Trellis backend/spec guidance before editing. +2. Update `cpa_key_policy_plus_plugin/go/assets/user.html`: + - remove the `rangeSelect` select element and all visibility/onchange logic, + - replace mutable `state.range` usage with a fixed `PRIMARY_RANGE = "24h"`, + - keep four-window quota cards from `/me`, + - add abort detection and ignore aborts inside refresh job error handling. +3. Add regression coverage in `cpa_key_policy_plus_plugin/go/main_test.go` for: + - no user range dropdown, + - fixed `range=24h` usage/events requests, + - visible fixed-24h labels, + - abort ignore helper presence. +4. Run `go test ./...` from `cpa_key_policy_plus_plugin/go`. +5. Run a final diff review focused on unrelated churn and the user-facing copy. + +## Execution Evidence + +- `go test ./...` in `cpa_key_policy_plus_plugin/go`: passed. +- `git diff --check`: passed with only CRLF conversion warnings. +- Before deployment, public `https://cpa-usage.konbakuyomu.us/` still served the + old user HTML: + - `rangeSelect`: present. + - `PRIMARY_RANGE`: absent. +- Built linux/amd64 plugin artifact: + - `cpa_key_policy_plus_plugin/dist/linux/amd64/cpa-key-policy-plus.so` + - SHA256 `4cfb5cdc0633bb428b8526c9146e6705457d753c240e7417e50a875e2fca4adc` + - `file`: ELF 64-bit x86-64 shared object. +- SJC deployment: + - uploaded only the new `cpa-key-policy-plus.so`; + - backed up prior plugin, Plus SQLite DB, CPA config, and CPA compose file to + `/opt/codex-stacks/backups/cpa-usage-range-ux-20260703-052620`; + - replaced `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-key-policy-plus.so`; + - restarted only the `cpa` container. +- Production verification after restart: + - remote plugin SHA256 matches local artifact: + `4cfb5cdc0633bb428b8526c9146e6705457d753c240e7417e50a875e2fca4adc`; + - CPA logs show `plugin_id=cpa-key-policy-plus` loaded and registered; + - public `https://cpa-usage.konbakuyomu.us/` returns `200`; + - public user HTML now has no `rangeSelect`, has `PRIMARY_RANGE`, has fixed + `range=${encodeURIComponent(PRIMARY_RANGE)}` requests, and includes + `refresh_cancelled`; + - `https://cpa.konbakuyomu.us/healthz` returns `200`; + - public blocked paths on `cpa.konbakuyomu.us` for Plus resource, + `/key-policy-plus/`, `/admin/`, and `/codexcont/` return `404`. diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/prd.md b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/prd.md new file mode 100644 index 0000000..bcec225 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/prd.md @@ -0,0 +1,41 @@ +# Fix CPA usage range UX + +## Goal + +Make the Key Policy Plus user usage dashboard less misleading by removing the +time-range dropdown and fixing refresh-cancel noise. The user page should present +one clear default live view for the last 24 hours, while still showing the +important quota windows side by side. + +## Requirements + +- The user page must no longer expose the `5h / 24h / 7d / month` range dropdown. +- The primary usage summary and request table must use the last 24 hours. +- The dashboard must keep the existing four-window quota visibility for `5H`, + `24H`, `7D`, and `month` so users can compare quota risk without switching + controls. +- Existing user/admin APIs and backend range calculations must remain available + and compatible. +- Canceled in-flight refresh requests caused by a newer refresh, page focus, or + visibility transition must not render as "usage sync failed" or "protection + sync failed" notices. + +## Acceptance Criteria + +- [x] The Key Policy Plus user HTML contains no `rangeSelect` dropdown or range + onchange handler. +- [x] The user page requests `/usage?range=24h` and `/events?range=24h&limit=100` + for the live usage view. +- [x] Visible labels for the main usage card, request table heading, and request + detail range read as `24 小时`. +- [x] The four-window quota cards remain visible, including `24H / 7D` and + `5H / 本月`. +- [x] Abort/cancel errors are ignored by user-page refresh logic and do not set + usage/protection sync error notices. +- [x] `go test ./...` passes in `cpa_key_policy_plus_plugin/go`. + +## Notes + +- Evidence from live HTML showed the production page is served by the Key Policy + Plus user surface, not the older Python usage portal. +- Scope excludes the admin page range selector and backend quota/window logic. diff --git a/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/task.json b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/task.json new file mode 100644 index 0000000..9cc0131 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-03-cpa-usage-range-ux/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-usage-range-ux", + "name": "cpa-usage-range-ux", + "title": "Fix CPA usage range UX", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-03", + "completedAt": "2026-07-03", + "branch": null, + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/check.jsonl b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/design.md b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/design.md new file mode 100644 index 0000000..f98166f --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/design.md @@ -0,0 +1,52 @@ +# Design + +## SQLite Discipline + +Plus uses one durable store for policy/usage and reads several auxiliary SQLite +databases. All of those connections must use a shared helper instead of raw +`sql.Open("sqlite", path)`. + +- Main Plus store: open with `_pragma=busy_timeout(5000)` and WAL, then set + `MaxOpenConns(1)` and `MaxIdleConns(1)` so one CPA plugin instance cannot + create competing write connections to the same SQLite file. +- Read-only auxiliary paths: open with `file:<path>?mode=ro&_pragma=busy_timeout(5000)` + where possible. Set a small connection pool and never write to these DBs. +- Existing schema creation can keep explicit PRAGMA statements, but the busy + timeout must be attached at connection-open time so every pooled connection + inherits it. + +## Native Key Mirror + +CPA/CPAMP remains source of truth for native `sk-...` keys and aliases. Plus +stores only safe hash/preview and policy fields. + +- Sync source: top-level CPA `api-keys` plus CPAMP `api_key_aliases`. +- Identity: native hash-derived ID remains the ledger key; alias is display and + inheritance signal only. +- Default admin projection: only `source=native_cpa`, `source_present=true`, + and `hidden=false`. +- Deleted official keys: mark removed/hidden internally and exclude from ordinary + table. Do not delete usage/audit rows. +- New official keys: enabled by default. If a unique same-alias removed/legacy + template exists, copy strategy fields. If no template exists, enabled with + empty limits. If inheritance is ambiguous, keep conflict diagnostics but do + not expose raw secrets. + +## UI Contract + +The Plus admin page is a strategy editor for current official keys, not a key +lifecycle page. + +- Remove or keep retired lifecycle controls disabled/hidden as already intended. +- Rename counts so they mean current official keys, not total historical rows. +- Render an explicit missing-limit hint when RPM is zero/empty and all quota + windows are unlimited. + +## Production Validation Boundary + +Internal API success is not enough. Acceptance requires: + +- CPAMP Plus admin API and UI refresh are stable. +- The public usage portal works. +- A real `/v1/responses` request reaches upstream and completes with `gpt-5.5`. +- Public admin/resource paths are still blocked. diff --git a/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/implement.jsonl b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/implement.md b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/implement.md new file mode 100644 index 0000000..e3844e6 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/implement.md @@ -0,0 +1,33 @@ +# Implementation Plan + +1. Load backend specs and relevant archived task evidence. +2. Add SQLite open helpers in Plus store code: + - main writable DB with busy timeout, WAL, and single-connection pool; + - read-only DB helper for CPAMP aliases and executor summaries; + - legacy import helper with busy timeout. +3. Adjust native sync defaults so brand-new official native keys are enabled + immediately unless an ambiguous inheritance conflict requires attention. +4. Update admin projection/UI text so current-key counts and missing-limit hints + are clear. +5. Add/adjust tests for SQLite discipline, native lifecycle, UI strings, and + admin API mirror behavior. +6. Run local validation: + - `go test ./...` in `cpa_key_policy_plus_plugin/go` + - executor tests if touched + - `git diff --check` + - Trellis task validation +7. Build linux/amd64 plugin with the existing local Go runtime, not a new Go + download. +8. Deploy cautiously: + - check SJC disk; + - backup current Plus `.so`; + - upload replacement; + - restart only CPA. +9. Production acceptance: + - repeat Plus admin keys/models API calls and check logs for no `SQLITE_BUSY`; + - verify key count/aliases match current official config; + - verify usage portal 200 and user APIs; + - smoke `/v1/responses` with enabled native key and `gpt-5.5`; + - verify public admin/resource paths remain 404. +10. Update spec with durable SQLite/native-key contract, commit, push, archive + task, and record journal evidence. diff --git a/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/prd.md b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/prd.md new file mode 100644 index 0000000..b7c7b2e --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/prd.md @@ -0,0 +1,86 @@ +# Fix CPA Key Policy Plus SQLite locks and native key mirror + +## Goal + +Make CPA Key Policy+ stable and operator-trustworthy again: + +- The CPAMP Plus admin page must stop surfacing `database is locked (5) (SQLITE_BUSY)` during normal refresh/key-sync/usage activity. +- Plus must behave as a policy mirror for current official CPA native keys: the ordinary table shows only currently-present official keys with current aliases. +- Newly-created official native keys are enabled by default per user decision, while the UI makes missing RPM/quota limits obvious. +- Real Codex requests through the production CPA chain must succeed before handoff, not just internal admin APIs. + +## Confirmed Evidence + +- The previous task restored `/v1/responses` by replacing invalidated Codex OAuth auth, but did not harden Plus SQLite access. +- Plus `OpenStore`, CPAMP alias reads, executor-summary fallback reads, and legacy import reads currently use plain `sql.Open("sqlite", path)` without a busy timeout helper. +- Production `policyplus.sqlite` was observed with multiple file descriptors in one CPA process, consistent with SQLite lock contention risk under concurrent admin/API/usage work. +- Current Plus sync hides removed native keys from the default table but still keeps removed/history rows internally; ordinary UI must remain a current-official-key mirror. +- User selected `new official key = enabled immediately`. + +## Requirements + +- Add durable SQLite lock hardening for Plus-owned and Plus-read SQLite paths. +- Keep official CPA/CPAMP as the only raw-key and alias lifecycle owner. +- Re-sync native keys from CPA config and CPAMP aliases before Plus admin key listing and relevant strategy mutations. +- Default Plus admin key rows must be exactly current official native keys; deleted official keys must disappear from the ordinary table after refresh. +- Preserve historical usage, audit, and protection summaries internally without exposing stale rows in ordinary admin UX. +- Newly-created official native keys must be enabled immediately; if they have no RPM or quota limits, the admin UI must visibly indicate unlimited/missing limits. +- Do not modify official CPA/CPAMP source and do not expose raw `sk-...`, OAuth tokens, cookies, full hashes, request/response bodies, or encrypted reasoning. +- Production verification must include true `/v1/responses` with `gpt-5.5`, usage portal, CPAMP Plus admin APIs, and public-path blocking. + +## Acceptance Criteria + +- [x] Local Plus tests cover SQLite open discipline, busy-timeout behavior, native key mirror lifecycle, default-enabled new native keys, and admin UI missing-limit hints. +- [x] `go test ./...` passes in `cpa_key_policy_plus_plugin/go`; executor tests run if request-chain code/config is touched. +- [x] Linux/amd64 Plus plugin is built with the existing local Go runtime, deployed with a backup, and CPA is restarted once. +- [x] Production Plus admin key API returns 200 repeatedly without `SQLITE_BUSY` and mirrors the current official key count/aliases. +- [x] Official key add/delete/alias-change behavior is smoke-tested or directly verified against source-of-truth config/alias DB plus Plus API. +- [x] `https://cpa-usage.konbakuyomu.us/` still returns 200 and user APIs continue to work. +- [x] A real enabled native key request to `/v1/responses` with `model=gpt-5.5` succeeds; any auth failure is diagnosed from provider/auth availability without leaking secrets. +- [x] Public `cpa.konbakuyomu.us` admin/resource/plugin paths remain blocked. + +## Notes + +- SQLite fixes must be code-level durable fixes, not just a CPA restart. +- The ordinary UI mirrors current official keys; internal state can retain history for billing/debugging. + +## Final Evidence + +- Local toolchain: used existing WSL Go + `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go`, + version `go1.22.6 linux/amd64`; no Go download was performed. +- Local validation passed: + `go test ./... -count=1 -timeout=120s` in + `cpa_key_policy_plus_plugin/go` and `cpa_codexcont_executor_plugin/go`, + `git diff --check`, and + `python ./.trellis/scripts/task.py validate .trellis/tasks/07-04-cpa-key-policy-plus-sqlite-native-chain`. +- Production artifact: + `/opt/codex-stacks/cpa/plugins/linux/amd64/cpa-key-policy-plus.so` + SHA256 `35c3179324883ebfcdbc7681e1304bb3bf3539fb62baa6579ba5e1cf49feab0e`. +- CPA logs after restart show both `cpa-codexcont-executor` and + `cpa-key-policy-plus` loaded. +- Production Plus key API returned HTTP `200` repeatedly; CPA logs for the + validation window contained no `SQLITE_BUSY` or `database is locked`. +- Native mirror verification: + official CPA config count `4`, Plus ordinary key count `4`, preview sets + matched exactly, no missing or extra Plus rows. Plus displayed names were + `1bcd36fb...4180a3`, `kuma的官key`, `QQ的官key`, and `阿伟的官key`. +- Plus ordinary rows were all current native rows: + enabled count `4`, `source_present` count `4`, no visible legacy or removed + row. +- Public usage portal root returned HTTP `200`. User API smoke with a native + key passed: + `/session`, `/me`, `/usage?range=24h`, + `/events?range=24h&limit=3`, and `/codexcont?limit=3` all returned HTTP + `200` with `ok=true`. +- Real `/v1/responses` smoke through production returned HTTP `200`, + `model=gpt-5.5`, no error code, and text `OK`. +- Over-limit smoke: + temporarily set the unaliased smoke key `1bcd36fb...4180a3` 5H limit to + `$0.00`; `/v1/responses` returned OpenAI-compatible error JSON with code + `five_hour_quota_exceeded` and a Chinese message containing + `5小时费用限额`; original policy was restored successfully. +- Public blocked paths all returned HTTP `404`: + `/v0/resource/plugins/cpa-key-policy-plus/admin`, + `/v0/resource/plugins/cpa-codexcont-executor/admin`, + `/key-policy-plus/`, `/management.html`, and `/governor/`. diff --git a/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/task.json b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/task.json new file mode 100644 index 0000000..8b5488d --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-cpa-key-policy-plus-sqlite-native-chain/task.json @@ -0,0 +1,26 @@ +{ + "id": "cpa-key-policy-plus-sqlite-native-chain", + "name": "cpa-key-policy-plus-sqlite-native-chain", + "title": "Fix CPA Key Policy Plus SQLite locks and native key mirror", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P1", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-04", + "completedAt": "2026-07-04", + "branch": null, + "base_branch": "codex/codexcont-executor-migration", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/check.jsonl b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/check.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/check.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/design.md b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/design.md new file mode 100644 index 0000000..be02298 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/design.md @@ -0,0 +1,68 @@ +# Design + +## Problem Split + +This task has two independent but user-visible failures: + +1. Executor model aliasing is correct for provider compatibility but currently + forwards all client tools unchanged. When the visible model `gpt-5.5` maps + to upstream `gpt-5.3-codex-spark`, upstream rejects unsupported built-in + tools such as `image_generation`. +2. Plus mirrors the correct native key count, but one official alias (`alicea`) + is missing because the production alias source is a WAL-mode CPAMP SQLite DB + and the old CPA file bind saw only a stale `usage.sqlite` view without the + live `usage.sqlite-wal` / `usage.sqlite-shm` files. + +## Executor Tool Compatibility + +The executor must preserve the visible model contract while adapting the +upstream request body for the actual target model. + +Data flow: + +```text +client body model=gpt-5.5, tools=[image_generation] + -> executor resolves upstream_model=gpt-5.3-codex-spark + -> executor rewrites body.model to Spark + -> executor filters unsupported tools for Spark + -> upstream receives compatible body + -> downstream stream and summaries still say gpt-5.5 +``` + +For the first fix, define a small compatibility layer keyed by upstream model. +For `gpt-5.3-codex-spark`, drop built-in tool entries whose `type` is +`image_generation`. Do not drop custom function tools. If the tools array +becomes empty, omit it from the upstream body. Record safe diagnostics such as +`filtered_tool_types:["image_generation"]`, not the request body. + +## Alias Source + +Plus already treats CPA config `api-keys` as native key source of truth. Alias +must come from the same source the official API key panel uses. + +Investigation should identify the real production storage shape. The code +should support: + +- existing `api_key_aliases(api_key_hash, alias)` table when present; +- a safe file/config/table fallback discovered in production; +- a directory-level read-only CPAMP data mount for WAL-mode SQLite, because a + single-file bind can hide newly written aliases from the reader; +- no fallback should ever store or return raw keys. + +Alias matching remains by SHA256 of the native raw key. The ledger key remains +the native hash-derived `native_<preview>` id; alias is display and inheritance +metadata only. + +## Rollout + +Build only changed plugin artifacts with the existing WSL Go runtime and +`-tags cliproxy_plugin -buildmode=c-shared`. Deploy to SJC with a backup and +restart only `cpa`. + +Production note: applying the new CPAMP data directory mount requires recreating +the `cpa` container. The current compose file has `pull_policy: always`; avoid +that on future plugin/config-only changes or expect an official CPA image pull +as part of recreate. + +Rollback is replacing the new `.so` with the timestamped backup and restarting +`cpa`. diff --git a/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/implement.jsonl b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/implement.jsonl new file mode 100644 index 0000000..9cd59d4 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/implement.jsonl @@ -0,0 +1 @@ +{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."} diff --git a/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/implement.md b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/implement.md new file mode 100644 index 0000000..b4035e6 --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/implement.md @@ -0,0 +1,107 @@ +# Implementation Plan + +1. Confirm production evidence without exposing secrets: + - executor alias config for `gpt-5.5`; + - current upstream failure body; + - official key alias storage schema and Plus admin projection. +2. Implement executor upstream body compatibility: + - add helper to rewrite model and filter unsupported built-in tools for the + resolved upstream model; + - use it in streaming and non-streaming executor paths; + - add safe diagnostics for filtered tools. +3. Implement Plus alias-source fallback: + - inspect the production alias source; + - extend alias loader/tests to read that source while preserving existing + `api_key_aliases` behavior. +4. Add tests: + - executor request with `image_generation` and `gpt-5.5` alias to Spark; + - no filtering for custom/function tools; + - Plus alias fallback returns `alicea` for the corresponding native hash. +5. Validate locally: + - `go test ./... -count=1 -timeout=120s` in + `cpa_codexcont_executor_plugin/go`; + - `go test ./... -count=1 -timeout=120s` in + `cpa_key_policy_plus_plugin/go`; + - `git diff --check`; + - Trellis task validation. +6. Build Linux plugin artifacts with existing local WSL Go: + - no Go download; + - build only the changed `.so` files using `-tags cliproxy_plugin + -buildmode=c-shared`; + - record SHA256. +7. Deploy to SJC: + - backup current plugin `.so`; + - upload replacement(s); + - restart only `cpa`; + - verify plugins loaded. +8. Production acceptance: + - Plus admin API shows exactly four native keys and includes `alicea`; + - real `/v1/responses` with `model=gpt-5.5` and `image_generation` no longer + fails with unsupported-tool error; + - normal `gpt-5.5` still returns `OK`; + - `cpa-usage` user APIs still return 200/ok; + - public admin/resource/plugin paths remain 404. +9. Update spec with the tool-compatibility and alias-source contracts, commit, + push, archive, and record journal. + +## Implementation Evidence + +- Local Go tests passed: + - `go test ./... -count=1 -timeout=120s` in + `cpa_codexcont_executor_plugin/go`. + - `go test ./... -count=1 -timeout=120s` in + `cpa_key_policy_plus_plugin/go`. +- WSL Go validation used the existing toolchain: + `/mnt/d/Dev/20_Software/_LocalRuntime/go/go1.22.6-linux-amd64/go/bin/go` + (`go1.22.6 linux/amd64`); no Go download was performed. +- Linux plugin artifacts were built with + `CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags cliproxy_plugin + -buildmode=c-shared`. +- Built artifact SHA256: + - `cpa-codexcont-executor.so`: + `af17e38f6b4a6fe8912c0f32cbc9f747d49db99545ce642bc31f08f783b07cd4`. + - `cpa-key-policy-plus.so`: + `f1e898032fdd7f0d3ed5f0f0d0ad14fe8a6fca79ead6561437ad7216904e2ec8`. + +## Production Evidence + +- Deployed both plugin artifacts to SJC and backed up previous binaries at + `/opt/codex-stacks/backups/executor-alias-tool-fix-20260704-061152`. +- Fixed the alias source root cause by adding a read-only directory mount: + `/opt/codex-stacks/cpamp/data:/CLIProxyAPI/cpamp-data:ro`, then changed Plus + `cpamp_alias_db_path` to `/CLIProxyAPI/cpamp-data/usage.sqlite`. +- Config backup before the mount change: + `/opt/codex-stacks/backups/cpamp-alias-wal-mount-20260704-061939`. +- Evidence for the alias bug: + - CPAMP host DB had `usage.sqlite`, `usage.sqlite-shm`, and + `usage.sqlite-wal`. + - The old CPA file-level mount exposed `cpamp-usage.sqlite` with a stale / + empty WAL view, so Plus saw three aliases but not the new `alicea`. + - After the directory mount, Plus admin API returned exactly four current + native keys and the `native_1bcd36fb_4180a3` row showed + `name=alicea`, `alias=alicea`, and `preview=1bcd36fb...4180a3`. +- `/v1/responses` production smoke: + - Internal non-stream `model=gpt-5.5` with `tools:[image_generation]`: + HTTP `200`, visible model `gpt-5.5`, no unsupported-tool error. + - Internal complete streaming `model=gpt-5.5` with + `tools:[image_generation]`: HTTP `200`, `response.completed` observed, + visible model `gpt-5.5`, no upstream model leak. + - Public `https://cpa.konbakuyomu.us/v1/responses` non-stream smoke: + HTTP `200`, visible model `gpt-5.5`, no unsupported-tool error. +- `cpa-usage` user API smoke with the `alicea` key succeeded through the + GET-only resource API: + `/session`, `/me`, `/usage?range=24h`, `/events?range=24h&limit=3`, and + `/codexcont?limit=3` all returned `200/ok`; `/me` matched + `native_1bcd36fb_4180a3`. +- Public boundary smoke: + - `https://cpa.konbakuyomu.us/healthz`: `200`. + - Public Plus/executor resource admin paths, `/codexcont/`, and + `/management.html`: `404`. + - `https://cpa-usage.konbakuyomu.us/`: `200`. + +## Caution + +The CPA compose file currently contains `pull_policy: always`. Recreating `cpa` +to apply the new mount pulled the official CPA image from `v7.2.49` to +`v7.2.50`. Smoke tests above passed after the pull, but future plugin-only or +mount-only deploys should avoid unintended image drift. diff --git a/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/prd.md b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/prd.md new file mode 100644 index 0000000..6abd2ea --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/prd.md @@ -0,0 +1,61 @@ +# Fix executor tool routing and native key aliases + +## Goal + +Restore the post-migration user experience: + +- Local Codex calls using the visible `gpt-5.5` model must not fail just because + the executor internally routes to `gpt-5.3-codex-spark`. +- CPA Key Policy+ must display the same four official key aliases as the CPA / + CPAMP API key panel, including `alicea`, and the selected row/detail panel + must refer to the same safe preview/id. + +## Requirements + +- Keep the client-visible model name (`gpt-5.5`) in downstream streams, + summaries, and Plus usage projections. +- Preserve the executor's internal upstream model aliasing when needed for the + current CPA provider registration, but make it compatible with client tools + that the upstream target does not support. +- The screenshot failure `Tool 'image_generation' is not supported with + gpt-5.3-codex-spark` must be prevented before the upstream call. +- Do not expose raw API keys, OAuth files, cookies, Authorization headers, full + hashes, request/response bodies, or encrypted reasoning in UI, logs, task + files, or final output. +- Plus must read official CPA/CPAMP aliases from the actual source of truth used + by the API key panel. If the configured `cpamp_alias_db_path` has no + `api_key_aliases` table, Plus must fall back to another safe configured source + rather than silently showing the hash preview. +- Plus ordinary key list must still show exactly current official native keys; + removed or legacy rows stay hidden. +- The fix must be deployed and verified on SJC before handoff. + +## Acceptance Criteria + +- [x] Trellis planning artifacts explain why the `gpt-5.5` to + `gpt-5.3-codex-spark` mapping exists and how tool compatibility is handled. +- [x] Executor unit tests cover a `gpt-5.5` request with + `tools:[{type:"image_generation"}]` routed to Spark, proving the upstream + request removes or neutralizes the unsupported tool while preserving the + visible model downstream. +- [x] Executor diagnostics/summaries do not leak raw request bodies while still + exposing safe evidence that tools were filtered. +- [x] Plus unit tests cover alias lookup from the real official alias source and + the fallback path when `api_key_aliases` is absent. +- [x] Plus admin API on SJC shows four current native keys with names: + `kuma的官key`, `QQ的官key`, `阿伟的官key`, and `alicea`. +- [x] Selecting the `alicea` row in Plus shows detail preview/id consistent with + the row preview, not a mismatched alias/hash. +- [x] Real production `/v1/responses` smoke with an enabled native key and + `model=gpt-5.5` plus `image_generation` no longer fails with unsupported tool. +- [x] Regression checks pass: + `go test ./...` in both plugin packages, `git diff --check`, Linux plugin + build(s), public blocked-path smoke, and `cpa-usage` user API smoke. + +## Notes + +- User asked why the mapping exists: it is a compatibility alias in the executor, + not a CPA native key issue. It lets the client keep asking for `gpt-5.5` while + the current provider-registered upstream model is `gpt-5.3-codex-spark`. +- The mapping became visible as a bug because the new request carried the + `image_generation` tool and Spark rejects that tool. diff --git a/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/task.json b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/task.json new file mode 100644 index 0000000..6eeb94e --- /dev/null +++ b/.trellis/tasks/archive/2026-07/07-04-executor-tool-routing-native-aliases/task.json @@ -0,0 +1,26 @@ +{ + "id": "executor-tool-routing-native-aliases", + "name": "executor-tool-routing-native-aliases", + "title": "Fix executor tool routing and native key aliases", + "description": "", + "status": "completed", + "dev_type": null, + "scope": null, + "package": null, + "priority": "P2", + "creator": "dxt98", + "assignee": "dxt98", + "createdAt": "2026-07-04", + "completedAt": "2026-07-04", + "branch": null, + "base_branch": "codex/codexcont-executor-migration", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": null, + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/.trellis/workflow.md b/.trellis/workflow.md new file mode 100644 index 0000000..06f8e6a --- /dev/null +++ b/.trellis/workflow.md @@ -0,0 +1,710 @@ +# Development Workflow + +--- + +## Core Principles + +1. **Plan before code** — figure out what to do before you start +2. **Specs injected, not remembered** — guidelines are injected via hook/skill, not recalled from memory +3. **Persist everything** — research, decisions, and lessons all go to files; conversations get compacted, files don't +4. **Incremental development** — one task at a time +5. **Capture learnings** — after each task, review and write new knowledge back to spec + +--- + +## Trellis System + +### Developer Identity + +On first use, initialize your identity: + +```bash +python ./.trellis/scripts/init_developer.py <your-name> +``` + +Creates `.trellis/.developer` (gitignored) + `.trellis/workspace/<your-name>/`. + +### Spec System + +`.trellis/spec/` holds coding guidelines organized by package and layer. + +- `.trellis/spec/<package>/<layer>/index.md` — entry point with **Pre-Development Checklist** + **Quality Check**. Actual guidelines live in the `.md` files it points to. +- `.trellis/spec/guides/index.md` — cross-package thinking guides. + +```bash +python ./.trellis/scripts/get_context.py --mode packages # list packages / layers +``` + +**When to update spec**: new pattern/convention found · bug-fix prevention to codify · new technical decision. + +### Task System + +Every task has its own directory under `.trellis/tasks/{MM-DD-name}/` holding `task.json`, `prd.md`, optional `design.md`, optional `implement.md`, optional `research/`, and context manifests (`implement.jsonl`, `check.jsonl`) for sub-agent-capable platforms. + +```bash +# Task lifecycle +python ./.trellis/scripts/task.py create "<title>" [--slug <name>] [--parent <dir>] +python ./.trellis/scripts/task.py start <name> # set active task (session-scoped when available) +python ./.trellis/scripts/task.py current --source # show active task and source +python ./.trellis/scripts/task.py finish # clear active task (triggers after_finish hooks) +python ./.trellis/scripts/task.py archive <name> # move to archive/{year-month}/ +python ./.trellis/scripts/task.py list [--mine] [--status <s>] +python ./.trellis/scripts/task.py list-archive + +# Code-spec context (injected into implement/check agents via JSONL). +# `implement.jsonl` / `check.jsonl` are seeded on `task create` for sub-agent-capable +# platforms; the AI curates real spec + research entries during planning when needed. +python ./.trellis/scripts/task.py add-context <name> <action> <file> <reason> +python ./.trellis/scripts/task.py list-context <name> [action] +python ./.trellis/scripts/task.py validate <name> + +# Task metadata +python ./.trellis/scripts/task.py set-branch <name> <branch> +python ./.trellis/scripts/task.py set-base-branch <name> <branch> # PR target +python ./.trellis/scripts/task.py set-scope <name> <scope> + +# Hierarchy (parent/child) +python ./.trellis/scripts/task.py add-subtask <parent> <child> +python ./.trellis/scripts/task.py remove-subtask <parent> <child> + +# PR creation +python ./.trellis/scripts/task.py create-pr [name] [--dry-run] +``` + +> Run `python ./.trellis/scripts/task.py --help` to see the authoritative, up-to-date list. + +**Current-task mechanism**: `task.py create` creates the task directory and (when session identity is available) auto-sets the per-session active-task pointer so the planning breadcrumb fires immediately. `task.py start` writes the same pointer (idempotent if already set) and flips `task.json.status` from `planning` to `in_progress`. State is stored under `.trellis/.runtime/sessions/`. If no context key is available from hook input, `TRELLIS_CONTEXT_ID`, or a platform-native session environment variable, there is no active task and `task.py start` fails with a session identity hint. `task.py finish` deletes the current session file (status unchanged). `task.py archive <task>` writes `status=completed`, moves the directory to `archive/`, and deletes any runtime session files that still point at the archived task. + +### Workspace System + +Records every AI session for cross-session tracking under `.trellis/workspace/<developer>/`. + +- `journal-N.md` — session log. **Max 2000 lines per file**; a new `journal-(N+1).md` is auto-created when exceeded. +- `index.md` — personal index (total sessions, last active). + +```bash +python ./.trellis/scripts/add_session.py --title "Title" --commit "hash" --summary "Summary" +``` + +### Context Script + +```bash +python ./.trellis/scripts/get_context.py # full session runtime +python ./.trellis/scripts/get_context.py --mode packages # available packages + spec layers +python ./.trellis/scripts/get_context.py --mode phase --step <X.Y> # detailed guide for a workflow step +``` + +--- + +<!-- + WORKFLOW-STATE BREADCRUMB CONTRACT (read this before editing the tag blocks below) + + The [workflow-state:STATUS] blocks embedded in the ## Phase Index section + below are the SINGLE source of truth for the per-turn `<workflow-state>` + breadcrumb that every supported AI platform's UserPromptSubmit hook + reads. inject-workflow-state.py (Python platforms) and + inject-workflow-state.js (OpenCode plugin) only parse them — there is no + fallback dict baked into the scripts after v0.5.0-rc.0. + + STATUS charset: [A-Za-z0-9_-]+. When the hook can't find a tag, it + degrades to a generic "Refer to workflow.md for current step." line — + intentionally visible so users notice and fix a broken workflow.md. + + INVARIANT (test/regression.test.ts): + Every workflow-walkthrough step marked `[required · once]` must have a + matching enforcement line in its phase's [workflow-state:*] block. The + breadcrumb is the only per-turn channel; if a mandatory step isn't + mentioned there, the AI silently skips it (Phase 1 planning gate + skip and Phase 3.4 commit skip both manifested via this gap). + + TAG ↔ PHASE scoping: + [workflow-state:no_task] → no active task; before Phase 1 + [workflow-state:planning] → all of Phase 1 (status='planning') + [workflow-state:planning-inline] → Codex inline variant of Phase 1 + [workflow-state:in_progress] → Phase 2 + Phase 3.1-3.4 + (status stays 'in_progress' from + task.py start until task.py archive) + [workflow-state:in_progress-inline] → Codex inline variant of Phase 2/3 + [workflow-state:completed] → currently DEAD: cmd_archive flips + status and moves the dir in the same + call, so the resolver loses the + pointer (block kept for a future + explicit in_progress→completed + transition) + + Editing checklist: + - When you change a [workflow-state:STATUS] block, also check the + matching phase's `[required · once]` walkthrough steps for sync + - Run `trellis update` after editing to push the new bodies to + downstream user projects (block-level managed replacement) + - Full runtime contract: + .trellis/spec/cli/backend/workflow-state-contract.md +--> + +## Phase Index + +``` +Phase 1: Plan → classify, get task-creation consent, then write planning artifacts +Phase 2: Execute → implement only after task status is in_progress +Phase 3: Finish → verify, update spec, commit, and wrap up +``` + +### Request Triage + +- Simple conversation or small task: ask only whether this turn should create a Trellis task. If the user says no, skip Trellis for this session. +- Complex task: ask whether you may create a Trellis task and enter planning. If the user says no, do not do broad inline implementation; explain, clarify scope, or suggest a smaller split. +- User approval to create a task is not approval to start implementation. Planning still happens first. + +### Planning Artifacts + +- `prd.md` — requirements, constraints, and acceptance criteria. Do not put technical design or execution checklists here. +- `design.md` — technical design for complex tasks: boundaries, contracts, data flow, tradeoffs, compatibility, rollout / rollback shape. +- `implement.md` — execution plan for complex tasks: ordered checklist, validation commands, review gates, and rollback points. +- `implement.jsonl` / `check.jsonl` — spec and research manifests for sub-agent context. They do not replace `implement.md`. +- Lightweight tasks may be PRD-only. Complex tasks must have `prd.md`, `design.md`, and `implement.md` before `task.py start`. + +### Parent / Child Task Trees + +Use a parent task when one user request contains several independently verifiable deliverables. The parent task owns the source requirement set, the task map, cross-child acceptance criteria, and final integration review; it normally should not be the implementation target unless it also has direct work. + +Use child tasks for deliverables that can be planned, implemented, checked, and archived independently. Parent/child structure is not a dependency system: if one child must wait for another, write that ordering in the child `prd.md` / `implement.md` and keep each child's acceptance criteria testable. + +Create new children with `task.py create "<title>" --slug <name> --parent <parent-dir>`. Link existing tasks with `task.py add-subtask <parent> <child>`, and unlink mistakes with `task.py remove-subtask <parent> <child>`. + +<!-- Per-turn breadcrumb: shown when there is no active task (before Phase 1) --> + +[workflow-state:no_task] +No active task. First classify the current turn and ask for task-creation consent before creating any Trellis task. +Simple conversation / small task: ask only whether this turn should create a Trellis task. If the user says no, skip Trellis for this session. +Complex task: ask the user if you can create a Trellis task and enter the planning phase. If the user says no, explain, clarify scope, or suggest a smaller split. +[/workflow-state:no_task] + +### Phase 1: Plan +- 1.0 Create task `[required · once]` (only after task-creation consent) +- 1.1 Requirement exploration `[required · repeatable]` (`prd.md`; complex tasks also need `design.md` + `implement.md`) +- 1.2 Research `[optional · repeatable]` +- 1.3 Configure context `[conditional · once]` — Claude Code, Cursor, OpenCode, Codex, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi +- 1.4 Activate task `[required · once]` (review gate, then `task.py start`; status → in_progress) +- 1.5 Completion criteria + +<!-- Per-turn breadcrumb: shown throughout Phase 1 (status='planning') --> + +[workflow-state:planning] +Load `trellis-brainstorm`; stay in planning. +Lightweight: `prd.md` can be enough. Complex: finish `prd.md`, `design.md`, and `implement.md`; ask for review before `task.py start`. +Multi-deliverable scope: consider a parent task plus independently verifiable child tasks; dependencies must be written in child artifacts, not implied by tree position. +Sub-agent mode: curate `implement.jsonl` and `check.jsonl` as spec/research manifests before start. +[/workflow-state:planning] + +<!-- Per-turn breadcrumb: shown throughout Phase 1 when codex.dispatch_mode=inline. + Codex-only opt-in alternate to [workflow-state:planning]. The main agent + edits code directly in Phase 2, so jsonl curation is skipped — + the inline workflow loads `trellis-before-dev` instead of injecting JSONL + into a sub-agent. --> + +[workflow-state:planning-inline] +Load `trellis-brainstorm`; stay in planning. +Lightweight: `prd.md` can be enough. Complex: finish `prd.md`, `design.md`, and `implement.md`; ask for review before `task.py start`. +Multi-deliverable scope: consider a parent task plus independently verifiable child tasks; dependencies must be written in child artifacts, not implied by tree position. +Inline mode: skip jsonl curation; Phase 2 reads artifacts/specs via `trellis-before-dev`. +[/workflow-state:planning-inline] + +### Phase 2: Execute +- 2.1 Implement `[required · repeatable]` +- 2.2 Quality check `[required · repeatable]` +- 2.3 Rollback `[on demand]` + +<!-- Per-turn breadcrumb: shown while status='in_progress'. + Scope: all of Phase 2 + Phase 3.1-3.4 (status stays 'in_progress' from + task.py start until task.py archive; only archive flips it). The body + therefore must cover every required step from implementation through + commit, including Phase 3.3 spec update and Phase 3.4 commit. --> + +Sub-agent dispatch protocol applies to all platforms and all sub-agents, including class-2 Codex/Copilot/Gemini/Qoder and `trellis-research`: every dispatch prompt starts with `Active task: <task path from task.py current>` before role-specific instructions. + +[workflow-state:in_progress] +Tools: `trellis-implement` / `trellis-research` are sub-agent types only (Task/Agent tool, NOT Skill; there is no skill by these names). `trellis-update-spec` is a skill. `trellis-check` exists as both; prefer the Agent form when verifying after code changes. +Flow: `trellis-implement` -> `trellis-check` -> `trellis-update-spec` -> commit (Phase 3.4) -> `/trellis:finish-work`. +Main-session default: dispatch implement/check sub-agents. Sub-agent self-exemption: if already running as `trellis-implement`, do NOT spawn another `trellis-implement` or `trellis-check`; if already running as `trellis-check`, do NOT spawn another `trellis-check` or `trellis-implement`. Dispatch is main session only. +Dispatch prompt starts with `Active task: <task path from task.py current>`. Read context: jsonl entries -> `prd.md` -> `design.md if present` -> `implement.md if present`. +[/workflow-state:in_progress] + +<!-- Per-turn breadcrumb: shown while status='in_progress' when + codex.dispatch_mode=inline. Codex-only opt-in alternate to + [workflow-state:in_progress]. The main session edits code directly + instead of dispatching sub-agents. --> + +[workflow-state:in_progress-inline] +Flow: `trellis-before-dev` -> edit -> `trellis-check` -> validation -> `trellis-update-spec` -> commit (Phase 3.4) -> `/trellis:finish-work`. +Do not dispatch implement/check sub-agents in inline mode. +Read context: `prd.md` -> `design.md if present` -> `implement.md if present`, plus relevant spec/research loaded by skills. +[/workflow-state:in_progress-inline] + +### Phase 3: Finish +- 3.1 Quality verification `[required · repeatable]` +- 3.2 Debug retrospective `[on demand]` +- 3.3 Spec update `[required · once]` +- 3.4 Commit changes `[required · once]` +- 3.5 Wrap-up reminder + +<!-- Per-turn breadcrumb: shown while status='completed'. + Currently DEAD in normal flow: cmd_archive writes status='completed' in + the same call that moves the task dir to archive/, so the active-task + resolver loses the pointer and the hook never fires on archived tasks. + Block preserved for a future status-transition redesign (e.g. an + explicit in_progress→completed command). Edit through the same spec + channel as the live blocks. --> + +[workflow-state:completed] +Code committed. Run `/trellis:finish-work`; if dirty, return to Phase 3.4 first. +[/workflow-state:completed] + +### Rules + +1. Identify which Phase you're in, then continue from the next step there +2. Run steps in order inside each Phase; `[required]` steps can't be skipped +3. Phases can roll back (e.g., Execute reveals a prd defect → return to Plan to fix, then re-enter Execute) +4. Steps tagged `[once]` are skipped if the output already exists; don't re-run +5. Artifact presence informs the next step; missing `design.md` / `implement.md` is valid for lightweight tasks and incomplete planning for complex tasks. + +### Active Task Routing + +When a user request matches one of these intents inside an active task, route first, then load the detailed phase step if needed. + +[Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +- Planning or unclear requirements -> `trellis-brainstorm`. +- `in_progress` implementation/check -> dispatch `trellis-implement` / `trellis-check`. +- Repeated debugging -> `trellis-break-loop`; spec updates -> `trellis-update-spec`. + +[/Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +[codex-inline, Kilo, Antigravity, Windsurf] + +- Planning or unclear requirements -> `trellis-brainstorm`. +- Before editing -> `trellis-before-dev`; after editing -> `trellis-check`. +- Repeated debugging -> `trellis-break-loop`; spec updates -> `trellis-update-spec`. + +[/codex-inline, Kilo, Antigravity, Windsurf] + +### Guardrails + +- Task creation approval is not implementation approval; implementation waits for `task.py start` after artifact review. +- PRD-only is valid for lightweight tasks; complex tasks need `design.md` + `implement.md`. +- Planning must be persisted to task artifacts; checks must run before reporting completion. + +### Loading Step Detail + +At each step, run this to fetch detailed guidance: + +```bash +python ./.trellis/scripts/get_context.py --mode phase --step <step> +# e.g. python ./.trellis/scripts/get_context.py --mode phase --step 1.1 +``` + +--- + +## Phase 1: Plan + +Goal: classify the request, get task-creation consent when a task is needed, and produce the planning artifacts required before implementation. + +#### 1.0 Create task `[required · once]` + +Create the task directory only after task-creation consent. The command sets status to `planning`, writes `task.json`, creates a default `prd.md`, and auto-targets the new task when session identity is available: + +```bash +python ./.trellis/scripts/task.py create "<task title>" --slug <name> +``` + +`--slug` is the human-readable name only. Do **not** include the `MM-DD-` date prefix; `task.py create` adds that prefix automatically. + +For task trees, create the parent task first and then create each child with `--parent <parent-dir>`. Do not start the parent just because children exist; start the child that owns the next independently verifiable deliverable. + +After this command succeeds, the per-turn breadcrumb auto-switches to `[workflow-state:planning]`, telling the AI to stay in planning. + +Run only `create` here — do not also run `start`. `start` flips status to `in_progress`, which switches the breadcrumb to the implementation phase before planning artifacts are reviewed. Save `start` for step 1.4. + +Skip when `python ./.trellis/scripts/task.py current --source` already points to a task. + +#### 1.1 Requirement exploration `[required · repeatable]` + +Load the `trellis-brainstorm` skill and explore requirements interactively with the user per the skill's guidance. + +The brainstorm skill will guide you to: +- Ask one question at a time +- Prefer researching over asking the user +- Prefer offering options over open-ended questions +- Update `prd.md` immediately after each user answer +- Split large scopes into a parent task plus child tasks when the deliverables can be verified independently +- Keep `prd.md` focused on requirements and acceptance criteria +- For complex tasks, produce `design.md` and `implement.md` before implementation starts + +When considering a parent/child split: +- Use a parent task when one request contains several independently verifiable deliverables. +- Parent tasks own source requirements, child-task mapping, cross-child acceptance criteria, and final integration review. +- Child tasks own actual deliverables that can be planned, implemented, checked, and archived independently. +- Parent/child structure is not a dependency system. If child B depends on child A, write that ordering in child B's `prd.md` / `implement.md`. +- Start the child task that owns the next deliverable. Do not start the parent unless the parent itself has direct implementation work. + +Return to this step whenever requirements change and revise the relevant artifact. + +#### 1.2 Research `[optional · repeatable]` + +Research can happen at any time during requirement exploration. It isn't limited to local code — you can use any available tool (MCP servers, skills, web search, etc.) to look up external information, including third-party library docs, industry practices, API references, etc. + +[Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +Spawn the research sub-agent: + +- **Agent type**: `trellis-research` +- **Task description**: Research <specific question> +- **Key requirement**: Research output MUST be persisted to `{TASK_DIR}/research/` + +[/Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +[codex-inline, Kilo, Antigravity, Windsurf] + +Do the research in the main session directly and write findings into `{TASK_DIR}/research/`. (For `codex-inline` this avoids the `fork_turns="none"` isolation that prevents `trellis-research` sub-agents from resolving the active task path.) + +[/codex-inline, Kilo, Antigravity, Windsurf] + +**Research artifact conventions**: +- One file per research topic (e.g. `research/auth-library-comparison.md`) +- Record third-party library usage examples, API references, version constraints in files +- Note relevant spec file paths you discovered for later reference + +Brainstorm and research can interleave freely — pause to research a technical question, then return to talk with the user. + +**Key principle**: Research output must be written to files, not left only in the chat. Conversations get compacted; files don't. + +#### 1.3 Configure context `[required · once]` + +[Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +Curate `implement.jsonl` and `check.jsonl` so the Phase 2 sub-agents get the right spec/research context. These files were seeded on `task create` with a single self-describing `_example` line; your job here is to fill in real entries. + +**Location**: `{TASK_DIR}/implement.jsonl` and `{TASK_DIR}/check.jsonl` (already exist). + +**Format**: one JSON object per line — `{"file": "<path>", "reason": "<why>"}`. Paths are repo-root relative. + +**What to put in**: +- **Spec files** — `.trellis/spec/<package>/<layer>/index.md` and any specific guideline files (`error-handling.md`, `conventions.md`, etc.) relevant to this task +- **Research files** — `{TASK_DIR}/research/*.md` that the sub-agent will need to consult + +**What NOT to put in**: +- Code files (`src/**`, `packages/**/*.ts`, etc.) — those are read by the sub-agent during implementation, not pre-registered here +- Files you're about to modify — same reason + +**Split between the two files**: +- `implement.jsonl` → specs + research the implement sub-agent needs to write code correctly +- `check.jsonl` → specs for the check sub-agent (quality guidelines, check conventions, same research if needed) + +These manifests do not replace `implement.md`. `implement.md` is the human-readable execution plan for a complex task; jsonl files only list context files to inject or load. + +**How to discover relevant specs**: + +```bash +python ./.trellis/scripts/get_context.py --mode packages +``` + +Lists every package + its spec layers with paths. Pick the entries that match this task's domain. + +**How to append entries**: + +Either edit the jsonl file directly in your editor, or use: + +```bash +python ./.trellis/scripts/task.py add-context "$TASK_DIR" implement "<path>" "<reason>" +python ./.trellis/scripts/task.py add-context "$TASK_DIR" check "<path>" "<reason>" +``` + +Delete the seed `_example` line once real entries exist (optional — it's skipped automatically by consumers). + +Skip when: `implement.jsonl` and `check.jsonl` have agent-curated entries (the seed row alone doesn't count). + +[/Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +[codex-inline, Kilo, Antigravity, Windsurf] + +Skip this step. Context is loaded directly by the `trellis-before-dev` skill in Phase 2. + +[/codex-inline, Kilo, Antigravity, Windsurf] + +#### 1.4 Activate task `[required · once]` + +After artifact review, flip the task status to `in_progress`: + +```bash +python ./.trellis/scripts/task.py start <task-dir> +``` + +For lightweight tasks, `prd.md` can be enough. For complex tasks, `prd.md`, `design.md`, and `implement.md` must exist and be reviewed before start. On sub-agent-capable platforms, curate jsonl manifests when extra spec or research context is needed; seed-only manifests are tolerated by consumers. + +After this command succeeds, the breadcrumb auto-switches to `[workflow-state:in_progress]`, and the rest of Phase 2 / 3 follows. + +If `task.py start` errors with a session-identity message (no context key from hook input, `TRELLIS_CONTEXT_ID`, or platform-native session env), follow the hint in the error to set up session identity, then retry. + +#### 1.5 Completion criteria + +| Condition | Required | +|------|:---:| +| `prd.md` exists | ✅ | +| User confirms task should enter implementation | ✅ | +| `task.py start` has been run (status = in_progress) | ✅ | +| `research/` has artifacts (complex tasks) | recommended | +| `design.md` exists (complex tasks) | ✅ | +| `implement.md` exists (complex tasks) | ✅ | + +[Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +| `implement.jsonl` / `check.jsonl` curated when extra spec or research context is needed | recommended | + +[/Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +--- + +## Phase 2: Execute + +Goal: turn reviewed planning artifacts into code that passes quality checks. + +#### 2.1 Implement `[required · repeatable]` + +[Claude Code, Cursor, OpenCode, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +Spawn the implement sub-agent: + +- **Agent type**: `trellis-implement` +- **Task description**: Implement the reviewed task artifacts, consulting materials under `{TASK_DIR}/research/`; finish by running project lint and type-check +- **Dispatch prompt guard**: Tell the spawned agent it is already the `trellis-implement` sub-agent and must implement directly, not spawn another `trellis-implement` / `trellis-check`. + +The platform hook/plugin auto-handles: +- Reads `implement.jsonl` and injects referenced spec/research files into the agent prompt +- Injects `prd.md`, `design.md` if present, and `implement.md` if present + +[/Claude Code, Cursor, OpenCode, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +[codex-sub-agent] + +Spawn the implement sub-agent: + +- **Agent type**: `trellis-implement` +- **Task description**: Implement the reviewed task artifacts, consulting materials under `{TASK_DIR}/research/`; finish by running project lint and type-check +- **Dispatch prompt guard**: The prompt MUST start with `Active task: <task path>`, then explicitly say the spawned agent is already `trellis-implement` and must implement directly without spawning another `trellis-implement` / `trellis-check`. + +The Codex sub-agent definition auto-handles the context load requirement: +- Resolves the active task with `task.py current --source`, then reads `prd.md`, `design.md` if present, and `implement.md` if present +- Reads `implement.jsonl` and requires the agent to load each referenced spec/research file before coding + +[/codex-sub-agent] + +[Kiro] + +Spawn the implement sub-agent: + +- **Agent type**: `trellis-implement` +- **Task description**: Implement the reviewed task artifacts, consulting materials under `{TASK_DIR}/research/`; finish by running project lint and type-check +- **Dispatch prompt guard**: Tell the spawned agent it is already the `trellis-implement` sub-agent and must implement directly, not spawn another `trellis-implement` / `trellis-check`. + +The platform prelude auto-handles the context load requirement: +- Reads `implement.jsonl` and injects referenced spec/research files into the agent prompt +- Injects `prd.md`, `design.md` if present, and `implement.md` if present + +[/Kiro] + +[codex-inline, Kilo, Antigravity, Windsurf] + +1. Load the `trellis-before-dev` skill to read project guidelines +2. Read `{TASK_DIR}/prd.md`, then `design.md` if present, then `implement.md` if present +3. Consult materials under `{TASK_DIR}/research/` +4. Implement the code per reviewed artifacts +5. Run project lint and type-check + +[/codex-inline, Kilo, Antigravity, Windsurf] + +#### 2.2 Quality check `[required · repeatable]` + +[Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +Spawn the check sub-agent: + +- **Agent type**: `trellis-check` +- **Task description**: Review all code changes against specs and task artifacts; fix any findings directly; ensure lint and type-check pass +- **Dispatch prompt guard**: Tell the spawned agent it is already the `trellis-check` sub-agent and must review/fix directly, not spawn another `trellis-check` / `trellis-implement`. + +The check agent's job: +- Review code changes against specs +- Review code changes against `prd.md`, `design.md` if present, and `implement.md` if present +- Auto-fix issues it finds +- Run lint and typecheck to verify + +[/Claude Code, Cursor, OpenCode, codex-sub-agent, Kiro, Gemini, Qoder, CodeBuddy, Copilot, Droid, Pi] + +[codex-inline, Kilo, Antigravity, Windsurf] + +Load the `trellis-check` skill and verify the code per its guidance: +- Spec compliance +- lint / type-check / tests +- Cross-layer consistency (when changes span layers) + +If issues are found → fix → re-check, until green. + +[/codex-inline, Kilo, Antigravity, Windsurf] + +#### 2.3 Rollback `[on demand]` + +- `check` reveals a prd defect → return to Phase 1, fix `prd.md`, then redo 2.1 +- Implementation went wrong → revert code, redo 2.1 +- Need more research → research (same as Phase 1.2), write findings into `research/` + +--- + +## Phase 3: Finish + +Goal: ensure code quality, capture lessons, record the work. + +#### 3.1 Quality verification `[required · repeatable]` + +Load the `trellis-check` skill and do a final verification: +- Spec compliance +- lint / type-check / tests +- Cross-layer consistency (when changes span layers) + +If issues are found → fix → re-check, until green. + +#### 3.2 Debug retrospective `[on demand]` + +If this task involved repeated debugging (the same issue was fixed multiple times), load the `trellis-break-loop` skill to: +- Classify the root cause +- Explain why earlier fixes failed +- Propose prevention + +The goal is to capture debugging lessons so the same class of issue doesn't recur. + +#### 3.3 Spec update `[required · once]` + +Load the `trellis-update-spec` skill and review whether this task produced new knowledge worth recording: +- Newly discovered patterns or conventions +- Pitfalls you hit +- New technical decisions + +Update the docs under `.trellis/spec/` accordingly. Even if the conclusion is "nothing to update", walk through the judgment. + +#### 3.4 Commit changes `[required · once]` + +The AI drives a batched commit of this task's code changes so `/finish-work` can run cleanly afterwards. Goal: produce work commits FIRST, then bookkeeping (archive + journal) commits land after — never interleaved. + +**Step-by-step**: + +1. **Inspect dirty state**: + ```bash + git status --porcelain + ``` + Snapshot every dirty path. If the working tree is clean, skip to 3.5. + +2. **Learn commit style** from recent history (so drafted messages blend in): + ```bash + git log --oneline -5 + ``` + Note the prefix convention (`feat:` / `fix:` / `chore:` / `docs:` ...), language (中文/English), and length style. + +3. **Classify dirty files into two groups**: + - **AI-edited this session** — files you wrote/edited via Edit/Write/Bash tool calls in this session. You know what changed and why. + - **Unrecognized** — dirty files you did NOT touch this session (could be the user's manual edits, leftover WIP from a previous session, or unrelated work). Do NOT silently include these. + +4. **Draft a commit plan**. Group AI-edited files into logical commits (1 commit per coherent change unit, not 1 commit per file). Each entry: `<commit message>` + file list. List unrecognized files separately at the bottom. + +5. **Present the plan once, ask for one-shot confirmation**. Format: + ``` + Proposed commits (in order): + 1. <message> + - <file> + - <file> + 2. <message> + - <file> + + Unrecognized dirty files (NOT in any commit — confirm include/exclude): + - <file> + - <file> + + Reply 'ok' / '行' to execute. Reply with edits, or '我自己来' / 'manual' to abort. + ``` + +6. **On confirmation**: run `git add <files>` + `git commit -m "<msg>"` for each batch in order. Do not amend. Do not push. + +7. **On rejection** (user replies "不行" / "我自己来" / "manual" / any pushback on the plan): stop. Do not attempt a second plan. The user will commit by hand; you skip ahead to 3.5 once they confirm. + +**Rules**: +- No `git commit --amend` anywhere — three-stage three-commit flow (work commits → archive commit → journal commit). +- Never push to remote in this step. +- If the user wants different message wording but accepts the file grouping, edit the message and re-confirm once — but if they reject the grouping, exit to manual mode. +- The batched plan is one prompt; do not prompt per commit. + +#### 3.5 Wrap-up reminder + +After the above, remind the user they can run `/finish-work` to wrap up (archive the task, record the session). + +--- + +## Customizing Trellis (for forks) + +This section is for developers who want to modify the Trellis workflow itself. All customization is done by editing this file; the scripts are parsers only. + +### Changing what a step means + +Edit the corresponding step's walkthrough body in the Phase 1 / 2 / 3 sections above. Critical invariants: +- No active task must triage first and ask for task-creation consent before creating a Trellis task. +- Planning must distinguish lightweight PRD-only tasks from complex tasks that require `prd.md`, `design.md`, and `implement.md` before start. +- Every required execution path must keep the Phase 3.4 commit reminder reachable before `/trellis:finish-work`. + +All tag blocks live in the `## Phase Index` section above, immediately after each phase summary: + +| Scope | Corresponding tag | +|---|---| +| No active task (before Phase 1) | `[workflow-state:no_task]` (after the Phase Index ASCII art) | +| All of Phase 1 (task created → ready for implementation) | `[workflow-state:planning]` (after Phase 1 summary) | +| Codex inline Phase 1 | `[workflow-state:planning-inline]` | +| Phase 2 + Phase 3.1–3.4 (implementation + check + wrap-up) | `[workflow-state:in_progress]` (after Phase 2 summary) | +| Codex inline Phase 2 + Phase 3.1–3.4 | `[workflow-state:in_progress-inline]` | +| After Phase 3.5 (archived) | `[workflow-state:completed]` (after Phase 3 summary; **currently DEAD**) | + +### Changing the per-turn prompt text + +Directly edit the body of the corresponding `[workflow-state:STATUS]` block. After editing, run `trellis update` (if you're a template maintainer) or restart your AI session (if you're customizing your own project) — no script changes required. + +### Adding a custom status + +Add a new block: + +``` +[workflow-state:my-status] +your per-turn prompt text +[/workflow-state:my-status] +``` + +Constraints: +- STATUS charset: `[A-Za-z0-9_-]+` (underscores and hyphens allowed, e.g. `in-review`, `blocked-by-team`) +- A lifecycle hook must write `task.json.status` to your custom value, otherwise the tag is never read +- Lifecycle hooks live in `task.json.hooks.after_*` and bind to one of `after_create / after_start / after_finish / after_archive` + +### Adding a lifecycle hook + +Add a `hooks` field to your `task.json`: + +```json +{ + "hooks": { + "after_finish": [ + "your-script-or-command-here" + ] + } +} +``` + +Supported events: `after_create / after_start / after_finish / after_archive`. Note that `after_finish` ≠ a status change (it only clears the active-task pointer); use `after_archive` for "task is done" notifications. + +### Full contract + +For the workflow state machine's runtime contract, the locations of all status writers, pseudo-statuses (`no_task` / `stale_<source_type>`), the hook reachability matrix, and other deep details, see: + +- `.trellis/spec/cli/backend/workflow-state-contract.md` — runtime contract + writer table + test invariants +- `.trellis/scripts/inject-workflow-state.py` — actual parser (reads workflow.md only, no embedded text) diff --git a/.trellis/workspace/dxt98/index.md b/.trellis/workspace/dxt98/index.md new file mode 100644 index 0000000..2a8e5e8 --- /dev/null +++ b/.trellis/workspace/dxt98/index.md @@ -0,0 +1,55 @@ +# Workspace Index - dxt98 + +> Journal tracking for AI development sessions. + +--- + +## Current Status + +<!-- @@@auto:current-status --> +- **Active File**: `journal-1.md` +- **Total Sessions**: 15 +- **Last Active**: 2026-07-04 +<!-- @@@/auto:current-status --> + +--- + +## Active Documents + +<!-- @@@auto:active-documents --> +| File | Lines | Status | +|------|-------|--------| +| `journal-1.md` | ~510 | Active | +<!-- @@@/auto:active-documents --> + +--- + +## Session History + +<!-- @@@auto:session-history --> +| # | Date | Title | Commits | Branch | +|---|------|-------|---------|--------| +| 15 | 2026-07-04 | Executor tool routing and native alias fix | `737befe` | `codex/codexcont-executor-migration` | +| 14 | 2026-07-04 | CPA Key Policy Plus SQLite/native key stabilization | `c799e47` | `codex/codexcont-executor-migration` | +| 13 | 2026-07-04 | Native key sync and Codex auth recovery | `837b48a` | `codex/codexcont-executor-migration` | +| 12 | 2026-07-03 | Migrate Key Policy Plus to native CPA keys | `b46c4d5` | `codex/codexcont-executor-migration` | +| 11 | 2026-07-03 | Retire Key Policy Plus session limits | `30fa42b` | `main` | +| 10 | 2026-07-02 | Key Policy Plus session cookie hotfix | `793b4ae`, `a5ddecb` | `main` | +| 9 | 2026-07-02 | CPA Key Policy Plus UX stability | `57aadff` | `main` | +| 8 | 2026-07-02 | CPA Key Policy Plus admin fixes | `4c6613b` | `main` | +| 7 | 2026-07-02 | CPA Key Policy Plus cutover | `160af51` | `main` | +| 6 | 2026-07-02 | CPA Governor and CodexCont engine rollout | `429bed5`, `d9047c8`, `50e77d0`, `69e429e` | `main` | +| 5 | 2026-07-02 | CPA usage request detail closeout | `432ca38` | `main` | +| 4 | 2026-07-02 | CPA usage quota admin | `762b9e6` | `main` | +| 3 | 2026-07-01 | CPA key management and usage portal | `b20a983`, `61911b0` | `main` | +| 2 | 2026-07-01 | CodexCont status dashboard | `82e54b8`, `5b578e1`, `42c1b14` | `main` | +| 1 | 2026-07-01 | SJC CPA migration closeout | `98bf0df`, `aa113c2` | `main` | +<!-- @@@/auto:session-history --> + +--- + +## Notes + +- Sessions are appended to journal files +- New journal file created when current exceeds 2000 lines +- Use `add_session.py` to record sessions \ No newline at end of file diff --git a/.trellis/workspace/dxt98/journal-1.md b/.trellis/workspace/dxt98/journal-1.md new file mode 100644 index 0000000..71fbf3b --- /dev/null +++ b/.trellis/workspace/dxt98/journal-1.md @@ -0,0 +1,510 @@ +# Journal - dxt98 (Part 1) + +> AI development session journal +> Started: 2026-07-01 + +--- + + + +## Session 1: SJC CPA migration closeout + +**Date**: 2026-07-01 +**Task**: SJC CPA migration closeout +**Branch**: `main` + +### Summary + +Recorded the completed SJC sub2api to CPA migration, captured Codex continuation/CPA integration contracts, and committed zstd request-body decoding tests. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `98bf0df` | (see git log) | +| `aa113c2` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 2: CodexCont status dashboard + +**Date**: 2026-07-01 +**Task**: CodexCont status dashboard +**Branch**: `main` + +### Summary + +Built and deployed the CodexCont admin dashboard, then upgraded it to a Chinese request-first protection status page with request summaries, SSE request updates, production validation, and captured the admin diagnostics contract in backend spec. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `82e54b8` | (see git log) | +| `5b578e1` | (see git log) | +| `42c1b14` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 3: CPA key management and usage portal + +**Date**: 2026-07-01 +**Task**: CPA key management and usage portal +**Branch**: `main` + +### Summary + +Deployed CPAMP, CPA Key Policy, and a separate user usage portal; clarified cpa_ versus sk key model, official component maintenance boundaries, hash contracts, and archived the completed task. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `b20a983` | (see git log) | +| `61911b0` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 4: CPA usage quota admin + +**Date**: 2026-07-02 +**Task**: CPA usage quota admin +**Branch**: `main` + +### Summary + +Added the custom CPA usage portal local quota admin, 5H/month windows, soft reset watermarks, server price correction evidence, and deployment validation without modifying CPA/CPAMP/Key Policy source. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `762b9e6` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 5: CPA usage request detail closeout + +**Date**: 2026-07-02 +**Task**: CPA usage request detail closeout +**Branch**: `main` + +### Summary + +Completed and deployed the CPA usage admin batch-save/request-detail task, including CPAMP-compatible cache semantics, safe key identity on CodexCont, all-key usage-admin events, server validation, and task archive. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `432ca38` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 6: CPA Governor and CodexCont engine rollout + +**Date**: 2026-07-02 +**Task**: CPA Governor and CodexCont engine rollout +**Branch**: `main` + +### Summary + +Built and deployed the CPA Governor plugin in passive mode, added the CodexCont engine surface, unified Governor admin/user pages, fixed Key Policy login sync and CPAMP embedded user-login header conflicts, recorded Key Policy rotation and no-store cache contracts, and archived the completed task. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `429bed5` | (see git log) | +| `d9047c8` | (see git log) | +| `50e77d0` | (see git log) | +| `69e429e` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 7: CPA Key Policy Plus cutover + +**Date**: 2026-07-02 +**Task**: CPA Key Policy Plus cutover +**Branch**: `main` + +### Summary + +Implemented and deployed cpa-key-policy-plus as the unified cpa_ key authority, migrated limits/state, retired usage-admin backend, verified cpa-usage login/API/UI, and kept public /v1/responses on the known-good CodexCont sidecar until executor-level folding is ready. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `160af51` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 8: CPA Key Policy Plus admin fixes + +**Date**: 2026-07-02 +**Task**: CPA Key Policy Plus admin fixes +**Branch**: `main` + +### Summary + +Fixed Key Policy+ admin create/save/reset transport, added model discovery and structured model/price editing, deployed the linux/amd64 plugin to SJC, added admin proxy management alias, and verified local/server smoke tests. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `4c6613b` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 9: CPA Key Policy Plus UX stability + +**Date**: 2026-07-02 +**Task**: CPA Key Policy Plus UX stability +**Branch**: `main` + +### Summary + +Stabilized CPA Key Policy+ admin and user UX, added archive/restore lifecycle, improved user refresh error recovery, deployed the linux/amd64 plugin to SJC, and verified cpa-usage production login, refresh, tabs, and route boundaries. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `57aadff` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 10: Key Policy Plus session cookie hotfix + +**Date**: 2026-07-02 +**Task**: Key Policy Plus session cookie hotfix +**Branch**: `main` + +### Summary + +Diagnosed cpa-usage login loops caused by stale path-specific Key Policy Plus session cookies; deployed a plugin hotfix that refreshes compatible cookie paths and accepts the first valid same-name session token; captured the cookie-path contract in backend spec. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `793b4ae` | (see git log) | +| `a5ddecb` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 11: Retire Key Policy Plus session limits + +**Date**: 2026-07-03 +**Task**: Retire Key Policy Plus session limits +**Branch**: `main` + +### Summary + +Retired Key Policy+ request concurrency and Codex active-window enforcement, added hard delete for keys, unified Usage/Governor chip styling, documented the RPM-only and hard-delete contracts, deployed to SJC, and cleaned disabled production keys. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `30fa42b` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 12: Migrate Key Policy Plus to native CPA keys + +**Date**: 2026-07-03 +**Task**: Migrate Key Policy Plus to native CPA keys +**Branch**: `codex/codexcont-executor-migration` + +### Summary + +Implemented and deployed CPA Key Policy+ as a passive policy layer over CPA native keys, verified cpa-usage portal, normal Responses calls, structured quota denial body, and documented the CPA executor ABI status/header limitation. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `b46c4d5` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 13: Native key sync and Codex auth recovery + +**Date**: 2026-07-04 +**Task**: Native key sync and Codex auth recovery +**Branch**: `codex/codexcont-executor-migration` + +### Summary + +Completed the native CPA key sync and gpt-5.5 routing rollout, then restored production /v1/responses by replacing the invalidated CPA Codex OAuth auth file with a fresh login-derived auth file. Verified non-stream and stream gpt-5.5 smokes, Plus usage page, and public admin-route blocks. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `837b48a` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 14: CPA Key Policy Plus SQLite/native key stabilization + +**Date**: 2026-07-04 +**Task**: CPA Key Policy Plus SQLite/native key stabilization +**Branch**: `codex/codexcont-executor-migration` + +### Summary + +Hardened Plus SQLite access, made native CPA keys a current-state mirror with default-enabled rows and missing-limit UI hints, deployed the c-shared plugin to SJC, and verified cpa-usage plus real gpt-5.5 /v1/responses end to end. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `c799e47` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete + + +## Session 15: Executor tool routing and native alias fix + +**Date**: 2026-07-04 +**Task**: Executor tool routing and native alias fix +**Branch**: `codex/codexcont-executor-migration` + +### Summary + +Fixed CodexCont Executor upstream tool filtering for gpt-5.5 alias routing, fixed Plus native key alias fallback for CPAMP WAL-backed alias DB, deployed and verified SJC smokes. + +### Main Changes + +(Add details) + +### Git Commits + +| Hash | Message | +|------|---------| +| `737befe` | (see git log) | + +### Testing + +- [OK] (Add test results) + +### Status + +[OK] **Completed** + +### Next Steps + +- None - task complete diff --git a/.trellis/workspace/index.md b/.trellis/workspace/index.md new file mode 100644 index 0000000..cb8e1f3 --- /dev/null +++ b/.trellis/workspace/index.md @@ -0,0 +1,125 @@ +# Workspace Index + +> Records of all AI Agent work records across all developers + +--- + +## Overview + +This directory tracks records for all developers working with AI Agents on this project. + +### File Structure + +``` +workspace/ +|-- index.md # This file - main index ++-- {developer}/ # Per-developer directory + |-- index.md # Personal index with session history + |-- tasks/ # Task files + | |-- *.json # Active tasks + | +-- archive/ # Archived tasks by month + +-- journal-N.md # Journal files (sequential: 1, 2, 3...) +``` + +--- + +## Active Developers + +| Developer | Last Active | Sessions | Active File | +|-----------|-------------|----------|-------------| +| (none yet) | - | - | - | + +--- + +## Getting Started + +### For New Developers + +Run the initialization script: + +```bash +python ./.trellis/scripts/init_developer.py <your-name> +``` + +This will: +1. Create your identity file (gitignored) +2. Create your progress directory +3. Create your personal index +4. Create initial journal file + +### For Returning Developers + +1. Get your developer name: + ```bash + python ./.trellis/scripts/get_developer.py + ``` + +2. Read your personal index: + ```bash + cat .trellis/workspace/$(python ./.trellis/scripts/get_developer.py)/index.md + ``` + +--- + +## Guidelines + +### Journal File Rules + +- **Max 2000 lines** per journal file +- When limit is reached, create `journal-{N+1}.md` +- Update your personal `index.md` when creating new files + +### Session Record Format + +Each session should include: +- Summary: One-line description +- Branch: Which branch the work was done on +- Main Changes: What was modified +- Git Commits: Commit hashes and messages +- Next Steps: What to do next + +--- + +## Session Template + +Use this template when recording sessions: + +```markdown +## Session {N}: {Title} + +**Date**: YYYY-MM-DD +**Task**: {task-name} +**Branch**: `{branch-name}` + +### Summary + +{One-line summary} + +### Main Changes + +- {Change 1} +- {Change 2} + +### Git Commits + +| Hash | Message | +|------|---------| +| `abc1234` | {commit message} | + +### Testing + +- [OK] {Test result} + +### Status + +[OK] **Completed** / # **In Progress** / [P] **Blocked** + +### Next Steps + +- {Next step 1} +- {Next step 2} +``` + +--- + +**Language**: All documentation must be written in **English**. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..c9c4c66 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,21 @@ +<!-- TRELLIS:START --> +# Trellis Instructions + +These instructions are for AI assistants working in this project. + +This project is managed by Trellis. The working knowledge you need lives under `.trellis/`: + +- `.trellis/workflow.md` — development phases, when to create tasks, skill routing +- `.trellis/spec/` — package- and layer-scoped coding guidelines (read before writing code in a given layer) +- `.trellis/workspace/` — per-developer journals and session traces +- `.trellis/tasks/` — active and archived tasks (PRDs, research, jsonl context) + +If a Trellis command is available on your platform (e.g. `/trellis:finish-work`, `/trellis:continue`), prefer it over manual steps. Not every platform exposes every command. + +If you're using Codex or another agent-capable tool, additional project-scoped helpers may live in: +- `.agents/skills/` — reusable Trellis skills +- `.codex/agents/` — optional custom subagents + +Managed by Trellis. Edits outside this block are preserved; edits inside may be overwritten by a future `trellis update`. + +<!-- TRELLIS:END --> diff --git a/README.md b/README.md index 3545c25..a30052f 100644 --- a/README.md +++ b/README.md @@ -107,6 +107,46 @@ Security guard: if a request supplies `Responses-API-Base`, the middleware will Do not commit secrets. `rt.json` and `free_rt.json` are ignored by `.gitignore`, and tokens in `config.toml` should be handled carefully. +## Dashboard + +CodexCont serves a lightweight read-only dashboard at: + +```text +http://127.0.0.1:8787/admin/ +``` + +It exposes service status, upstream health, in-memory request metrics, recent request protection summaries, and live redacted logs through SSE. The request view distinguishes protected-clean, auto-continued, risky-uncontinued, passthrough, failed, and incomplete requests. Log history is memory-only and bounded by: + +```toml +[admin] +max_log_events = 800 +key_policy_state_path = "" # optional; set to the CPA Key Policy state JSON to show safe key names +``` + +When running in Docker, this must be a path visible inside the CodexCont +container. For example, mount the CPA plugin state read-only and set +`key_policy_state_path = "/data/plugin-state/cpa-key-policy-state.json"`. + +Do not expose `/admin/` on a public API hostname unless it is protected by an external access layer such as Cloudflare Access. + +## CPA Usage Portal + +This repository also includes a small self-service sidecar in `cpa_usage_portal/`. +It is designed to sit next to CPA Manager Plus and CPA Key Policy: users log in +with their own `cpa_...` key and can only see usage filtered to that key. The +portal validates login with the raw key's Key Policy hash, but filters CPAMP +usage with `sha256(Key Policy id)` because the plugin authenticates to CPA as +the key id. Raw API keys, OAuth tokens, management keys, request bodies, +response bodies, and encrypted reasoning content are never returned. + +Run locally: + +```bash +.venv/Scripts/python.exe run_usage_portal.py +``` + +Docker deployment templates live in `deploy/cpa-usage-portal/`. + ## When continuation is applied The middleware folds only when all of the following are true: @@ -163,25 +203,32 @@ Current offline coverage includes: - header transparency - upstream URL resolution - auth safety guard +- dashboard diagnostics and admin route smoke +- CPA usage portal hash/session/filtering/redaction/retention behavior - EOF/upstream-error behavior ## Project layout ```text middleware/ + admin.py # read-only dashboard/admin routes app.py # Starlette app and route handler codex.py # truncation math and continuation payload builders config.py # config.toml loader and dataclasses creds.py # upstream header/auth construction + dashboard.html # static dashboard page + diagnostics.py # in-memory metrics, ring buffer, and SSE subscribers proxy.py # fold_stream state machine sse.py # incremental SSE parser/serializer store.py # in-memory ID store for optional stateful repair tests/ test_middleware.py + test_cpa_usage_portal.py fixtures/ run.py # uvicorn entrypoint +run_usage_portal.py # CPA usage portal entrypoint config.example.toml # example runtime configuration; copy to config.toml for local use ``` diff --git a/README_zh.md b/README_zh.md index 37376ae..e2d776e 100644 --- a/README_zh.md +++ b/README_zh.md @@ -105,6 +105,119 @@ chatgpt_account_id = "" # 非空时作为 chatgpt-account-id 发送 不要提交密钥。`.gitignore` 已忽略 `rt.json` 和 `free_rt.json`；如果把 token 写入 `config.toml`，也请谨慎管理。 +## 状态面板 + +CodexCont 内置一个只读状态面板： + +```text +http://127.0.0.1:8787/admin/ +``` + +它可以查看服务状态、上游健康、内存中的请求指标，以及通过 SSE 实时推送的脱敏日志。当前面板优先展示“最近请求”的中文保护结果，能直接区分： + +- `已保护 / 无需续写`：请求进入 CodexCont 保护链，未命中 516/518n-2 指纹。 +- `已自动续写`：检测到 516/518n-2，并已打开隐藏续写轮。 +- `疑似截断 / 未续写`：检测到截断指纹，但保护条件或上限阻止了续写。 +- `透传` / `失败` / `不完整`：分别表示未进入折叠保护、请求失败、上游未完整结束。 + +面板里的“末轮思考量”只表示最后一轮上游响应的 reasoning token；如果某个请求被自动续写，真正触发续写的 516/518n-2 会显示在“命中轮”列里。 + +日志和最近请求都只保存在进程内存中，默认日志上限为： + +```toml +[admin] +max_log_events = 800 +key_policy_state_path = "" # 可选：填 CPA Key Policy state JSON 路径后显示安全的 Key 名称 +``` + +Docker 部署时这里必须填容器内部可读到的路径。例如把 CPA plugin state +只读挂载到容器内，然后设置 +`key_policy_state_path = "/data/plugin-state/cpa-key-policy-state.json"`。 + +不要把 `/admin/` 直接暴露在公网 API 域名上；生产环境应放在 Cloudflare Access 这类外层访问控制之后。 + +## CPA 用量自助页 + +本仓库还包含一个独立的 CPA 用量自助页 sidecar： + +```bash +.venv/Scripts/python.exe run_usage_portal.py +``` + +它不是 CPA fork，也不是 CPAMP 的替代品。推荐生产链路是： + +```text +CPA Key Policy 负责 key 和限额 +CPAMP 负责请求级用量采集 +cpa_usage_portal 只给普通用户看自己的用量 +``` + +关键环境变量： + +```text +CPA_USAGE_PORTAL_KEY_POLICY_STATE=/data/cpa-key-policy-state.json +CPA_USAGE_PORTAL_CPAMP_URL=http://cpamp:18317 +CPA_USAGE_PORTAL_CPAMP_ADMIN_KEY_FILE=/run/secrets/cpamp_admin_key +CPA_USAGE_PORTAL_SESSION_SECRET_FILE=/run/secrets/session_secret +``` + +用户只在登录时提交自己的 `cpa_...` API Key。服务端会用 `sha256` 校验 Key Policy state，并在 HttpOnly cookie 中只保存用于会话校验的哈希和安全元数据；页面和接口不会返回原始 key、完整 key hash、OAuth token、CPA/CPAMP 管理密钥、请求正文、响应正文或 encrypted reasoning 内容。 + +Docker 部署模板在： + +```text +deploy/cpa-usage-portal/ +``` + +## CPAMP 与 Key Policy 配合 + +计划中的生产组合是： + +- CPAMP：管理员面板和请求级监控，放在 `cpa-admin.konbakuyomu.us` + Cloudflare Access 后面。 +- CPA Key Policy：生成 `cpa_...` key，配置每个 key 的模型权限、RPM、每日/每周 USD 限额。 +- CPA 用量自助页：普通用户用自己的 key 登录，只能看自己的用量和最近请求。 + +Key Policy 有两个容易混淆的标识： + +- `key_hash`：原始 `cpa_...` key 的 `sha256:<hex>`，只用于登录校验。 +- `id`：Key Policy 交给 CPA 的 Principal；CPAMP monitoring 里的 `api_key_hash` 实际是 `sha256(id)`。 + +因此门户登录时用原始 key hash 找到 Key Policy 记录，查询 CPAMP 时改用该记录 `id` 的 hash。所有 CPAMP 查询都会强制带当前登录 key 对应的 `api_key_hash` 过滤。 + +### Key 体系 + +生产上推荐把普通用户统一迁移到 Key Policy 生成的 `cpa_...` key： + +```text +Codex Base URL: https://cpa.konbakuyomu.us/v1 +Codex API Key: cpa_... +用量自助页: https://cpa-usage.konbakuyomu.us/ +``` + +CPA 原生 `api-keys` 里的 `sk...` key 只建议保留为管理员兼容/救急入口，不作为普通用户分发体系。它不天然带 Key Policy 的用户身份、模型 allowlist、RPM、每日/每周限额，也不作为用量自助页的登录凭据。 + +管理面板也有两类密钥： + +- CPAMP 管理面板登录使用 CPAMP admin key。 +- CPA 原生 management API 使用 CPA management key。 + +当前 `https://cpa-admin.konbakuyomu.us/management.html` 指向 CPAMP，所以登录它需要 CPAMP admin key，而不是 CPA management key。 + +### 维护边界 + +CPA、CPAMP、CPA Key Policy 都保持官方来源，不在本仓库二开： + +- CPA：官方镜像 `eceasy/cli-proxy-api:latest`。 +- CPAMP：官方镜像 `seakee/cpa-manager-plus:latest`。 +- CPA Key Policy：官方 release 的 `cpa-key-policy.so`，挂载到 CPA 插件目录。 + +本仓库自定义维护的只有： + +- `CodexCont`：负责 516/518n-2 续写保护。 +- `cpa_usage_portal`：普通用户用量自助页。 + +服务器上它们是独立 stack / 独立容器，后续可以分别更新 CPA、CPAMP、Key Policy、CodexCont 和用户自助页。 + ## 什么时候会执行续写折叠 只有同时满足以下条件时，中间件才会执行折叠逻辑： @@ -161,25 +274,40 @@ uv run python tests/test_middleware.py - header 透明转发 - 上游 URL 解析 - 鉴权安全保护 +- dashboard diagnostics 和 admin route smoke +- CPA 用量自助页的 key hash/session/过滤/脱敏/retention 行为 - EOF / 上游错误处理 ## 项目结构 ```text middleware/ + admin.py # 只读状态面板 / admin 路由 app.py # Starlette 应用和路由处理 codex.py # 截断数学和续写 payload 构造 config.py # config.toml 加载和 dataclass 配置 creds.py # 上游 header / auth 构造 + dashboard.html # 静态状态面板页面 + diagnostics.py # 内存指标、环形日志和 SSE 订阅 proxy.py # fold_stream 状态机 sse.py # 增量 SSE 解析和序列化 store.py # 可选 stateful repair 使用的内存 ID 存储 +cpa_usage_portal/ + app.py # 用户用量自助页 Starlette 应用 + key_policy.py # CPA Key Policy state 只读解析 + cpamp.py # CPAMP monitoring API client + redaction.py # 用户可见事件脱敏投影 + retention.py # CPAMP SQLite 7 天保留辅助 + static/dashboard.html # 中文自助页 + tests/ test_middleware.py + test_cpa_usage_portal.py fixtures/ run.py # uvicorn 入口 +run_usage_portal.py # CPA 用量自助页入口 config.example.toml # 示例运行配置；复制为 config.toml 后本地使用 ``` diff --git a/config.example.toml b/config.example.toml index 39b9a03..ba2ef80 100644 --- a/config.example.toml +++ b/config.example.toml @@ -50,3 +50,6 @@ rechunk_size = 16 [log] level = "info" dump_rounds_dir = "" # if set, dump per-round upstream SSE (codex_mw_r{n}.sse.txt) + +[admin] +max_log_events = 800 # memory-only dashboard log retention; no log files are written diff --git a/cpa_codexcont_executor_plugin/README.md b/cpa_codexcont_executor_plugin/README.md new file mode 100644 index 0000000..e912dc1 --- /dev/null +++ b/cpa_codexcont_executor_plugin/README.md @@ -0,0 +1,67 @@ +# CPA CodexCont Executor Plugin + +`cpa-codexcont-executor` is an executor-only CPA plugin that replaces the old +Docker-hosted CodexCont sidecar for streaming Responses continuation +protection. + +It does not own the ordinary user portal. `cpa-key-policy-plus` remains the +owner of `https://cpa-usage.konbakuyomu.us/`, user login, quotas, RPM, and usage +APIs. + +It does own the read-only CPAMP-side `CodexCont Executor` monitor. That page is +for rolling protection summaries and executor health only; it must not expose +user login, key editing, quota editing, raw request/response bodies, or +encrypted reasoning. + +The plugin declares non-exclusive `frontend_auth_provider=true` and +`usage_plugin=true` only so CPA/CPAMP registers its resource menu. +`frontend_auth.authenticate` always returns unauthenticated and `usage.handle` +is intentionally a no-op; billing, quota, RPM, and user usage remain owned by +`cpa-key-policy-plus`. + +## Local Test + +```powershell +cd D:\Dev\20_Software\23_Reference\llm-gateway\CodexCont\cpa_codexcont_executor_plugin\go +go test ./... +``` + +## Linux Build + +```bash +cd cpa_codexcont_executor_plugin/go +CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags cliproxy_plugin -buildmode=c-shared -o cpa-codexcont-executor.so . +``` + +## Minimal CPA Config + +```yaml +plugins: + enabled: true + dir: /CLIProxyAPI/plugins + configs: + cpa-codexcont-executor: + enabled: true + route_enabled: false + state_db_path: /CLIProxyAPI/plugin-state/cpa-codexcont-executor/executor.sqlite + fail_mode: fallback +``` + +`route_enabled: false` leaves CPA's normal upstream route untouched. Turn it on +only after local and server executor folding validation passes. + +Both `plugin.register` and `plugin.reconfigure` must return the full plugin +registration object. CPA decodes reconfigure through the same metadata and +capability path; a lightweight acknowledgement causes CPA to drop the plugin +from the active resource snapshot. + +## CPAMP Monitor + +The plugin registers a read-only admin resource: + +```text +/v0/resource/plugins/cpa-codexcont-executor/admin +``` + +The page polls the plugin's safe status and summary endpoints and is intended +to replace Governor's CodexCont realtime monitoring role during cleanup. diff --git a/cpa_codexcont_executor_plugin/go/assets/admin.html b/cpa_codexcont_executor_plugin/go/assets/admin.html new file mode 100644 index 0000000..cd35d69 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/assets/admin.html @@ -0,0 +1,306 @@ +<!doctype html> +<html lang="zh-CN"> +<head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width,initial-scale=1"> + <title>CodexCont Executor + + + +
+
+
+
CE
+
+

CodexCont Executor

+

实时滚动监控

+
+
+
+ 连接中 + route - + +
+
+ +
+ +
+
+

最近保护请求

+ +
+
+ + + + + + + + + + + + + + + +
时间请求保护模型耗时命中轮末轮 reasoning续写操作
+
+
+
+ + + + diff --git a/cpa_codexcont_executor_plugin/go/go.mod b/cpa_codexcont_executor_plugin/go/go.mod new file mode 100644 index 0000000..a0371cc --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/go.mod @@ -0,0 +1,24 @@ +module codexcont/cpa-codexcont-executor-plugin + +go 1.22 + +require ( + gopkg.in/yaml.v3 v3.0.1 + modernc.org/sqlite v1.33.1 +) + +require ( + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/ncruces/go-strftime v0.1.9 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + golang.org/x/sys v0.22.0 // indirect + modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect + modernc.org/libc v1.55.3 // indirect + modernc.org/mathutil v1.6.0 // indirect + modernc.org/memory v1.8.0 // indirect + modernc.org/strutil v1.2.0 // indirect + modernc.org/token v1.1.0 // indirect +) diff --git a/cpa_codexcont_executor_plugin/go/go.sum b/cpa_codexcont_executor_plugin/go/go.sum new file mode 100644 index 0000000..6ea7e08 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/go.sum @@ -0,0 +1,53 @@ +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= +github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic= +golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI= +golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw= +golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ= +modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ= +modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y= +modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s= +modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE= +modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ= +modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw= +modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4= +modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U= +modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w= +modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4= +modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo= +modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E= +modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU= +modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4= +modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= +modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc= +modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss= +modernc.org/sqlite v1.33.1 h1:trb6Z3YYoeM9eDL1O8do81kP+0ejv+YzgyFo+Gwy0nM= +modernc.org/sqlite v1.33.1/go.mod h1:pXV2xHxhzXZsgT/RtTFAPY6JJDEvOTcTdwADQCCWD4k= +modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA= +modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/cpa_codexcont_executor_plugin/go/internal/executor/codex.go b/cpa_codexcont_executor_plugin/go/internal/executor/codex.go new file mode 100644 index 0000000..f116e25 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/internal/executor/codex.go @@ -0,0 +1,148 @@ +package executor + +import ( + "encoding/json" + "math" +) + +func IsTruncationPattern(tokens int64, ok bool, step int) bool { + return ok && tokens >= int64(step-2) && (tokens+2)%int64(step) == 0 +} + +func TierN(tokens int64, ok bool, step int) (int64, bool) { + if !IsTruncationPattern(tokens, ok, step) { + return 0, false + } + return (tokens + 2) / int64(step), true +} + +func ShouldContinue(tokens int64, ok bool, minN, maxN, step int) bool { + n, match := TierN(tokens, ok, step) + if !match { + return false + } + if n < int64(minN) { + return false + } + return maxN <= 0 || n <= int64(maxN) +} + +func ReasoningTokens(usage map[string]any) (int64, bool) { + details := mapValue(usage, "output_tokens_details") + return intValue(details["reasoning_tokens"]) +} + +func CommentaryMessage(text string) map[string]any { + return map[string]any{ + "type": "message", + "role": "assistant", + "phase": "commentary", + "content": []any{ + map[string]any{"type": "output_text", "text": text}, + }, + } +} + +func BuildRoundPayload(base map[string]any, input []any, cfg Config, dropPreviousResponseID bool) map[string]any { + out := cloneMap(base) + out["stream"] = true + out["input"] = input + if cfg.ForceIncludeEncrypted || base["include"] != nil { + out["include"] = mergeInclude(base["include"], cfg.ForceIncludeEncrypted) + } + if dropPreviousResponseID { + delete(out, "previous_response_id") + } + return out +} + +func BuildFirstPayload(base map[string]any) map[string]any { + out := cloneMap(base) + out["stream"] = true + return out +} + +func mergeInclude(raw any, forceEncrypted bool) []any { + seen := map[string]bool{} + var out []any + if list, ok := raw.([]any); ok { + for _, item := range list { + text := toString(item) + if text == "" || seen[text] { + continue + } + seen[text] = true + out = append(out, text) + } + } + if forceEncrypted && !seen[EncryptedInclude] { + out = append(out, EncryptedInclude) + } + return out +} + +func mapValue(raw any, key string) map[string]any { + m, _ := raw.(map[string]any) + if key == "" { + return m + } + nested, _ := m[key].(map[string]any) + return nested +} + +func anyList(raw any) []any { + if raw == nil { + return nil + } + if list, ok := raw.([]any); ok { + return append([]any(nil), list...) + } + return nil +} + +func cloneMap(in map[string]any) map[string]any { + out := make(map[string]any, len(in)) + for k, v := range in { + out[k] = v + } + return out +} + +func cloneEvent(in map[string]any) map[string]any { + raw, err := json.Marshal(in) + if err != nil { + return cloneMap(in) + } + var out map[string]any + if err := json.Unmarshal(raw, &out); err != nil { + return cloneMap(in) + } + return out +} + +func intValue(raw any) (int64, bool) { + switch v := raw.(type) { + case int: + return int64(v), true + case int64: + return v, true + case float64: + if math.Trunc(v) == v { + return int64(v), true + } + case json.Number: + n, err := v.Int64() + return n, err == nil + } + return 0, false +} + +func toString(raw any) string { + if raw == nil { + return "" + } + if text, ok := raw.(string); ok { + return text + } + return "" +} diff --git a/cpa_codexcont_executor_plugin/go/internal/executor/config.go b/cpa_codexcont_executor_plugin/go/internal/executor/config.go new file mode 100644 index 0000000..0665751 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/internal/executor/config.go @@ -0,0 +1,90 @@ +package executor + +import ( + "strings" + "time" +) + +const EncryptedInclude = "reasoning.encrypted_content" +const DefaultCodexUpstreamModel = "gpt-5.3-codex-spark" + +type Config struct { + Enabled bool `yaml:"enabled"` + RouteEnabled bool `yaml:"route_enabled"` + StateDBPath string `yaml:"state_db_path"` + FailMode string `yaml:"fail_mode"` + UpstreamModel string `yaml:"upstream_model"` + UpstreamModelAliases map[string]string `yaml:"upstream_model_aliases"` + TruncationStep int `yaml:"truncation_step"` + MaxContinue int `yaml:"max_continue"` + MinN int `yaml:"min_n"` + MaxN int `yaml:"max_n"` + MarkerText string `yaml:"marker_text"` + ForceIncludeEncrypted bool `yaml:"force_include_encrypted"` + MaxTotalOutputTokens int `yaml:"max_total_output_tokens"` + PollIntervalMS int `yaml:"poll_interval_ms"` +} + +func DefaultConfig() Config { + return Config{ + Enabled: true, + RouteEnabled: false, + StateDBPath: "cpa-codexcont-executor.sqlite", + FailMode: "fallback", + UpstreamModelAliases: map[string]string{ + "gpt-5.4": DefaultCodexUpstreamModel, + "gpt-5.5": DefaultCodexUpstreamModel, + }, + TruncationStep: 518, + MaxContinue: 8, + MinN: 1, + MarkerText: "Continue thinking...", + ForceIncludeEncrypted: true, + PollIntervalMS: 1500, + } +} + +func (c Config) Normalize() Config { + if strings.TrimSpace(c.StateDBPath) == "" { + c.StateDBPath = DefaultConfig().StateDBPath + } + c.FailMode = strings.ToLower(strings.TrimSpace(c.FailMode)) + if c.FailMode == "" { + c.FailMode = "fallback" + } + c.UpstreamModel = strings.TrimSpace(c.UpstreamModel) + defaultAliases := DefaultConfig().UpstreamModelAliases + clean := make(map[string]string, len(defaultAliases)+len(c.UpstreamModelAliases)) + for alias, target := range defaultAliases { + clean[alias] = target + } + for alias, target := range c.UpstreamModelAliases { + alias = strings.TrimSpace(alias) + target = strings.TrimSpace(target) + if alias == "" || target == "" { + continue + } + clean[alias] = target + } + c.UpstreamModelAliases = clean + if c.TruncationStep <= 0 { + c.TruncationStep = DefaultConfig().TruncationStep + } + if c.MaxContinue <= 0 { + c.MaxContinue = DefaultConfig().MaxContinue + } + if c.MinN <= 0 { + c.MinN = DefaultConfig().MinN + } + if strings.TrimSpace(c.MarkerText) == "" { + c.MarkerText = DefaultConfig().MarkerText + } + if c.PollIntervalMS <= 0 { + c.PollIntervalMS = DefaultConfig().PollIntervalMS + } + return c +} + +func SessionTTL() time.Duration { + return 24 * time.Hour +} diff --git a/cpa_codexcont_executor_plugin/go/internal/executor/fold.go b/cpa_codexcont_executor_plugin/go/internal/executor/fold.go new file mode 100644 index 0000000..e90b37d --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/internal/executor/fold.go @@ -0,0 +1,551 @@ +package executor + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "time" +) + +var terminalTypes = map[string]bool{ + "response.completed": true, + "response.failed": true, + "response.incomplete": true, +} + +type StreamReader interface { + Read(context.Context) ([]byte, bool, error) + Close() error +} + +type StreamOpener func(context.Context, []byte, int) (StreamReader, error) +type Emitter func(context.Context, []byte) error + +type FoldResult struct { + RequestID string + Protection string + Summary map[string]any +} + +type seqCounter struct { + n int +} + +func (s *seqCounter) Next() int { + v := s.n + s.n++ + return v +} + +type bufferEntry struct { + oi string + itemType string + events []map[string]any + item map[string]any +} + +func FoldStream(ctx context.Context, cfg Config, baseBody map[string]any, open StreamOpener, emit Emitter) (*FoldResult, error) { + cfg = cfg.Normalize() + startedAt := time.Now() + displayModel := toString(baseBody["model"]) + origInput := anyList(baseBody["input"]) + replayTail := []any{} + finalOutput := []any{} + totalUsage := map[string]any{} + var firstUsage map[string]any + roundsInfo := []map[string]any{} + seq := &seqCounter{} + dsOI := 0 + sawDone := false + continuationCount := 0 + var firstTruncRound any + var firstTruncTokens any + var firstTruncN any + var firstTruncDecision string + var latestReasoning any + var finalStatus string + var stoppedReason string + var failureReason string + var baseResponse map[string]any + var requestID string + + firstPayload := BuildFirstPayload(baseBody) + firstRaw, err := marshalPayload(firstPayload) + if err != nil { + return nil, err + } + reader, err := open(ctx, firstRaw, 1) + if err != nil { + return nil, err + } + defer reader.Close() + + roundNo := 0 + for { + roundNo++ + itemKind := map[string]string{} + oiMap := map[string]int{} + var outBuffer []bufferEntry + var roundReasoning []map[string]any + var terminal map[string]any + var usage map[string]any + parser := &SSEParser{} + + readTerminal := false + for !readTerminal { + chunk, done, readErr := reader.Read(ctx) + if readErr != nil { + failureReason = "upstream_stream_error" + return emitIncomplete(ctx, emit, baseResponse, finalOutput, agentUsage(firstUsage, totalUsage, nil, false), seq, "upstream_error", roundsInfo, totalUsage, startedAt, requestID, failureReason) + } + events := parser.Feed(chunk) + if done { + events = append(events, parser.Close()...) + } + for _, framed := range events { + if framed.Done { + sawDone = true + continue + } + ev := cloneEvent(framed.Data) + typ := toString(ev["type"]) + if typ == "response.created" || typ == "response.in_progress" { + if roundNo == 1 { + if displayModel != "" { + if resp := mapValue(ev, "response"); resp != nil { + resp["model"] = displayModel + } + } + if typ == "response.created" { + baseResponse = mapValue(ev, "response") + requestID = firstString(baseResponse["id"], requestID) + } + ev["sequence_number"] = seq.Next() + if err := emit(ctx, SerializeEvent(ev)); err != nil { + return nil, err + } + } + continue + } + if terminalTypes[typ] { + terminal = ev + usage = mapValue(mapValue(ev, "response"), "usage") + readTerminal = true + break + } + upKey := oiKey(ev["output_index"]) + if typ == "response.output_item.added" { + item := mapValue(ev, "item") + itemType := toString(item["type"]) + if itemType == "reasoning" { + itemKind[upKey] = "reasoning" + oiMap[upKey] = dsOI + ev["output_index"] = dsOI + dsOI++ + ev["sequence_number"] = seq.Next() + if err := emit(ctx, SerializeEvent(ev)); err != nil { + return nil, err + } + } else { + itemKind[upKey] = "buffered" + outBuffer = append(outBuffer, bufferEntry{ + oi: upKey, + itemType: itemType, + events: []map[string]any{ev}, + item: item, + }) + } + continue + } + switch itemKind[upKey] { + case "reasoning": + if mapped, ok := oiMap[upKey]; ok { + ev["output_index"] = mapped + } + ev["sequence_number"] = seq.Next() + if typ == "response.output_item.done" { + item := mapValue(ev, "item") + roundReasoning = append(roundReasoning, item) + finalOutput = append(finalOutput, item) + } + if err := emit(ctx, SerializeEvent(ev)); err != nil { + return nil, err + } + case "buffered": + if entry := findBuffer(outBuffer, upKey); entry >= 0 { + outBuffer[entry].events = append(outBuffer[entry].events, ev) + if typ == "response.output_item.done" { + item := mapValue(ev, "item") + if len(item) > 0 { + outBuffer[entry].item = item + } + } + } + default: + ev["sequence_number"] = seq.Next() + if err := emit(ctx, SerializeEvent(ev)); err != nil { + return nil, err + } + } + } + if done || readTerminal { + break + } + } + + _ = reader.Close() + sawTerminal := terminal != nil + sumUsage(totalUsage, usage) + if roundNo == 1 { + firstUsage = usage + } + rt, rtOK := ReasoningTokens(usage) + if rtOK { + latestReasoning = rt + } + n, nOK := TierN(rt, rtOK, cfg.TruncationStep) + roundInfo := map[string]any{"round": roundNo} + if rtOK { + roundInfo["reasoning_tokens"] = rt + } + if nOK { + roundInfo["n"] = n + } + + hasEncrypted := hasReplayableEncrypted(roundReasoning) + withinCaps := cfg.MaxTotalOutputTokens <= 0 || usageOutputTokens(totalUsage) < int64(cfg.MaxTotalOutputTokens) + doContinue := cfg.Enabled && + sawTerminal && + ShouldContinue(rt, rtOK, cfg.MinN, cfg.MaxN, cfg.TruncationStep) && + hasEncrypted && + roundNo <= cfg.MaxContinue && + withinCaps + + decision := "clean" + if doContinue { + decision = "continue" + } else if !sawTerminal { + decision = "upstream_eof" + stoppedReason = "upstream_eof" + } else if IsTruncationPattern(rt, rtOK, cfg.TruncationStep) { + switch { + case !hasEncrypted: + decision = "no_encrypted_content" + stoppedReason = "no_encrypted_content" + case roundNo > cfg.MaxContinue: + decision = "max_continue" + stoppedReason = "max_continue" + case !withinCaps: + decision = "max_total_output_tokens" + stoppedReason = "max_total_output_tokens" + default: + decision = "tier_out_of_window" + stoppedReason = "tier_out_of_window" + } + } + roundInfo["decision"] = decision + roundInfo["truncation_match"] = IsTruncationPattern(rt, rtOK, cfg.TruncationStep) + roundInfo["buffered"] = bufferedTypes(outBuffer) + roundsInfo = append(roundsInfo, roundInfo) + if IsTruncationPattern(rt, rtOK, cfg.TruncationStep) && firstTruncRound == nil { + firstTruncRound = roundNo + firstTruncTokens = rt + if nOK { + firstTruncN = n + } + firstTruncDecision = decision + } + + if doContinue { + continuationCount++ + lastReasoning := roundReasoning[len(roundReasoning)-1] + replayTail = append(replayTail, mapsToAny(roundReasoning)...) + replayTail = append(replayTail, CommentaryMessage(cfg.MarkerText)) + nextInput := append([]any{}, origInput...) + nextInput = append(nextInput, replayTail...) + nextPayload := BuildRoundPayload(baseBody, nextInput, cfg, true) + nextRaw, err := marshalPayload(nextPayload) + if err != nil { + return nil, err + } + _ = lastReasoning + reader, err = open(ctx, nextRaw, roundNo+1) + if err != nil { + failureReason = "continuation_upstream_error" + return emitIncomplete(ctx, emit, baseResponse, finalOutput, agentUsage(firstUsage, totalUsage, usage, false), seq, "upstream_error", roundsInfo, totalUsage, startedAt, requestID, failureReason) + } + continue + } + + if !sawTerminal { + finalStatus = "incomplete" + return emitIncomplete(ctx, emit, baseResponse, finalOutput, agentUsage(firstUsage, totalUsage, usage, false), seq, "upstream_eof", roundsInfo, totalUsage, startedAt, requestID, "") + } + + for _, entry := range outBuffer { + for _, ev := range entry.events { + if _, ok := ev["output_index"]; ok { + ev["output_index"] = dsOI + } + ev["sequence_number"] = seq.Next() + if err := emit(ctx, SerializeEvent(ev)); err != nil { + return nil, err + } + } + dsOI++ + if len(entry.item) > 0 { + finalOutput = append(finalOutput, entry.item) + } + } + resp := mapValue(terminal, "response") + finalStatus = firstString(resp["status"], "completed") + term := reconstructTerminal(terminal, baseResponse, finalOutput, agentUsage(firstUsage, totalUsage, usage, true), seq.Next(), roundsInfo, stoppedReason, totalUsage) + if err := emit(ctx, SerializeEvent(term)); err != nil { + return nil, err + } + if sawDone { + if err := emit(ctx, SerializeDone()); err != nil { + return nil, err + } + } + protection := protectionValue(continuationCount, stoppedReason, failureReason, finalStatus) + summary := summaryMap(requestID, baseBody, startedAt, protection, finalStatus, stoppedReason, failureReason, roundsInfo, latestReasoning, continuationCount, firstTruncRound, firstTruncTokens, firstTruncN, firstTruncDecision) + return &FoldResult{RequestID: requestID, Protection: protection, Summary: summary}, nil + } +} + +func emitIncomplete(ctx context.Context, emit Emitter, baseResponse map[string]any, finalOutput []any, usage map[string]any, seq *seqCounter, reason string, rounds []map[string]any, totalUsage map[string]any, startedAt time.Time, requestID, failureReason string) (*FoldResult, error) { + ev := syntheticIncomplete(baseResponse, finalOutput, usage, seq.Next(), reason, rounds, totalUsage) + if err := emit(ctx, SerializeEvent(ev)); err != nil { + return nil, err + } + protection := protectionValue(0, reason, failureReason, "incomplete") + summary := summaryMap(requestID, nil, startedAt, protection, "incomplete", reason, failureReason, rounds, latestFromRounds(rounds), 0, nil, nil, nil, "") + return &FoldResult{RequestID: requestID, Protection: protection, Summary: summary}, nil +} + +func reconstructTerminal(terminal, baseResponse map[string]any, output []any, usage map[string]any, seq int, rounds []map[string]any, stoppedReason string, billedUsage map[string]any) map[string]any { + resp := cloneMap(baseResponse) + if len(resp) == 0 { + resp = cloneMap(mapValue(terminal, "response")) + } + resp["output"] = output + resp["usage"] = usage + tresp := mapValue(terminal, "response") + resp["status"] = firstString(tresp["status"], "completed") + if details, ok := tresp["incomplete_details"]; ok { + resp["incomplete_details"] = details + } + withProxyMetadata(resp, rounds, stoppedReason, billedUsage) + typ := firstString(terminal["type"], "response.completed") + return map[string]any{"type": typ, "response": resp, "sequence_number": seq} +} + +func syntheticIncomplete(baseResponse map[string]any, output []any, usage map[string]any, seq int, reason string, rounds []map[string]any, billedUsage map[string]any) map[string]any { + resp := cloneMap(baseResponse) + resp["output"] = output + resp["usage"] = usage + resp["status"] = "incomplete" + resp["incomplete_details"] = map[string]any{"reason": reason} + withProxyMetadata(resp, rounds, reason, billedUsage) + return map[string]any{"type": "response.incomplete", "response": resp, "sequence_number": seq} +} + +func withProxyMetadata(resp map[string]any, rounds []map[string]any, stoppedReason string, billedUsage map[string]any) { + md := mapValue(resp, "metadata") + if md == nil { + md = map[string]any{} + } + md["proxy_rounds"] = rounds + if len(billedUsage) > 0 { + md["proxy_billed_usage"] = billedUsage + } + if stoppedReason != "" { + md["proxy_stopped_reason"] = stoppedReason + } + resp["metadata"] = md +} + +func agentUsage(first, total, finalRound map[string]any, flushedFinal bool) map[string]any { + inTok, _ := intValue(first["input_tokens"]) + cached, cachedOK := intValue(mapValue(first, "input_tokens_details")["cached_tokens"]) + reasoning, _ := intValue(mapValue(total, "output_tokens_details")["reasoning_tokens"]) + finalNonReason := int64(0) + if flushedFinal && finalRound != nil { + fo, _ := intValue(finalRound["output_tokens"]) + fr, _ := intValue(mapValue(finalRound, "output_tokens_details")["reasoning_tokens"]) + if fo > fr { + finalNonReason = fo - fr + } + } + outTok := reasoning + finalNonReason + usage := map[string]any{ + "input_tokens": inTok, + "output_tokens": outTok, + "total_tokens": inTok + outTok, + "output_tokens_details": map[string]any{"reasoning_tokens": reasoning}, + } + if cachedOK { + usage["input_tokens_details"] = map[string]any{"cached_tokens": cached} + } + return usage +} + +func sumUsage(acc map[string]any, usage map[string]any) { + if usage == nil { + return + } + for _, key := range []string{"input_tokens", "output_tokens", "total_tokens"} { + if value, ok := intValue(usage[key]); ok { + current, _ := intValue(acc[key]) + acc[key] = current + value + } + } + if cached, ok := intValue(mapValue(usage, "input_tokens_details")["cached_tokens"]); ok { + details := mapValue(acc, "input_tokens_details") + if details == nil { + details = map[string]any{} + } + current, _ := intValue(details["cached_tokens"]) + details["cached_tokens"] = current + cached + acc["input_tokens_details"] = details + } + if reasoning, ok := intValue(mapValue(usage, "output_tokens_details")["reasoning_tokens"]); ok { + details := mapValue(acc, "output_tokens_details") + if details == nil { + details = map[string]any{} + } + current, _ := intValue(details["reasoning_tokens"]) + details["reasoning_tokens"] = current + reasoning + acc["output_tokens_details"] = details + } +} + +func summaryMap(requestID string, baseBody map[string]any, startedAt time.Time, protection, finalStatus, stoppedReason, failureReason string, rounds []map[string]any, latestReasoning any, continuationCount int, firstRound, firstTokens, firstN any, firstDecision string) map[string]any { + if requestID == "" { + requestID = fmt.Sprintf("codexcont-%d", startedAt.UnixNano()) + } + endedAt := time.Now() + model := "" + if baseBody != nil { + model = firstString(baseBody["model"], "") + } + out := map[string]any{ + "request_id": requestID, + "model": model, + "started_at": startedAt.UTC().Format(time.RFC3339Nano), + "updated_at": endedAt.UTC().Format(time.RFC3339Nano), + "ended_at": endedAt.UTC().Format(time.RFC3339Nano), + "duration_ms": endedAt.Sub(startedAt).Milliseconds(), + "status": finalStatus, + "final_status": finalStatus, + "protection": protection, + "rounds": rounds, + "latest_round": len(rounds), + "continuation_count": continuationCount, + "stopped_reason": stoppedReason, + "failure_reason": failureReason, + } + if latestReasoning != nil { + out["latest_reasoning_tokens"] = latestReasoning + } + if firstRound != nil { + out["first_truncation_round"] = firstRound + out["first_truncation_reasoning_tokens"] = firstTokens + out["first_truncation_n"] = firstN + out["first_truncation_decision"] = firstDecision + } + return out +} + +func protectionValue(continuations int, stoppedReason, failureReason, finalStatus string) string { + if failureReason != "" { + return "failed" + } + if finalStatus == "incomplete" && stoppedReason != "" && stoppedReason != "max_continue" && stoppedReason != "no_encrypted_content" { + return "incomplete" + } + if stoppedReason == "no_encrypted_content" || stoppedReason == "max_continue" || stoppedReason == "max_total_output_tokens" || stoppedReason == "tier_out_of_window" { + if continuations > 0 { + return "auto_continued" + } + return "risk_uncontinued" + } + if continuations > 0 { + return "auto_continued" + } + return "protected_clean" +} + +func findBuffer(items []bufferEntry, key string) int { + for i := range items { + if items[i].oi == key { + return i + } + } + return -1 +} + +func bufferedTypes(items []bufferEntry) []string { + out := make([]string, 0, len(items)) + for _, item := range items { + if item.itemType == "" { + out = append(out, "unknown") + continue + } + out = append(out, item.itemType) + } + return out +} + +func hasReplayableEncrypted(items []map[string]any) bool { + if len(items) == 0 { + return false + } + return firstString(items[len(items)-1]["encrypted_content"], "") != "" +} + +func usageOutputTokens(usage map[string]any) int64 { + value, _ := intValue(usage["output_tokens"]) + return value +} + +func mapsToAny(items []map[string]any) []any { + out := make([]any, 0, len(items)) + for _, item := range items { + out = append(out, item) + } + return out +} + +func latestFromRounds(rounds []map[string]any) any { + for i := len(rounds) - 1; i >= 0; i-- { + if value, ok := rounds[i]["reasoning_tokens"]; ok { + return value + } + } + return nil +} + +func oiKey(raw any) string { + return fmt.Sprint(raw) +} + +func firstString(raw any, fallback string) string { + if text, ok := raw.(string); ok && text != "" { + return text + } + return fallback +} + +func marshalPayload(payload map[string]any) ([]byte, error) { + if payload == nil { + return nil, errors.New("payload is required") + } + return jsonMarshal(payload) +} + +var jsonMarshal = func(v any) ([]byte, error) { + return json.Marshal(v) +} diff --git a/cpa_codexcont_executor_plugin/go/internal/executor/fold_test.go b/cpa_codexcont_executor_plugin/go/internal/executor/fold_test.go new file mode 100644 index 0000000..c36e580 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/internal/executor/fold_test.go @@ -0,0 +1,364 @@ +package executor + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "strings" + "testing" +) + +type fakeRoundReader struct { + chunks [][]byte + idx int + err error +} + +func (r *fakeRoundReader) Read(context.Context) ([]byte, bool, error) { + if r.err != nil { + return nil, false, r.err + } + if r.idx >= len(r.chunks) { + return nil, true, nil + } + chunk := r.chunks[r.idx] + r.idx++ + return chunk, r.idx >= len(r.chunks), nil +} + +func (r *fakeRoundReader) Close() error { return nil } + +func round(events ...map[string]any) [][]byte { + var out [][]byte + for _, ev := range events { + out = append(out, SerializeEvent(ev)) + } + return out +} + +func created(id string) map[string]any { + return map[string]any{"type": "response.created", "response": map[string]any{"id": id, "status": "in_progress"}} +} + +func reasoning(id string, encrypted bool) []map[string]any { + item := map[string]any{"id": id, "type": "reasoning", "content": []any{}} + if encrypted { + item["encrypted_content"] = "enc-" + id + } + return []map[string]any{ + {"type": "response.output_item.added", "output_index": 0, "item": item}, + {"type": "response.output_item.done", "output_index": 0, "item": item}, + } +} + +func message(index int, text string) []map[string]any { + item := map[string]any{"id": "msg-" + text, "type": "message", "role": "assistant"} + return []map[string]any{ + {"type": "response.output_item.added", "output_index": index, "item": item}, + {"type": "response.output_text.delta", "output_index": index, "item_id": item["id"], "content_index": 0, "delta": text}, + {"type": "response.output_item.done", "output_index": index, "item": item}, + } +} + +func completed(reasoningTokens, outputTokens int) map[string]any { + return map[string]any{ + "type": "response.completed", + "response": map[string]any{ + "id": "resp-terminal", + "status": "completed", + "output": []any{}, + "input_tokens": 0, + "usage": map[string]any{ + "input_tokens": 100, + "output_tokens": outputTokens, + "total_tokens": 100 + outputTokens, + "input_tokens_details": map[string]any{ + "cached_tokens": 20, + }, + "output_tokens_details": map[string]any{ + "reasoning_tokens": reasoningTokens, + }, + }, + }, + } +} + +func parseEvents(t *testing.T, raw []byte) []map[string]any { + t.Helper() + parser := &SSEParser{} + var out []map[string]any + for _, ev := range parser.Feed(raw) { + if ev.Data != nil { + out = append(out, ev.Data) + } + } + for _, ev := range parser.Close() { + if ev.Data != nil { + out = append(out, ev.Data) + } + } + return out +} + +func terminalEvent(t *testing.T, events []map[string]any) map[string]any { + t.Helper() + for _, ev := range events { + if terminalTypes[toString(ev["type"])] { + return ev + } + } + t.Fatal("terminal event not found") + return nil +} + +func TestFoldStreamAutoContinuesAndReconstructsTerminal(t *testing.T) { + cfg := DefaultConfig() + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": []any{map[string]any{"role": "user", "content": "hi"}}} + first := []map[string]any{created("resp-a")} + first = append(first, reasoning("rs-a", true)...) + first = append(first, message(1, "BAD")...) + first = append(first, completed(516, 536)) + second := []map[string]any{created("resp-b")} + second = append(second, reasoning("rs-b", true)...) + second = append(second, message(1, "GOOD")...) + second = append(second, completed(120, 150)) + rounds := map[int][][]byte{1: round(first...), 2: round(second...)} + var opened [][]byte + var emitted bytes.Buffer + result, err := FoldStream(context.Background(), cfg, base, func(_ context.Context, body []byte, roundNo int) (StreamReader, error) { + opened = append(opened, body) + return &fakeRoundReader{chunks: rounds[roundNo]}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if result.Protection != "auto_continued" || result.Summary["continuation_count"] != 1 { + t.Fatalf("summary = %#v", result.Summary) + } + if len(opened) != 2 { + t.Fatalf("opened rounds = %d", len(opened)) + } + if strings.Contains(emitted.String(), "BAD") || !strings.Contains(emitted.String(), "GOOD") { + t.Fatalf("folded stream leaked/truncated output incorrectly:\n%s", emitted.String()) + } + var secondBody map[string]any + if err := json.Unmarshal(opened[1], &secondBody); err != nil { + t.Fatal(err) + } + input := secondBody["input"].([]any) + if len(input) < 3 { + t.Fatalf("continuation input too short: %#v", input) + } + if !strings.Contains(string(opened[1]), EncryptedInclude) || !strings.Contains(string(opened[1]), cfg.MarkerText) { + t.Fatalf("continuation payload missing encrypted include or marker: %s", string(opened[1])) + } + events := parseEvents(t, emitted.Bytes()) + term := terminalEvent(t, events) + resp := mapValue(term, "response") + md := mapValue(resp, "metadata") + if got := len(md["proxy_rounds"].([]any)); got != 2 { + t.Fatalf("proxy_rounds len = %d metadata=%#v", got, md) + } + usage := mapValue(resp, "usage") + if usage["output_tokens"] != float64(666) { + t.Fatalf("agent output usage = %#v", usage) + } + assertSequenceMonotonic(t, events) +} + +func TestFoldStreamFirstRoundPreservesStringInput(t *testing.T) { + cfg := DefaultConfig() + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": "say hi"} + events := []map[string]any{created("resp-a"), completed(42, 60)} + var opened [][]byte + var emitted bytes.Buffer + _, err := FoldStream(context.Background(), cfg, base, func(_ context.Context, body []byte, _ int) (StreamReader, error) { + opened = append(opened, body) + return &fakeRoundReader{chunks: round(events...)}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if len(opened) != 1 { + t.Fatalf("opened rounds = %d", len(opened)) + } + var firstBody map[string]any + if err := json.Unmarshal(opened[0], &firstBody); err != nil { + t.Fatal(err) + } + if firstBody["input"] != "say hi" { + t.Fatalf("first round input was rewritten: %#v", firstBody["input"]) + } + if !strings.Contains(emitted.String(), "response.completed") { + t.Fatalf("terminal missing:\n%s", emitted.String()) + } +} + +func TestFoldStreamAcceptsLineChunkedSSE(t *testing.T) { + cfg := DefaultConfig() + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": []any{}} + events := []map[string]any{created("resp-a"), completed(42, 60)} + var chunks [][]byte + for _, ev := range events { + raw := SerializeEvent(ev) + for _, line := range strings.Split(strings.TrimSuffix(string(raw), "\n"), "\n") { + chunks = append(chunks, []byte(line)) + } + } + var emitted bytes.Buffer + result, err := FoldStream(context.Background(), cfg, base, func(context.Context, []byte, int) (StreamReader, error) { + return &fakeRoundReader{chunks: chunks}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if result.Protection != "protected_clean" { + t.Fatalf("protection=%s summary=%#v stream=%s", result.Protection, result.Summary, emitted.String()) + } + term := terminalEvent(t, parseEvents(t, emitted.Bytes())) + if term["type"] != "response.completed" { + t.Fatalf("terminal = %#v", term) + } +} + +func TestFoldStreamMissingEncryptedDoesNotContinue(t *testing.T) { + cfg := DefaultConfig() + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": []any{}} + events := []map[string]any{created("resp-a")} + events = append(events, reasoning("rs-a", false)...) + events = append(events, message(1, "VISIBLE")...) + events = append(events, completed(516, 536)) + var opened int + var emitted bytes.Buffer + result, err := FoldStream(context.Background(), cfg, base, func(context.Context, []byte, int) (StreamReader, error) { + opened++ + return &fakeRoundReader{chunks: round(events...)}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if opened != 1 || result.Protection != "risk_uncontinued" { + t.Fatalf("opened=%d summary=%#v", opened, result.Summary) + } + term := terminalEvent(t, parseEvents(t, emitted.Bytes())) + md := mapValue(mapValue(term, "response"), "metadata") + if md["proxy_stopped_reason"] != "no_encrypted_content" || !strings.Contains(emitted.String(), "VISIBLE") { + t.Fatalf("metadata/output = %#v\n%s", md, emitted.String()) + } +} + +func TestFoldStreamEOFEmitsIncompleteAndDropsBufferedOutput(t *testing.T) { + cfg := DefaultConfig() + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": []any{}} + events := []map[string]any{created("resp-a")} + events = append(events, message(0, "HALF")...) + var emitted bytes.Buffer + result, err := FoldStream(context.Background(), cfg, base, func(context.Context, []byte, int) (StreamReader, error) { + return &fakeRoundReader{chunks: round(events...)}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if result.Protection != "incomplete" || strings.Contains(emitted.String(), "HALF") { + t.Fatalf("result=%#v stream=%s", result.Summary, emitted.String()) + } + term := terminalEvent(t, parseEvents(t, emitted.Bytes())) + if term["type"] != "response.incomplete" { + t.Fatalf("terminal = %#v", term) + } +} + +func TestFoldStreamMaxContinueStopsWithMetadata(t *testing.T) { + cfg := DefaultConfig() + cfg.MaxContinue = 1 + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": []any{}} + r1 := []map[string]any{created("resp-a")} + r1 = append(r1, reasoning("rs-a", true)...) + r1 = append(r1, completed(516, 516)) + r2 := []map[string]any{created("resp-b")} + r2 = append(r2, reasoning("rs-b", true)...) + r2 = append(r2, completed(1034, 1034)) + rounds := map[int][][]byte{1: round(r1...), 2: round(r2...)} + var opened int + var emitted bytes.Buffer + _, err := FoldStream(context.Background(), cfg, base, func(_ context.Context, _ []byte, roundNo int) (StreamReader, error) { + opened++ + return &fakeRoundReader{chunks: rounds[roundNo]}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if opened != 2 { + t.Fatalf("opened = %d", opened) + } + term := terminalEvent(t, parseEvents(t, emitted.Bytes())) + md := mapValue(mapValue(term, "response"), "metadata") + if md["proxy_stopped_reason"] != "max_continue" { + t.Fatalf("metadata = %#v", md) + } +} + +func TestFoldStreamContinuationOpenErrorEmitsIncomplete(t *testing.T) { + cfg := DefaultConfig() + base := map[string]any{"model": "gpt-5.5", "stream": true, "input": []any{}} + r1 := []map[string]any{created("resp-a")} + r1 = append(r1, reasoning("rs-a", true)...) + r1 = append(r1, completed(516, 516)) + var opened int + var emitted bytes.Buffer + result, err := FoldStream(context.Background(), cfg, base, func(_ context.Context, _ []byte, roundNo int) (StreamReader, error) { + opened++ + if roundNo == 2 { + return nil, errors.New("upstream unavailable") + } + return &fakeRoundReader{chunks: round(r1...)}, nil + }, func(_ context.Context, payload []byte) error { + _, _ = emitted.Write(payload) + return nil + }) + if err != nil { + t.Fatal(err) + } + if opened != 2 || result.Protection != "failed" { + t.Fatalf("opened=%d summary=%#v", opened, result.Summary) + } + term := terminalEvent(t, parseEvents(t, emitted.Bytes())) + resp := mapValue(term, "response") + if term["type"] != "response.incomplete" || mapValue(resp, "incomplete_details")["reason"] != "upstream_error" { + t.Fatalf("terminal = %#v", term) + } +} + +func assertSequenceMonotonic(t *testing.T, events []map[string]any) { + t.Helper() + prev := -1 + for _, ev := range events { + raw, ok := intValue(ev["sequence_number"]) + if !ok { + continue + } + if int(raw) <= prev { + t.Fatalf("sequence not monotonic after %d: %#v", prev, ev) + } + prev = int(raw) + } +} diff --git a/cpa_codexcont_executor_plugin/go/internal/executor/sse.go b/cpa_codexcont_executor_plugin/go/internal/executor/sse.go new file mode 100644 index 0000000..8559033 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/internal/executor/sse.go @@ -0,0 +1,117 @@ +package executor + +import ( + "bytes" + "encoding/json" + "strings" +) + +type SSEEvent struct { + Done bool + Data map[string]any +} + +type SSEParser struct { + buffer []byte + dataLines []string +} + +func (p *SSEParser) Feed(chunk []byte) []SSEEvent { + if len(chunk) == 0 { + return nil + } + if len(p.buffer) == 0 && !bytes.ContainsAny(chunk, "\r\n") { + line := strings.TrimSuffix(string(chunk), "\r") + if isSSELineChunk(line) { + return p.processLine(line) + } + } + p.buffer = append(p.buffer, chunk...) + var out []SSEEvent + for { + idx := bytes.IndexByte(p.buffer, '\n') + if idx < 0 { + break + } + raw := p.buffer[:idx] + p.buffer = p.buffer[idx+1:] + line := strings.TrimSuffix(string(raw), "\r") + out = append(out, p.processLine(line)...) + } + return out +} + +func (p *SSEParser) Close() []SSEEvent { + var out []SSEEvent + if len(p.buffer) > 0 { + line := strings.TrimSuffix(string(p.buffer), "\r") + p.buffer = nil + out = append(out, p.processLine(line)...) + } + if ev, ok := p.flush(); ok { + out = append(out, ev) + } + return out +} + +func (p *SSEParser) processLine(line string) []SSEEvent { + if line == "" { + if ev, ok := p.flush(); ok { + return []SSEEvent{ev} + } + return nil + } + if strings.HasPrefix(line, ":") { + return nil + } + if strings.HasPrefix(line, "event:") { + if len(p.dataLines) > 0 { + if ev, ok := p.flush(); ok { + return []SSEEvent{ev} + } + } + return nil + } + if strings.HasPrefix(line, "data:") { + value := line[5:] + value = strings.TrimPrefix(value, " ") + p.dataLines = append(p.dataLines, value) + } + return nil +} + +func isSSELineChunk(line string) bool { + return line == "" || + strings.HasPrefix(line, ":") || + strings.HasPrefix(line, "event:") || + strings.HasPrefix(line, "data:") +} + +func (p *SSEParser) flush() (SSEEvent, bool) { + if len(p.dataLines) == 0 { + return SSEEvent{}, false + } + payload := strings.Join(p.dataLines, "\n") + p.dataLines = nil + if payload == "[DONE]" { + return SSEEvent{Done: true}, true + } + var data map[string]any + if err := json.Unmarshal([]byte(payload), &data); err != nil { + return SSEEvent{}, false + } + return SSEEvent{Data: data}, true +} + +func SerializeEvent(event map[string]any) []byte { + typ := toString(event["type"]) + if typ == "" { + typ = "message" + } + raw, _ := json.Marshal(event) + return []byte("event: " + typ + "\ndata: " + string(raw) + "\n\n") +} + +func SerializeDone() []byte { + return []byte("data: [DONE]\n\n") +} diff --git a/cpa_codexcont_executor_plugin/go/internal/executor/store.go b/cpa_codexcont_executor_plugin/go/internal/executor/store.go new file mode 100644 index 0000000..831a5ef --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/internal/executor/store.go @@ -0,0 +1,134 @@ +package executor + +import ( + "context" + "database/sql" + "encoding/json" + "os" + "path/filepath" + "strings" + "time" + + _ "modernc.org/sqlite" +) + +type Store struct { + db *sql.DB +} + +type CodexSummary struct { + RequestID string `json:"request_id"` + KeyID string `json:"key_id,omitempty"` + Model string `json:"model,omitempty"` + Protection string `json:"protection,omitempty"` + Summary map[string]any `json:"summary"` + UpdatedAt time.Time `json:"updated_at"` +} + +func OpenStore(path string) (*Store, error) { + if strings.TrimSpace(path) == "" { + path = DefaultConfig().StateDBPath + } + dir := filepath.Dir(path) + if dir != "." { + if err := os.MkdirAll(dir, 0o755); err != nil { + return nil, err + } + } + db, err := sql.Open("sqlite", path) + if err != nil { + return nil, err + } + store := &Store{db: db} + if err := store.EnsureSchema(context.Background()); err != nil { + _ = db.Close() + return nil, err + } + return store, nil +} + +func (s *Store) Close() error { + if s == nil || s.db == nil { + return nil + } + return s.db.Close() +} + +func (s *Store) EnsureSchema(ctx context.Context) error { + stmts := []string{ + `pragma journal_mode=wal`, + `create table if not exists codexcont_summaries ( + request_id text primary key, + key_id text, + model text, + protection text, + summary_json text, + updated_at integer not null + )`, + `create table if not exists audit_log ( + id integer primary key autoincrement, + timestamp integer not null, + actor text, + action text not null, + target text, + detail_json text + )`, + } + for _, stmt := range stmts { + if _, err := s.db.ExecContext(ctx, stmt); err != nil { + return err + } + } + return nil +} + +func (s *Store) SaveCodexSummary(ctx context.Context, requestID, keyID, model, protection string, summary any) error { + if s == nil || s.db == nil { + return nil + } + raw, _ := json.Marshal(summary) + _, err := s.db.ExecContext(ctx, `insert into codexcont_summaries(request_id, key_id, model, protection, summary_json, updated_at) + values(?, ?, ?, ?, ?, ?) + on conflict(request_id) do update set key_id=excluded.key_id, model=excluded.model, + protection=excluded.protection, summary_json=excluded.summary_json, updated_at=excluded.updated_at`, + requestID, keyID, model, protection, string(raw), time.Now().Unix()) + return err +} + +func (s *Store) RecentCodexSummaries(ctx context.Context, keyID string, limit int) ([]CodexSummary, error) { + if s == nil || s.db == nil { + return nil, nil + } + if limit <= 0 || limit > 200 { + limit = 100 + } + query := `select request_id, key_id, model, protection, summary_json, updated_at from codexcont_summaries` + args := []any{} + if strings.TrimSpace(keyID) != "" && keyID != "all" { + query += ` where key_id = ?` + args = append(args, keyID) + } + query += ` order by updated_at desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []CodexSummary + for rows.Next() { + var item CodexSummary + var raw string + var ts int64 + if err := rows.Scan(&item.RequestID, &item.KeyID, &item.Model, &item.Protection, &raw, &ts); err != nil { + return nil, err + } + item.UpdatedAt = time.Unix(ts, 0) + _ = json.Unmarshal([]byte(raw), &item.Summary) + if item.Summary == nil { + item.Summary = map[string]any{} + } + out = append(out, item) + } + return out, rows.Err() +} diff --git a/cpa_codexcont_executor_plugin/go/main.go b/cpa_codexcont_executor_plugin/go/main.go new file mode 100644 index 0000000..e4eb593 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/main.go @@ -0,0 +1,1055 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "net/http" + "os" + "sort" + "strconv" + "strings" + "sync" + "time" + + "codexcont/cpa-codexcont-executor-plugin/internal/executor" + _ "embed" + "gopkg.in/yaml.v3" +) + +const pluginID = "cpa-codexcont-executor" +const executorModelScopeBoth = "both" +const executorFormatOpenAIResponse = "openai-response" + +//go:embed assets/admin.html +var adminHTMLTemplate string + +type envelope struct { + OK bool `json:"ok"` + Result json.RawMessage `json:"result,omitempty"` + Error *envelopeError `json:"error,omitempty"` +} + +type envelopeError struct { + Code string `json:"code"` + Message string `json:"message"` +} + +type lifecycleRequest struct { + ConfigYAML []byte `json:"config_yaml"` +} + +type registration struct { + SchemaVersion uint32 `json:"schema_version"` + Metadata metadata `json:"metadata"` + Capabilities capabilities `json:"capabilities"` +} + +type metadata struct { + Name string `json:"Name"` + Version string `json:"Version"` + Author string `json:"Author"` + GitHubRepository string `json:"GitHubRepository"` + ConfigFields []configField `json:"ConfigFields"` +} + +type capabilities struct { + FrontendAuthProvider bool `json:"frontend_auth_provider"` + FrontendAuthProviderExclusive bool `json:"frontend_auth_provider_exclusive"` + ModelRouter bool `json:"model_router"` + Executor bool `json:"executor"` + ExecutorModelScope string `json:"executor_model_scope"` + ExecutorInputFormats []string `json:"executor_input_formats"` + ExecutorOutputFormats []string `json:"executor_output_formats"` + UsagePlugin bool `json:"usage_plugin"` + ManagementAPI bool `json:"management_api"` +} + +type runtimeState struct { + mu sync.RWMutex + cfg executor.Config + store *executor.Store +} + +var state = runtimeState{cfg: executor.DefaultConfig()} +var monitor = newSummaryMonitor(160) + +var hostCall = func(method string, payload any) (json.RawMessage, error) { + return nil, fmt.Errorf("host callback unavailable for %s", method) +} + +func main() { + if os.Getenv("CPA_CODEXCONT_EXECUTOR_NOOP") != "" { + return + } +} + +func handleMethod(method string, request []byte) ([]byte, error) { + switch method { + case methodPluginRegister: + return pluginRegister(request) + case methodPluginReconfigure: + return pluginReconfigure(request) + case methodFrontendAuthIdentifier: + return okEnvelope(map[string]string{"identifier": pluginID}) + case methodFrontendAuthAuthenticate: + return frontendAuth(request) + case methodModelRoute: + return routeModel(request) + case methodExecutorIdentifier: + return okEnvelope(map[string]any{"identifier": pluginID, "type": "codexcont_executor"}) + case methodExecutorExecute: + return executorExecute(request) + case methodExecutorExecuteStream: + return executorExecuteStream(request) + case methodExecutorCountTokens: + return okEnvelope(executorResponse{Payload: []byte(`{"input_tokens":0}`)}) + case methodUsageHandle: + return usageHandle(request) + case methodManagementRegister: + return managementRegister() + case methodManagementHandle: + return managementHandle(request) + default: + return errorEnvelope("unknown_method", method), nil + } +} + +func pluginRegister(raw []byte) ([]byte, error) { + if len(raw) > 0 { + if err := applyLifecycleConfig(raw); err != nil { + return errorEnvelope("config_error", err.Error()), nil + } + } + return okEnvelope(pluginRegistration()) +} + +func pluginReconfigure(raw []byte) ([]byte, error) { + if err := applyLifecycleConfig(raw); err != nil { + return errorEnvelope("config_error", err.Error()), nil + } + return okEnvelope(pluginRegistration()) +} + +func pluginRegistration() registration { + return registration{ + SchemaVersion: schemaVersion, + Metadata: metadata{ + Name: pluginID, + Version: "0.1.0", + Author: "konbakuyomu/CodexCont", + GitHubRepository: "https://local/CodexCont", + ConfigFields: []configField{ + {Name: "enabled", Type: configBoolean, Description: "Enable the plugin."}, + {Name: "route_enabled", Type: configBoolean, Description: "Route streaming Responses requests to the CodexCont executor."}, + {Name: "state_db_path", Type: configString, Description: "SQLite path for safe executor summaries."}, + {Name: "fail_mode", Type: configEnum, EnumValues: []string{"fallback", "fail_closed"}, Description: "Fallback behavior when the executor is disabled."}, + {Name: "upstream_model", Type: configString, Description: "Optional provider-registered model used for internal upstream rounds."}, + {Name: "upstream_model_aliases", Type: configString, Description: "Optional YAML map from client-visible model aliases to internal upstream models."}, + {Name: "truncation_step", Type: configInteger, Description: "Reasoning truncation step. Default 518."}, + {Name: "max_continue", Type: configInteger, Description: "Maximum hidden continuation rounds."}, + {Name: "marker_text", Type: configString, Description: "Hidden commentary continuation marker."}, + }, + }, + Capabilities: capabilities{ + FrontendAuthProvider: true, + ModelRouter: true, + Executor: true, + ExecutorModelScope: executorModelScopeBoth, + ExecutorInputFormats: []string{executorFormatOpenAIResponse}, + ExecutorOutputFormats: []string{executorFormatOpenAIResponse}, + UsagePlugin: true, + ManagementAPI: true, + }, + } +} + +func frontendAuth(_ []byte) ([]byte, error) { + return okEnvelope(frontendAuthResponse{Authenticated: false, Metadata: map[string]string{"mode": "observability_only"}}) +} + +func applyLifecycleConfig(raw []byte) error { + var req lifecycleRequest + if err := json.Unmarshal(raw, &req); err != nil && len(raw) > 0 { + return err + } + cfg := executor.DefaultConfig() + if len(req.ConfigYAML) > 0 { + if err := yaml.Unmarshal(req.ConfigYAML, &cfg); err != nil { + return err + } + } + cfg = cfg.Normalize() + store, err := executor.OpenStore(cfg.StateDBPath) + if err != nil { + return err + } + state.mu.Lock() + old := state.store + state.cfg = cfg + state.store = store + state.mu.Unlock() + if old != nil { + _ = old.Close() + } + return nil +} + +func loadedConfig() executor.Config { + state.mu.RLock() + defer state.mu.RUnlock() + return state.cfg.Normalize() +} + +func loadedStore() *executor.Store { + state.mu.RLock() + defer state.mu.RUnlock() + return state.store +} + +func shutdownPlugin() { + state.mu.Lock() + defer state.mu.Unlock() + if state.store != nil { + _ = state.store.Close() + state.store = nil + } +} + +func routeModel(raw []byte) ([]byte, error) { + var req modelRouteRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + cfg := loadedConfig() + if !cfg.Enabled || !cfg.RouteEnabled || !req.Stream || !isResponsesRequest(req.SourceFormat, req.Body) { + return okEnvelope(modelRouteResponse{Handled: false}) + } + return okEnvelope(modelRouteResponse{ + Handled: true, + TargetKind: routeTargetSelf, + Reason: "cpa_codexcont_executor_enabled", + }) +} + +func isResponsesRequest(source string, body []byte) bool { + source = strings.ToLower(strings.TrimSpace(source)) + if strings.Contains(source, "response") || source == "responses" { + return true + } + text := string(body) + return strings.Contains(text, `"model"`) && + (strings.Contains(text, `"input"`) || + strings.Contains(text, `"previous_response_id"`) || + strings.Contains(text, `"reasoning.encrypted_content"`)) +} + +func executorUnavailable() ([]byte, error) { + cfg := loadedConfig() + if cfg.RouteEnabled && cfg.FailMode == "fail_closed" { + return errorEnvelope("codexcont_executor_disabled", "CodexCont executor route is disabled"), nil + } + return errorEnvelope("codexcont_executor_disabled", "CodexCont executor is not handling this request"), nil +} + +func executorExecute(raw []byte) ([]byte, error) { + var req executorCallRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + execReq := normalizedExecutorRequest(req) + cfg := loadedConfig() + if !cfg.Enabled || !cfg.RouteEnabled { + return executorUnavailable() + } + payload := firstBytes(execReq.Payload, execReq.OriginalRequest) + upstreamModel := resolveUpstreamModel(cfg, execReq, payload) + upstreamPayload, _ := rewriteUpstreamPayload(payload, upstreamModel) + result, err := hostCall(methodHostModelExecute, hostModelExecutionRequest{ + EntryProtocol: executorProtocol(execReq.SourceFormat), + ExitProtocol: executorProtocol(execReq.Format), + Model: upstreamModel, + Stream: false, + Body: upstreamPayload, + Headers: cloneHeader(execReq.Headers), + Query: cloneValues(execReq.Query), + Alt: execReq.Alt, + HostCallbackID: req.HostCallbackID, + }) + if err != nil { + return errorEnvelope("host_model_execute_error", err.Error()), nil + } + var resp hostModelExecutionResponse + if err := json.Unmarshal(result, &resp); err != nil { + return errorEnvelope("host_model_execute_decode_error", err.Error()), nil + } + if resp.StatusCode >= 400 { + return errorEnvelope("host_model_execute_http_error", fmt.Sprintf("upstream returned %d", resp.StatusCode)), nil + } + return okEnvelope(executorResponse{Payload: resp.Body, Headers: resp.Headers}) +} + +func executorExecuteStream(raw []byte) ([]byte, error) { + var req executorCallRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + execReq := normalizedExecutorRequest(req) + cfg := loadedConfig() + if !cfg.Enabled || !cfg.RouteEnabled { + return executorUnavailable() + } + if strings.TrimSpace(req.StreamID) == "" { + return errorEnvelope("stream_id_required", "stream_id is required for executor.execute_stream"), nil + } + payload := firstBytes(execReq.Payload, execReq.OriginalRequest) + var base map[string]any + if err := json.Unmarshal(payload, &base); err != nil { + return errorEnvelope("invalid_responses_payload", err.Error()), nil + } + go foldHostStream(req, execReq, cfg, base) + return okEnvelope(executorStreamResponse{Headers: http.Header{"Content-Type": []string{"text/event-stream"}}}) +} + +func foldHostStream(req executorCallRequest, execReq executorRequest, cfg executor.Config, base map[string]any) { + ctx := context.Background() + targetStreamID := req.StreamID + monitorID := monitor.Start(execReq, base) + diagnostics := newStreamDiagnostics() + var streamErr string + defer func() { + _, _ = hostCall(methodHostStreamClose, hostStreamCloseRequest{StreamID: targetStreamID, Error: streamErr}) + }() + opener := func(ctx context.Context, body []byte, round int) (executor.StreamReader, error) { + return openHostModelStream(ctx, req, execReq, cfg, body, round, diagnostics) + } + emitter := func(ctx context.Context, payload []byte) error { + _, err := hostCall(methodHostStreamEmit, hostStreamEmitRequest{StreamID: targetStreamID, Payload: payload}) + return err + } + result, err := executor.FoldStream(ctx, cfg, base, opener, emitter) + if err != nil { + streamErr = brief(err.Error(), 400) + monitor.Fail(monitorID, streamErr) + _, _ = hostCall(methodHostStreamEmit, hostStreamEmitRequest{StreamID: targetStreamID, Error: streamErr}) + return + } + if result != nil { + monitor.Finish(monitorID, saveFoldSummary(execReq, result, diagnostics.Snapshot())) + } +} + +type hostStreamReader struct { + streamID string + round int + diagnostics *streamDiagnostics + firstRead bool + readCount int +} + +func openHostModelStream(_ context.Context, req executorCallRequest, execReq executorRequest, cfg executor.Config, body []byte, round int, diagnostics *streamDiagnostics) (executor.StreamReader, error) { + requestedModel := firstNonEmpty(modelFromBody(body), execReq.Model) + upstreamModel := resolveUpstreamModel(cfg, execReq, body) + upstreamBody, filteredTools := rewriteUpstreamPayload(body, upstreamModel) + hostReq := hostModelExecutionRequest{ + EntryProtocol: executorProtocol(execReq.SourceFormat), + ExitProtocol: executorProtocol(execReq.Format), + Model: upstreamModel, + Stream: true, + Body: upstreamBody, + Headers: cloneHeader(execReq.Headers), + Query: cloneValues(execReq.Query), + Alt: execReq.Alt, + HostCallbackID: req.HostCallbackID, + } + if diagnostics != nil { + diagnostics.RecordOpen(round, hostReq, requestedModel, modelFromBody(upstreamBody), len(upstreamBody), filteredTools) + } + result, err := hostCall(methodHostModelExecuteStream, hostReq) + if err != nil { + if diagnostics != nil { + diagnostics.RecordOpenError(round, err) + } + return nil, err + } + var resp hostModelStreamResponse + if err := json.Unmarshal(result, &resp); err != nil { + if diagnostics != nil { + diagnostics.RecordOpenError(round, err) + } + return nil, err + } + if diagnostics != nil { + diagnostics.RecordOpenResponse(round, resp) + } + if resp.StatusCode >= 400 { + return nil, fmt.Errorf("upstream returned %d", resp.StatusCode) + } + if resp.StreamID == "" { + return nil, fmt.Errorf("host returned empty stream id") + } + return &hostStreamReader{streamID: resp.StreamID, round: round, diagnostics: diagnostics}, nil +} + +func (r *hostStreamReader) Read(context.Context) ([]byte, bool, error) { + result, err := hostCall(methodHostModelStreamRead, hostModelStreamReadRequest{StreamID: r.streamID}) + if err != nil { + r.recordRead(nil, false, err) + return nil, false, err + } + var chunk hostModelStreamReadResponse + if err := json.Unmarshal(result, &chunk); err != nil { + r.recordRead(nil, false, err) + return nil, false, err + } + if chunk.Error != "" { + err := fmt.Errorf("%s", chunk.Error) + r.recordRead(chunk.Payload, chunk.Done, err) + return chunk.Payload, chunk.Done, err + } + r.recordRead(chunk.Payload, chunk.Done, nil) + return chunk.Payload, chunk.Done, nil +} + +func (r *hostStreamReader) recordRead(payload []byte, done bool, err error) { + if r == nil { + return + } + r.readCount++ + if r.diagnostics != nil { + r.diagnostics.RecordRead(r.round, r.readCount, payload, done, err) + } + if !r.firstRead { + r.firstRead = true + } +} + +func (r *hostStreamReader) Close() error { + _, err := hostCall(methodHostModelStreamClose, hostModelStreamCloseRequest{StreamID: r.streamID}) + return err +} + +func saveFoldSummary(req executorRequest, result *executor.FoldResult, diagnostics map[string]any) map[string]any { + if result == nil { + return nil + } + summary := cloneSummary(result.Summary) + keyID := strings.TrimSpace(req.AuthID) + if keyID != "" { + summary["key_identity"] = map[string]any{"known": true, "id": keyID} + } + model := firstNonEmpty(req.Model, fmt.Sprint(summary["model"])) + if model != "" { + summary["model"] = model + } + if diagnostics != nil { + summary["diagnostics"] = diagnostics + } + requestID := firstNonEmpty(result.RequestID, fmt.Sprint(summary["request_id"])) + store := loadedStore() + if store == nil { + return summary + } + _ = store.SaveCodexSummary(context.Background(), requestID, keyID, model, result.Protection, summary) + return summary +} + +func usageHandle(_ []byte) ([]byte, error) { + return okEnvelope(map[string]any{"handled": false, "observability_only": true}) +} + +func managementRegister() ([]byte, error) { + return okEnvelope(managementRegistrationResponse{ + Routes: []managementRoute{ + {Method: http.MethodGet, Path: "/plugins/cpa-codexcont-executor/status", Description: "CodexCont executor status."}, + {Method: http.MethodGet, Path: "/plugins/cpa-codexcont-executor/summaries", Description: "Safe CodexCont executor summaries."}, + }, + Resources: []resourceRoute{ + {Path: "/admin", Menu: "CodexCont Executor", Description: "Realtime CodexCont executor monitor"}, + {Path: "/admin/api/status"}, + {Path: "/admin/api/summaries"}, + }, + }) +} + +func managementHandle(raw []byte) ([]byte, error) { + var req managementRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + path := strings.TrimSpace(req.Path) + switch { + case path == "/v0/resource/plugins/cpa-codexcont-executor/admin" || path == "/admin": + return managementHTML(adminHTML()) + case strings.Contains(path, "/admin/api/status"): + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "executor": statusPayload()}) + case strings.Contains(path, "/admin/api/summaries"): + return summariesResponse(req) + case strings.HasSuffix(path, "/status"): + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "executor": statusPayload()}) + case strings.HasSuffix(path, "/summaries"): + return summariesResponse(req) + default: + return jsonResponse(http.StatusNotFound, map[string]any{"ok": false, "error": "not_found"}) + } +} + +func summariesResponse(req managementRequest) ([]byte, error) { + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + limit := limitFromQuery(req.Query.Get("limit"), 100, 200) + items, err := store.RecentCodexSummaries(context.Background(), req.Query.Get("key_id"), limit) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + out := make([]map[string]any, 0, len(items)) + seen := map[string]bool{} + for _, summary := range monitor.Recent(limit) { + id := fmt.Sprint(summary["request_id"]) + if id != "" { + seen[id] = true + } + out = append(out, summary) + if len(out) >= limit { + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "summaries": out}) + } + } + for _, item := range items { + if seen[item.RequestID] { + continue + } + out = append(out, item.Summary) + if len(out) >= limit { + break + } + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "summaries": out}) +} + +func statusPayload() map[string]any { + cfg := loadedConfig() + return map[string]any{ + "plugin_id": pluginID, + "enabled": cfg.Enabled, + "route_enabled": cfg.RouteEnabled, + "state_db_path": cfg.StateDBPath, + "truncation_step": cfg.TruncationStep, + "max_continue": cfg.MaxContinue, + "mode": "executor_only", + "monitor": "cpamp_admin_resource", + "upstream_model": cfg.UpstreamModel, + "alias_count": len(cfg.UpstreamModelAliases), + } +} + +func okEnvelope(result any) ([]byte, error) { + raw, err := json.Marshal(result) + if err != nil { + return nil, err + } + return json.Marshal(envelope{OK: true, Result: raw}) +} + +func errorEnvelope(code, message string) []byte { + raw, _ := json.Marshal(envelope{OK: false, Error: &envelopeError{Code: code, Message: message}}) + return raw +} + +func jsonResponse(status int, body any) ([]byte, error) { + raw, err := json.Marshal(body) + if err != nil { + return nil, err + } + return okEnvelope(managementResponse{ + StatusCode: status, + Headers: http.Header{ + "Content-Type": []string{"application/json; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + }, + Body: raw, + }) +} + +func managementHTML(html string) ([]byte, error) { + return okEnvelope(managementResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{ + "Content-Type": []string{"text/html; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + }, + Body: []byte(html), + }) +} + +func adminHTML() string { + return adminHTMLTemplate +} + +func limitFromQuery(raw string, fallback, maxValue int) int { + limit, err := strconv.Atoi(strings.TrimSpace(raw)) + if err != nil || limit <= 0 { + return fallback + } + if limit > maxValue { + return maxValue + } + return limit +} + +func firstBytes(values ...[]byte) []byte { + for _, value := range values { + if len(value) > 0 { + return value + } + } + return nil +} + +func normalizedExecutorRequest(req executorCallRequest) executorRequest { + if !isZeroExecutorRequest(req.executorRequest) { + return req.executorRequest + } + return req.NestedExecutorRequest +} + +func isZeroExecutorRequest(req executorRequest) bool { + return strings.TrimSpace(req.AuthID) == "" && + strings.TrimSpace(req.AuthProvider) == "" && + strings.TrimSpace(req.Model) == "" && + strings.TrimSpace(req.Format) == "" && + !req.Stream && + strings.TrimSpace(req.Alt) == "" && + len(req.Headers) == 0 && + len(req.Query) == 0 && + len(req.OriginalRequest) == 0 && + strings.TrimSpace(req.SourceFormat) == "" && + len(req.Payload) == 0 && + len(req.Metadata) == 0 && + len(req.StorageJSON) == 0 && + len(req.AuthMetadata) == 0 && + len(req.AuthAttributes) == 0 +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func executorProtocol(value string) string { + value = strings.ToLower(strings.TrimSpace(value)) + switch value { + case "", "responses", "openai-responses", "openai_responses": + return executorFormatOpenAIResponse + default: + return value + } +} + +func resolveUpstreamModel(cfg executor.Config, req executorRequest, body []byte) string { + cfg = cfg.Normalize() + requested := firstNonEmpty(modelFromBody(body), req.Model) + if cfg.UpstreamModel != "" { + return cfg.UpstreamModel + } + if len(cfg.UpstreamModelAliases) > 0 { + if target := strings.TrimSpace(cfg.UpstreamModelAliases[requested]); target != "" { + return target + } + requestedLower := strings.ToLower(strings.TrimSpace(requested)) + for alias, target := range cfg.UpstreamModelAliases { + if strings.ToLower(strings.TrimSpace(alias)) == requestedLower { + if target = strings.TrimSpace(target); target != "" { + return target + } + } + } + } + return requested +} + +func rewritePayloadModel(body []byte, model string) []byte { + out, _ := rewriteUpstreamPayload(body, model) + return out +} + +func rewriteUpstreamPayload(body []byte, model string) ([]byte, []string) { + model = strings.TrimSpace(model) + if model == "" || len(body) == 0 { + return append([]byte(nil), body...), nil + } + var raw map[string]any + if err := json.Unmarshal(body, &raw); err != nil { + return append([]byte(nil), body...), nil + } + filtered := filterUnsupportedTools(raw, model) + current, _ := raw["model"].(string) + if strings.TrimSpace(current) == model && len(filtered) == 0 { + return append([]byte(nil), body...), nil + } + raw["model"] = model + out, err := json.Marshal(raw) + if err != nil { + return append([]byte(nil), body...), nil + } + return out, filtered +} + +func filterUnsupportedTools(raw map[string]any, upstreamModel string) []string { + if raw == nil || !strings.EqualFold(strings.TrimSpace(upstreamModel), executor.DefaultCodexUpstreamModel) { + return nil + } + tools, ok := raw["tools"].([]any) + if !ok || len(tools) == 0 { + return nil + } + kept := make([]any, 0, len(tools)) + filteredSet := map[string]bool{} + for _, tool := range tools { + toolMap, ok := tool.(map[string]any) + if !ok { + kept = append(kept, tool) + continue + } + toolType := strings.TrimSpace(fmt.Sprint(toolMap["type"])) + if strings.EqualFold(toolType, "image_generation") { + filteredSet["image_generation"] = true + continue + } + kept = append(kept, tool) + } + if len(filteredSet) == 0 { + return nil + } + if len(kept) == 0 { + delete(raw, "tools") + } else { + raw["tools"] = kept + } + if isImageGenerationToolChoice(raw["tool_choice"]) { + delete(raw, "tool_choice") + } + filtered := make([]string, 0, len(filteredSet)) + for toolType := range filteredSet { + filtered = append(filtered, toolType) + } + sort.Strings(filtered) + return filtered +} + +func isImageGenerationToolChoice(value any) bool { + switch choice := value.(type) { + case string: + return strings.EqualFold(strings.TrimSpace(choice), "image_generation") + case map[string]any: + if strings.EqualFold(strings.TrimSpace(fmt.Sprint(choice["type"])), "image_generation") { + return true + } + if strings.EqualFold(strings.TrimSpace(fmt.Sprint(choice["name"])), "image_generation") { + return true + } + } + return false +} + +func cloneHeader(in http.Header) http.Header { + out := http.Header{} + for key, values := range in { + out[key] = append([]string(nil), values...) + } + return out +} + +func cloneValues(in map[string][]string) map[string][]string { + out := map[string][]string{} + for key, values := range in { + out[key] = append([]string(nil), values...) + } + return out +} + +func cloneSummary(in map[string]any) map[string]any { + out := map[string]any{} + for key, value := range in { + out[key] = value + } + return out +} + +type streamDiagnostics struct { + mu sync.Mutex + rounds map[int]map[string]any + order []int +} + +func newStreamDiagnostics() *streamDiagnostics { + return &streamDiagnostics{rounds: map[int]map[string]any{}} +} + +func (d *streamDiagnostics) RecordOpen(round int, req hostModelExecutionRequest, requestedModel, bodyModel string, bodyBytes int, filteredTools []string) { + if d == nil { + return + } + d.mu.Lock() + defer d.mu.Unlock() + item := d.ensureRoundLocked(round) + item["entry_protocol"] = req.EntryProtocol + item["exit_protocol"] = req.ExitProtocol + item["model"] = req.Model + item["requested_model"] = requestedModel + item["body_model"] = bodyModel + item["body_bytes"] = bodyBytes + item["stream"] = req.Stream + item["host_callback"] = strings.TrimSpace(req.HostCallbackID) != "" + if len(filteredTools) > 0 { + item["filtered_tool_types"] = append([]string(nil), filteredTools...) + } +} + +func (d *streamDiagnostics) RecordOpenResponse(round int, resp hostModelStreamResponse) { + if d == nil { + return + } + d.mu.Lock() + defer d.mu.Unlock() + item := d.ensureRoundLocked(round) + item["open_status_code"] = resp.StatusCode + item["stream_id_present"] = strings.TrimSpace(resp.StreamID) != "" + if contentType := resp.Headers.Get("Content-Type"); contentType != "" { + item["response_content_type"] = contentType + } +} + +func (d *streamDiagnostics) RecordOpenError(round int, err error) { + if d == nil || err == nil { + return + } + d.mu.Lock() + defer d.mu.Unlock() + item := d.ensureRoundLocked(round) + item["open_error"] = brief(err.Error(), 160) +} + +func (d *streamDiagnostics) RecordRead(round int, readNo int, payload []byte, done bool, err error) { + if d == nil { + return + } + d.mu.Lock() + defer d.mu.Unlock() + item := d.ensureRoundLocked(round) + if readNo <= 0 { + readNo = 1 + } + item["read_count"] = readNo + item["last_read_payload_bytes"] = len(payload) + item["last_read_done"] = done + item["last_read_kind"] = payloadKind(payload) + if readNo == 1 { + item["first_read_payload_bytes"] = len(payload) + item["first_read_done"] = done + item["first_read_kind"] = payloadKind(payload) + } + if err != nil { + item["last_read_error"] = brief(err.Error(), 160) + if readNo == 1 { + item["first_read_error"] = brief(err.Error(), 160) + } + } +} + +func (d *streamDiagnostics) Snapshot() map[string]any { + if d == nil { + return nil + } + d.mu.Lock() + defer d.mu.Unlock() + if len(d.order) == 0 { + return nil + } + rounds := make([]map[string]any, 0, len(d.order)) + for _, round := range d.order { + rounds = append(rounds, cloneSummary(d.rounds[round])) + } + return map[string]any{"rounds": rounds} +} + +func (d *streamDiagnostics) ensureRoundLocked(round int) map[string]any { + if round <= 0 { + round = 1 + } + item := d.rounds[round] + if item == nil { + item = map[string]any{"round": round} + d.rounds[round] = item + d.order = append(d.order, round) + } + return item +} + +func modelFromBody(body []byte) string { + var raw map[string]any + if err := json.Unmarshal(body, &raw); err != nil { + return "" + } + return fmt.Sprint(raw["model"]) +} + +func brief(text string, limit int) string { + if limit <= 0 || len(text) <= limit { + return text + } + return text[:limit] +} + +func payloadKind(payload []byte) string { + trimmed := bytes.TrimSpace(payload) + switch { + case len(trimmed) == 0: + return "empty" + case bytes.HasPrefix(trimmed, []byte("event:")): + return "sse_event" + case bytes.HasPrefix(trimmed, []byte("data:")): + return "sse_data" + case bytes.Equal(trimmed, []byte("[DONE]")): + return "done" + case json.Valid(trimmed): + return "json" + default: + return "other" + } +} + +type summaryMonitor struct { + mu sync.Mutex + limit int + items map[string]map[string]any + order []string +} + +func newSummaryMonitor(limit int) *summaryMonitor { + return &summaryMonitor{limit: limit, items: map[string]map[string]any{}} +} + +func (m *summaryMonitor) Start(req executorRequest, base map[string]any) string { + if m == nil { + return "" + } + now := time.Now().UTC().Format(time.RFC3339Nano) + id := fmt.Sprintf("processing-%d", time.Now().UnixNano()) + model := firstNonEmpty(req.Model, fmt.Sprint(base["model"])) + item := map[string]any{ + "request_id": id, + "model": model, + "started_at": now, + "updated_at": now, + "status": "processing", + "final_status": "processing", + "protection": "processing", + "latest_round": 0, + "continuation_count": 0, + "rounds": []map[string]any{}, + } + if keyID := strings.TrimSpace(req.AuthID); keyID != "" { + item["key_identity"] = map[string]any{"known": true, "id": keyID} + } + m.upsert(id, item) + return id +} + +func (m *summaryMonitor) Finish(processingID string, summary map[string]any) { + if m == nil || len(summary) == 0 { + return + } + m.mu.Lock() + defer m.mu.Unlock() + if processingID != "" { + m.removeLocked(processingID) + } + id := processingID + if requestID := strings.TrimSpace(fmt.Sprint(summary["request_id"])); requestID != "" && requestID != "" { + id = requestID + } + m.upsertLocked(id, cloneSummary(summary)) +} + +func (m *summaryMonitor) Fail(processingID, reason string) { + if m == nil || processingID == "" { + return + } + m.mu.Lock() + defer m.mu.Unlock() + item := m.items[processingID] + if item == nil { + return + } + now := time.Now().UTC().Format(time.RFC3339Nano) + item["updated_at"] = now + item["ended_at"] = now + item["status"] = "failed" + item["final_status"] = "failed" + item["protection"] = "failed" + item["failure_reason"] = reason +} + +func (m *summaryMonitor) Recent(limit int) []map[string]any { + if m == nil { + return nil + } + if limit <= 0 || limit > m.limit { + limit = m.limit + } + m.mu.Lock() + defer m.mu.Unlock() + out := make([]map[string]any, 0, limit) + for i := len(m.order) - 1; i >= 0 && len(out) < limit; i-- { + if item := m.items[m.order[i]]; item != nil { + out = append(out, cloneSummary(item)) + } + } + return out +} + +func (m *summaryMonitor) upsert(id string, item map[string]any) { + m.mu.Lock() + defer m.mu.Unlock() + m.upsertLocked(id, item) +} + +func (m *summaryMonitor) upsertLocked(id string, item map[string]any) { + if strings.TrimSpace(id) == "" { + id = fmt.Sprintf("summary-%d", time.Now().UnixNano()) + item["request_id"] = id + } + if _, exists := m.items[id]; !exists { + m.order = append(m.order, id) + } + m.items[id] = item + m.trimLocked() +} + +func (m *summaryMonitor) removeLocked(id string) { + delete(m.items, id) + for i, value := range m.order { + if value == id { + m.order = append(m.order[:i], m.order[i+1:]...) + return + } + } +} + +func (m *summaryMonitor) trimLocked() { + if m.limit <= 0 { + m.limit = 160 + } + for len(m.order) > m.limit { + oldest := m.order[0] + m.order = m.order[1:] + delete(m.items, oldest) + } +} diff --git a/cpa_codexcont_executor_plugin/go/main_test.go b/cpa_codexcont_executor_plugin/go/main_test.go new file mode 100644 index 0000000..171e7cd --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/main_test.go @@ -0,0 +1,666 @@ +package main + +import ( + "encoding/json" + "net/http" + "path/filepath" + "strings" + "sync" + "testing" + "time" + + "codexcont/cpa-codexcont-executor-plugin/internal/executor" +) + +func unwrapEnvelope(t *testing.T, raw []byte, out any) { + t.Helper() + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + if !env.OK { + t.Fatalf("envelope error: %#v", env.Error) + } + if err := json.Unmarshal(env.Result, out); err != nil { + t.Fatal(err) + } +} + +func configureTestState(t *testing.T, routeEnabled bool) { + t.Helper() + configureTestStateWithConfig(t, func(cfg *executor.Config) { + cfg.RouteEnabled = routeEnabled + }) +} + +func configureTestStateWithConfig(t *testing.T, mutate func(*executor.Config)) { + t.Helper() + path := filepath.Join(t.TempDir(), "executor.sqlite") + store, err := executor.OpenStore(path) + if err != nil { + t.Fatal(err) + } + cfg := executor.DefaultConfig() + cfg.StateDBPath = path + if mutate != nil { + mutate(&cfg) + } + cfg = cfg.Normalize() + state.mu.Lock() + if state.store != nil { + _ = state.store.Close() + } + state.cfg = cfg + state.store = store + monitor = newSummaryMonitor(160) + state.mu.Unlock() + t.Cleanup(func() { + state.mu.Lock() + if state.store != nil { + _ = state.store.Close() + state.store = nil + } + state.mu.Unlock() + }) +} + +func TestRegistrationIsExecutorOnly(t *testing.T) { + raw, err := handleMethod(methodPluginRegister, nil) + if err != nil { + t.Fatal(err) + } + var reg registration + unwrapEnvelope(t, raw, ®) + if reg.Metadata.Name != pluginID { + t.Fatalf("name = %q", reg.Metadata.Name) + } + if !reg.Capabilities.FrontendAuthProvider || reg.Capabilities.FrontendAuthProviderExclusive { + t.Fatalf("executor plugin should expose non-exclusive frontend auth shim: %#v", reg.Capabilities) + } + if !reg.Capabilities.ModelRouter || !reg.Capabilities.Executor || !reg.Capabilities.ManagementAPI || !reg.Capabilities.UsagePlugin { + t.Fatalf("executor capabilities missing: %#v", reg.Capabilities) + } + if reg.Capabilities.ExecutorModelScope != executorModelScopeBoth { + t.Fatalf("executor model scope = %q, want official CPA scope %q", reg.Capabilities.ExecutorModelScope, executorModelScopeBoth) + } + if strings.Join(reg.Capabilities.ExecutorInputFormats, ",") != executorFormatOpenAIResponse || + strings.Join(reg.Capabilities.ExecutorOutputFormats, ",") != executorFormatOpenAIResponse { + t.Fatalf("executor must declare CPA's native Responses format to avoid request translation drift: %#v", reg.Capabilities) + } +} + +func TestExecutorIdentifierUsesOfficialField(t *testing.T) { + raw, err := handleMethod(methodExecutorIdentifier, nil) + if err != nil { + t.Fatal(err) + } + var resp struct { + Identifier string `json:"identifier"` + ID string `json:"id"` + } + unwrapEnvelope(t, raw, &resp) + if resp.Identifier != pluginID { + t.Fatalf("identifier = %q, want %q", resp.Identifier, pluginID) + } + if resp.ID != "" { + t.Fatalf("executor.identifier must not rely on non-CPA id field: %#v", resp) + } +} + +func TestReconfigureReturnsFullRegistration(t *testing.T) { + path := filepath.Join(t.TempDir(), "executor.sqlite") + t.Cleanup(shutdownPlugin) + raw, err := handleMethod(methodPluginReconfigure, mustJSON(t, lifecycleRequest{ConfigYAML: []byte("enabled: true\nroute_enabled: false\nstate_db_path: " + path + "\nupstream_model_aliases:\n gpt-5.4: gpt-5.3-codex-spark\n")})) + if err != nil { + t.Fatal(err) + } + var reg registration + unwrapEnvelope(t, raw, ®) + if reg.Metadata.Name != pluginID || !reg.Capabilities.ManagementAPI { + t.Fatalf("reconfigure must return full registration for CPA active snapshot: %#v", reg) + } + cfg := loadedConfig() + if cfg.UpstreamModelAliases["gpt-5.4"] != "gpt-5.3-codex-spark" || cfg.UpstreamModelAliases["gpt-5.5"] != "gpt-5.3-codex-spark" { + t.Fatalf("upstream aliases not loaded: %#v", cfg.UpstreamModelAliases) + } +} + +func TestManagementRegisterExposesOnlyAdminMonitorResource(t *testing.T) { + raw, err := managementRegister() + if err != nil { + t.Fatal(err) + } + var reg managementRegistrationResponse + unwrapEnvelope(t, raw, ®) + foundAdmin := false + menuCount := 0 + for _, resource := range reg.Resources { + if resource.Path == "/admin" && resource.Menu == "CodexCont Executor" { + foundAdmin = true + } + if resource.Menu == "CodexCont Executor" { + menuCount++ + } + if strings.HasPrefix(resource.Path, "/user") || strings.Contains(resource.Path, "/keys") || strings.Contains(resource.Path, "/quota") { + t.Fatalf("executor resource overlaps user/key control plane: %#v", resource) + } + } + if !foundAdmin { + t.Fatalf("executor monitor admin resource missing: %#v", reg.Resources) + } + for _, route := range reg.Routes { + if strings.TrimSpace(route.Menu) != "" { + t.Fatalf("management route should stay hidden from CPAMP sidebar: %#v", route) + } + if strings.Contains(route.Path, "/user") || strings.Contains(route.Path, "/keys") || strings.Contains(route.Path, "/quota") { + t.Fatalf("executor route overlaps user/key control plane: %#v", route) + } + } + if menuCount != 1 { + t.Fatalf("executor should expose exactly one CPAMP menu entry, got %d in %#v", menuCount, reg) + } +} + +func TestRouteSwitch(t *testing.T) { + body := []byte(`{"model":"gpt-5.5","stream":true}`) + configureTestState(t, false) + raw, err := routeModel(mustJSON(t, modelRouteRequest{SourceFormat: "openai", Stream: true, Body: body})) + if err != nil { + t.Fatal(err) + } + var resp modelRouteResponse + unwrapEnvelope(t, raw, &resp) + if resp.Handled { + t.Fatalf("disabled route handled request: %#v", resp) + } + configureTestState(t, true) + raw, err = routeModel(mustJSON(t, modelRouteRequest{SourceFormat: "openai-response", Stream: true, Body: body})) + if err != nil { + t.Fatal(err) + } + unwrapEnvelope(t, raw, &resp) + if !resp.Handled || resp.TargetKind != routeTargetSelf { + t.Fatalf("enabled route did not handle streaming response: %#v", resp) + } + raw, err = routeModel(mustJSON(t, modelRouteRequest{SourceFormat: "openai", Stream: true, Body: []byte(`{"model":"gpt-5.5","stream":true,"messages":[{"role":"user","content":"hi"}]}`)})) + if err != nil { + t.Fatal(err) + } + unwrapEnvelope(t, raw, &resp) + if resp.Handled { + t.Fatalf("chat-completions request should not be handled: %#v", resp) + } + raw, err = routeModel(mustJSON(t, modelRouteRequest{SourceFormat: "openai", Stream: false, Body: []byte(`{"model":"gpt-5.5"}`)})) + if err != nil { + t.Fatal(err) + } + unwrapEnvelope(t, raw, &resp) + if resp.Handled { + t.Fatalf("non-stream request should not be handled: %#v", resp) + } +} + +func TestManagementStatusNoStore(t *testing.T) { + configureTestState(t, true) + raw, err := managementHandle(mustJSON(t, managementRequest{Method: http.MethodGet, Path: "/plugins/cpa-codexcont-executor/status"})) + if err != nil { + t.Fatal(err) + } + var resp managementResponse + unwrapEnvelope(t, raw, &resp) + if resp.StatusCode != http.StatusOK || resp.Headers.Get("Cache-Control") != "no-store" { + t.Fatalf("management response = %#v", resp) + } +} + +func TestManagementAdminMonitorHTML(t *testing.T) { + configureTestState(t, false) + raw, err := managementHandle(mustJSON(t, managementRequest{Method: http.MethodGet, Path: "/v0/resource/plugins/cpa-codexcont-executor/admin"})) + if err != nil { + t.Fatal(err) + } + var resp managementResponse + unwrapEnvelope(t, raw, &resp) + html := string(resp.Body) + if resp.StatusCode != http.StatusOK || resp.Headers.Get("Cache-Control") != "no-store" || !strings.Contains(resp.Headers.Get("Content-Type"), "text/html") { + t.Fatalf("admin monitor response = %#v", resp) + } + for _, want := range []string{"实时滚动监控", "/v0/resource/plugins/cpa-codexcont-executor/admin/api", "summaries?limit=120"} { + if !strings.Contains(html, want) { + t.Fatalf("admin monitor html missing %q", want) + } + } + for _, forbidden := range []string{"/user/api", "keys/create", "quota"} { + if strings.Contains(html, forbidden) { + t.Fatalf("admin monitor html should not expose %q", forbidden) + } + } +} + +func TestUsageHandleIsObservabilityOnlyNoop(t *testing.T) { + raw, err := handleMethod(methodUsageHandle, []byte(`{"api_key":"key-1"}`)) + if err != nil { + t.Fatal(err) + } + var body map[string]any + unwrapEnvelope(t, raw, &body) + if body["handled"] != false || body["observability_only"] != true { + t.Fatalf("usage handle should be no-op observability shim: %#v", body) + } +} + +func TestFrontendAuthIsObservabilityOnlyNoop(t *testing.T) { + raw, err := handleMethod(methodFrontendAuthAuthenticate, []byte(`{"Path":"/v1/responses"}`)) + if err != nil { + t.Fatal(err) + } + var resp frontendAuthResponse + unwrapEnvelope(t, raw, &resp) + if resp.Authenticated || resp.Metadata["mode"] != "observability_only" { + t.Fatalf("frontend auth should not authenticate requests: %#v", resp) + } +} + +func TestSummariesIncludeProcessingMonitor(t *testing.T) { + configureTestState(t, true) + id := monitor.Start(executorRequest{AuthID: "key-1", Model: "gpt-5.5"}, map[string]any{"model": "gpt-5.5"}) + if id == "" { + t.Fatal("processing monitor id is empty") + } + raw, err := managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodGet, + Path: "/v0/resource/plugins/cpa-codexcont-executor/admin/api/summaries", + Query: map[string][]string{"limit": {"10"}}, + })) + if err != nil { + t.Fatal(err) + } + var resp managementResponse + unwrapEnvelope(t, raw, &resp) + var body struct { + OK bool `json:"ok"` + Summaries []map[string]any `json:"summaries"` + } + if err := json.Unmarshal(resp.Body, &body); err != nil { + t.Fatal(err) + } + if len(body.Summaries) != 1 || body.Summaries[0]["protection"] != "processing" { + t.Fatalf("processing summary missing: %#v", body.Summaries) + } + if body.Summaries[0]["request_id"] != id { + t.Fatalf("request id = %#v, want %s", body.Summaries[0]["request_id"], id) + } +} + +func TestExecuteStreamAcceptsOfficialFlattenedRPCPayload(t *testing.T) { + configureTestState(t, true) + originalHostCall := hostCall + t.Cleanup(func() { hostCall = originalHostCall }) + + requestBody := []byte(`{"model":"gpt-5.5","stream":true,"input":[{"role":"user","content":"hi"}],"tools":[{"type":"image_generation"},{"type":"function","name":"lookup"}],"tool_choice":"image_generation"}`) + upstreamChunks := [][]byte{ + executor.SerializeEvent(map[string]any{"type": "response.created", "response": map[string]any{"id": "resp-flat", "status": "in_progress"}}), + executor.SerializeEvent(map[string]any{"type": "response.completed", "response": map[string]any{ + "id": "resp-flat", + "status": "completed", + "usage": map[string]any{ + "input_tokens": 10, + "output_tokens": 20, + "total_tokens": 30, + "output_tokens_details": map[string]any{ + "reasoning_tokens": 12, + }, + }, + }}), + } + var mu sync.Mutex + var openedBody []byte + var openedModel string + var openedProtocol string + var openedHostCallbackID string + var emitted strings.Builder + readIndex := 0 + done := make(chan struct{}) + closeOnce := sync.Once{} + hostCall = func(method string, payload any) (json.RawMessage, error) { + mu.Lock() + defer mu.Unlock() + switch method { + case methodHostModelExecuteStream: + req := payload.(hostModelExecutionRequest) + openedModel = req.Model + openedBody = append([]byte(nil), req.Body...) + openedProtocol = req.EntryProtocol + "->" + req.ExitProtocol + openedHostCallbackID = req.HostCallbackID + return rawJSON(t, hostModelStreamResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{"Content-Type": []string{"text/event-stream"}}, + StreamID: "upstream-flat", + }), nil + case methodHostModelStreamRead: + if readIndex >= len(upstreamChunks) { + return rawJSON(t, hostModelStreamReadResponse{Done: true}), nil + } + chunk := upstreamChunks[readIndex] + readIndex++ + return rawJSON(t, hostModelStreamReadResponse{Payload: chunk, Done: readIndex >= len(upstreamChunks)}), nil + case methodHostModelStreamClose: + return rawJSON(t, map[string]any{}), nil + case methodHostStreamEmit: + req := payload.(hostStreamEmitRequest) + emitted.Write(req.Payload) + return rawJSON(t, map[string]any{}), nil + case methodHostStreamClose: + closeOnce.Do(func() { close(done) }) + return rawJSON(t, map[string]any{}), nil + default: + t.Fatalf("unexpected host call %s", method) + return nil, nil + } + } + + raw, err := executorExecuteStream(mustJSON(t, map[string]any{ + "AuthID": "key-1", + "Model": "gpt-5.5", + "Format": "openai-response", + "Stream": true, + "SourceFormat": "openai-response", + "Payload": requestBody, + "OriginalRequest": requestBody, + "stream_id": "client-flat", + "host_callback_id": "callback-flat", + })) + if err != nil { + t.Fatal(err) + } + var resp executorStreamResponse + unwrapEnvelope(t, raw, &resp) + if resp.Headers.Get("Content-Type") != "text/event-stream" { + t.Fatalf("stream headers = %#v", resp.Headers) + } + select { + case <-done: + case <-time.After(time.Second): + t.Fatal("executor stream did not finish") + } + mu.Lock() + defer mu.Unlock() + var opened map[string]any + if err := json.Unmarshal(openedBody, &opened); err != nil { + t.Fatalf("opened body is not JSON: %v body=%q", err, openedBody) + } + if openedModel != "gpt-5.3-codex-spark" || opened["model"] != "gpt-5.3-codex-spark" || len(openedBody) == 0 || strings.Contains(string(openedBody), "reasoning.encrypted_content") { + t.Fatalf("opened upstream request did not use stable model alias: request=%q body=%s", openedModel, string(openedBody)) + } + if strings.Contains(string(openedBody), "image_generation") || !strings.Contains(string(openedBody), `"type":"function"`) { + t.Fatalf("opened upstream body should filter only unsupported Spark built-ins: %s", openedBody) + } + if openedProtocol != "openai-response->openai-response" { + t.Fatalf("opened protocol = %q, want openai-response->openai-response", openedProtocol) + } + if openedHostCallbackID != "callback-flat" { + t.Fatalf("host model stream callback id = %q, want callback-flat", openedHostCallbackID) + } + if !strings.Contains(emitted.String(), "response.completed") { + t.Fatalf("emitted stream missing terminal event:\n%s", emitted.String()) + } + summaryRaw, err := managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodGet, + Path: "/v0/resource/plugins/cpa-codexcont-executor/admin/api/summaries", + Query: map[string][]string{"limit": {"5"}}, + })) + if err != nil { + t.Fatal(err) + } + var summaryResp managementResponse + unwrapEnvelope(t, summaryRaw, &summaryResp) + var summaryBody struct { + Summaries []map[string]any `json:"summaries"` + } + if err := json.Unmarshal(summaryResp.Body, &summaryBody); err != nil { + t.Fatal(err) + } + if len(summaryBody.Summaries) == 0 { + t.Fatal("expected saved executor summary") + } + gotSummary := summaryBody.Summaries[0] + if gotSummary["model"] != "gpt-5.5" { + t.Fatalf("summary model = %#v, want gpt-5.5", gotSummary["model"]) + } + diagnostics, _ := gotSummary["diagnostics"].(map[string]any) + diagRounds, _ := diagnostics["rounds"].([]any) + if len(diagRounds) == 0 { + t.Fatalf("summary diagnostics missing: %#v", gotSummary) + } + firstDiag, _ := diagRounds[0].(map[string]any) + if firstDiag["model"] != "gpt-5.3-codex-spark" || firstDiag["requested_model"] != "gpt-5.5" || firstDiag["body_model"] != "gpt-5.3-codex-spark" { + t.Fatalf("diagnostics should record gpt-5.5 alias routing: %#v", firstDiag) + } + filtered, _ := firstDiag["filtered_tool_types"].([]any) + if len(filtered) != 1 || filtered[0] != "image_generation" { + t.Fatalf("diagnostics should safely record filtered tool types: %#v", firstDiag) + } + if _, leaked := firstDiag["body"]; leaked { + t.Fatalf("diagnostics must not include request body: %#v", firstDiag) + } + if firstDiag["host_callback"] != true || firstDiag["stream_id_present"] != true { + t.Fatalf("unsafe or incomplete diagnostics: %#v", firstDiag) + } + if firstDiag["first_read_payload_bytes"].(float64) <= 0 { + t.Fatalf("first read diagnostics did not record payload length: %#v", firstDiag) + } +} + +func TestExecuteStreamUsesConfiguredUpstreamModelAlias(t *testing.T) { + configureTestStateWithConfig(t, func(cfg *executor.Config) { + cfg.RouteEnabled = true + cfg.UpstreamModelAliases = map[string]string{"gpt-5.4": "gpt-5.3-codex-spark"} + }) + originalHostCall := hostCall + t.Cleanup(func() { hostCall = originalHostCall }) + + requestBody := []byte(`{"model":"gpt-5.4","stream":true,"input":[{"role":"user","content":"hi"}]}`) + upstreamChunks := [][]byte{ + executor.SerializeEvent(map[string]any{"type": "response.created", "response": map[string]any{"id": "resp-alias", "model": "gpt-5.3-codex-spark", "status": "in_progress"}}), + executor.SerializeEvent(map[string]any{"type": "response.completed", "response": map[string]any{ + "id": "resp-alias", + "model": "gpt-5.3-codex-spark", + "status": "completed", + "usage": map[string]any{ + "input_tokens": 10, + "output_tokens": 20, + "total_tokens": 30, + "output_tokens_details": map[string]any{ + "reasoning_tokens": 12, + }, + }, + }}), + } + var mu sync.Mutex + var openedModel string + var openedBodyModel string + var emitted strings.Builder + readIndex := 0 + done := make(chan struct{}) + closeOnce := sync.Once{} + hostCall = func(method string, payload any) (json.RawMessage, error) { + mu.Lock() + defer mu.Unlock() + switch method { + case methodHostModelExecuteStream: + req := payload.(hostModelExecutionRequest) + openedModel = req.Model + openedBodyModel = modelFromBody(req.Body) + return rawJSON(t, hostModelStreamResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{"Content-Type": []string{"text/event-stream"}}, + StreamID: "upstream-alias", + }), nil + case methodHostModelStreamRead: + if readIndex >= len(upstreamChunks) { + return rawJSON(t, hostModelStreamReadResponse{Done: true}), nil + } + chunk := upstreamChunks[readIndex] + readIndex++ + return rawJSON(t, hostModelStreamReadResponse{Payload: chunk, Done: readIndex >= len(upstreamChunks)}), nil + case methodHostModelStreamClose: + return rawJSON(t, map[string]any{}), nil + case methodHostStreamEmit: + req := payload.(hostStreamEmitRequest) + emitted.Write(req.Payload) + return rawJSON(t, map[string]any{}), nil + case methodHostStreamClose: + closeOnce.Do(func() { close(done) }) + return rawJSON(t, map[string]any{}), nil + default: + t.Fatalf("unexpected host call %s", method) + return nil, nil + } + } + + raw, err := executorExecuteStream(mustJSON(t, map[string]any{ + "AuthID": "key-1", + "Model": "gpt-5.4", + "Format": "openai-response", + "Stream": true, + "SourceFormat": "openai-response", + "Payload": requestBody, + "OriginalRequest": requestBody, + "stream_id": "client-alias", + "host_callback_id": "callback-alias", + })) + if err != nil { + t.Fatal(err) + } + var resp executorStreamResponse + unwrapEnvelope(t, raw, &resp) + select { + case <-done: + case <-time.After(time.Second): + t.Fatal("executor stream did not finish") + } + mu.Lock() + defer mu.Unlock() + if openedModel != "gpt-5.3-codex-spark" || openedBodyModel != "gpt-5.3-codex-spark" { + t.Fatalf("upstream model mismatch: request=%q body=%q", openedModel, openedBodyModel) + } + if !strings.Contains(emitted.String(), `"model":"gpt-5.4"`) { + t.Fatalf("downstream stream should keep client-visible model:\n%s", emitted.String()) + } + if strings.Contains(emitted.String(), `"model":"gpt-5.3-codex-spark"`) { + t.Fatalf("downstream stream leaked upstream model:\n%s", emitted.String()) + } + summaryRaw, err := managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodGet, + Path: "/v0/resource/plugins/cpa-codexcont-executor/admin/api/summaries", + Query: map[string][]string{"limit": {"5"}}, + })) + if err != nil { + t.Fatal(err) + } + var summaryResp managementResponse + unwrapEnvelope(t, summaryRaw, &summaryResp) + var summaryBody struct { + Summaries []map[string]any `json:"summaries"` + } + if err := json.Unmarshal(summaryResp.Body, &summaryBody); err != nil { + t.Fatal(err) + } + if len(summaryBody.Summaries) == 0 { + t.Fatal("expected saved executor summary") + } + gotSummary := summaryBody.Summaries[0] + if gotSummary["model"] != "gpt-5.4" { + t.Fatalf("summary model = %#v, want client model", gotSummary["model"]) + } + diagnostics, _ := gotSummary["diagnostics"].(map[string]any) + diagRounds, _ := diagnostics["rounds"].([]any) + firstDiag, _ := diagRounds[0].(map[string]any) + if firstDiag["model"] != "gpt-5.3-codex-spark" || firstDiag["requested_model"] != "gpt-5.4" || firstDiag["body_model"] != "gpt-5.3-codex-spark" { + t.Fatalf("diagnostics should show safe alias routing evidence: %#v", firstDiag) + } +} + +func TestRewritePayloadModelLeavesInvalidJSONUntouched(t *testing.T) { + raw := []byte(`not-json`) + got := rewritePayloadModel(raw, "gpt-5.3-codex-spark") + if string(got) != string(raw) { + t.Fatalf("invalid JSON should be copied unchanged: %q", got) + } +} + +func TestRewriteUpstreamPayloadFiltersUnsupportedSparkBuiltins(t *testing.T) { + raw := []byte(`{"model":"gpt-5.5","tools":[{"type":"image_generation"},{"type":"function","name":"lookup"}],"tool_choice":{"type":"image_generation"}}`) + got, filtered := rewriteUpstreamPayload(raw, "gpt-5.3-codex-spark") + if strings.Join(filtered, ",") != "image_generation" { + t.Fatalf("filtered tools = %#v", filtered) + } + var body map[string]any + if err := json.Unmarshal(got, &body); err != nil { + t.Fatal(err) + } + if body["model"] != "gpt-5.3-codex-spark" { + t.Fatalf("model was not rewritten: %s", got) + } + if _, ok := body["tool_choice"]; ok { + t.Fatalf("image_generation tool_choice should be removed: %s", got) + } + tools, _ := body["tools"].([]any) + if len(tools) != 1 { + t.Fatalf("expected only custom/function tool to remain: %#v body=%s", tools, got) + } + tool, _ := tools[0].(map[string]any) + if tool["type"] != "function" || tool["name"] != "lookup" { + t.Fatalf("function tool should be preserved: %#v", tool) + } +} + +func TestRewriteUpstreamPayloadRemovesEmptyTools(t *testing.T) { + raw := []byte(`{"model":"gpt-5.5","tools":[{"type":"image_generation"}]}`) + got, filtered := rewriteUpstreamPayload(raw, "gpt-5.3-codex-spark") + if strings.Join(filtered, ",") != "image_generation" { + t.Fatalf("filtered tools = %#v", filtered) + } + var body map[string]any + if err := json.Unmarshal(got, &body); err != nil { + t.Fatal(err) + } + if _, ok := body["tools"]; ok { + t.Fatalf("empty tools array should be omitted: %s", got) + } +} + +func TestExecutorProtocolDefaultsToOpenAIResponse(t *testing.T) { + for _, value := range []string{"", "responses", "openai-responses", "openai_responses"} { + if got := executorProtocol(value); got != "openai-response" { + t.Fatalf("executorProtocol(%q) = %q", value, got) + } + } + if got := executorProtocol("openai-response"); got != "openai-response" { + t.Fatalf("executorProtocol(openai-response) = %q", got) + } +} + +func TestExecutorRequestNormalizationKeepsLegacyNestedPayload(t *testing.T) { + body := []byte(`{"model":"gpt-5.5","stream":true}`) + req := normalizedExecutorRequest(executorCallRequest{ + NestedExecutorRequest: executorRequest{Model: "gpt-5.5", Payload: body}, + }) + if req.Model != "gpt-5.5" || string(req.Payload) != string(body) { + t.Fatalf("normalized nested request = %#v", req) + } +} + +func mustJSON(t *testing.T, value any) []byte { + t.Helper() + return rawJSON(t, value) +} + +func rawJSON(t *testing.T, value any) []byte { + t.Helper() + raw, err := json.Marshal(value) + if err != nil { + t.Fatal(err) + } + return raw +} diff --git a/cpa_codexcont_executor_plugin/go/plugin_export.go b/cpa_codexcont_executor_plugin/go/plugin_export.go new file mode 100644 index 0000000..f0db1d4 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/plugin_export.go @@ -0,0 +1,166 @@ +//go:build cliproxy_plugin + +package main + +/* +#include +#include + +typedef struct { + void* ptr; + size_t len; +} cliproxy_buffer; + +typedef int (*cliproxy_host_call_fn)(void*, const char*, const uint8_t*, size_t, cliproxy_buffer*); +typedef void (*cliproxy_host_free_fn)(void*, size_t); + +typedef struct { + uint32_t abi_version; + void* host_ctx; + cliproxy_host_call_fn call; + cliproxy_host_free_fn free_buffer; +} cliproxy_host_api; + +typedef int (*cliproxy_plugin_call_fn)(char*, uint8_t*, size_t, cliproxy_buffer*); +typedef void (*cliproxy_plugin_free_fn)(void*, size_t); +typedef void (*cliproxy_plugin_shutdown_fn)(void); + +typedef struct { + uint32_t abi_version; + cliproxy_plugin_call_fn call; + cliproxy_plugin_free_fn free_buffer; + cliproxy_plugin_shutdown_fn shutdown; +} cliproxy_plugin_api; + +extern int cliproxyPluginCall(char*, uint8_t*, size_t, cliproxy_buffer*); +extern void cliproxyPluginFree(void*, size_t); +extern void cliproxyPluginShutdown(void); + +static const cliproxy_host_api* stored_host; + +static void store_host_api(const cliproxy_host_api* host) { + stored_host = host; +} + +static int call_host_api(const char* method, const uint8_t* request, size_t request_len, cliproxy_buffer* response) { + if (stored_host == NULL || stored_host->call == NULL) { + return 1; + } + return stored_host->call(stored_host->host_ctx, method, request, request_len, response); +} + +static void free_host_buffer(void* ptr, size_t len) { + if (stored_host != NULL && stored_host->free_buffer != NULL && ptr != NULL) { + stored_host->free_buffer(ptr, len); + } +} +*/ +import "C" + +import ( + "encoding/json" + "fmt" + "unsafe" +) + +//export cliproxy_plugin_init +func cliproxy_plugin_init(host *C.cliproxy_host_api, plugin *C.cliproxy_plugin_api) C.int { + if plugin == nil { + return 1 + } + C.store_host_api(host) + hostCall = cgoHostCall + plugin.abi_version = C.uint32_t(abiVersion) + plugin.call = C.cliproxy_plugin_call_fn(C.cliproxyPluginCall) + plugin.free_buffer = C.cliproxy_plugin_free_fn(C.cliproxyPluginFree) + plugin.shutdown = C.cliproxy_plugin_shutdown_fn(C.cliproxyPluginShutdown) + return 0 +} + +//export cliproxyPluginCall +func cliproxyPluginCall(method *C.char, request *C.uint8_t, requestLen C.size_t, response *C.cliproxy_buffer) C.int { + if response != nil { + response.ptr = nil + response.len = 0 + } + if method == nil { + writeCResponse(response, errorEnvelope("invalid_method", "method is required")) + return 1 + } + var requestBytes []byte + if request != nil && requestLen > 0 { + requestBytes = C.GoBytes(unsafe.Pointer(request), C.int(requestLen)) + } + raw, errHandle := handleMethod(C.GoString(method), requestBytes) + if errHandle != nil { + writeCResponse(response, errorEnvelope("plugin_error", errHandle.Error())) + return 1 + } + writeCResponse(response, raw) + return 0 +} + +//export cliproxyPluginFree +func cliproxyPluginFree(ptr unsafe.Pointer, _ C.size_t) { + if ptr != nil { + C.free(ptr) + } +} + +//export cliproxyPluginShutdown +func cliproxyPluginShutdown() { + shutdownPlugin() +} + +func writeCResponse(response *C.cliproxy_buffer, raw []byte) { + if response == nil || len(raw) == 0 { + return + } + ptr := C.CBytes(raw) + if ptr == nil { + return + } + response.ptr = ptr + response.len = C.size_t(len(raw)) +} + +func cgoHostCall(method string, payload any) (json.RawMessage, error) { + rawPayload, err := json.Marshal(payload) + if err != nil { + return nil, err + } + cMethod := C.CString(method) + defer C.free(unsafe.Pointer(cMethod)) + var response C.cliproxy_buffer + var requestPtr *C.uint8_t + if len(rawPayload) > 0 { + cPayload := C.CBytes(rawPayload) + if cPayload == nil { + return nil, fmt.Errorf("allocate host callback") + } + defer C.free(cPayload) + requestPtr = (*C.uint8_t)(cPayload) + } + callCode := C.call_host_api(cMethod, requestPtr, C.size_t(len(rawPayload)), &response) + var rawResponse []byte + if response.ptr != nil && response.len > 0 { + rawResponse = C.GoBytes(response.ptr, C.int(response.len)) + } + if response.ptr != nil { + C.free_host_buffer(response.ptr, response.len) + } + if callCode != 0 || len(rawResponse) == 0 { + return nil, fmt.Errorf("host callback %s returned code=%d", method, int(callCode)) + } + var env envelope + if err := json.Unmarshal(rawResponse, &env); err != nil { + return nil, err + } + if !env.OK { + if env.Error != nil { + return nil, fmt.Errorf("%s: %s", env.Error.Code, env.Error.Message) + } + return nil, fmt.Errorf("host callback failed") + } + return env.Result, nil +} diff --git a/cpa_codexcont_executor_plugin/go/plugin_types.go b/cpa_codexcont_executor_plugin/go/plugin_types.go new file mode 100644 index 0000000..43a5b26 --- /dev/null +++ b/cpa_codexcont_executor_plugin/go/plugin_types.go @@ -0,0 +1,202 @@ +package main + +import ( + "net/http" + "net/url" +) + +const ( + abiVersion uint32 = 1 + schemaVersion uint32 = 1 + + methodPluginRegister = "plugin.register" + methodPluginReconfigure = "plugin.reconfigure" + methodFrontendAuthIdentifier = "frontend_auth.identifier" + methodFrontendAuthAuthenticate = "frontend_auth.authenticate" + methodModelRoute = "model.route" + methodExecutorIdentifier = "executor.identifier" + methodExecutorExecute = "executor.execute" + methodExecutorExecuteStream = "executor.execute_stream" + methodExecutorCountTokens = "executor.count_tokens" + methodUsageHandle = "usage.handle" + methodManagementRegister = "management.register" + methodManagementHandle = "management.handle" + methodHostModelExecute = "host.model.execute" + methodHostModelExecuteStream = "host.model.execute_stream" + methodHostModelStreamRead = "host.model.stream_read" + methodHostModelStreamClose = "host.model.stream_close" + methodHostStreamEmit = "host.stream.emit" + methodHostStreamClose = "host.stream.close" + methodHostLog = "host.log" +) + +const ( + configString = "string" + configBoolean = "boolean" + configInteger = "integer" + configEnum = "enum" + + routeTargetSelf = "self" +) + +type configField struct { + Name string `json:"Name"` + Type string `json:"Type"` + EnumValues []string `json:"EnumValues,omitempty"` + Description string `json:"Description"` +} + +type frontendAuthRequest struct { + Method string `json:"Method"` + Path string `json:"Path"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` +} + +type frontendAuthResponse struct { + Authenticated bool `json:"Authenticated"` + Principal string `json:"Principal,omitempty"` + Metadata map[string]string `json:"Metadata,omitempty"` +} + +type modelRouteRequest struct { + SourceFormat string `json:"SourceFormat"` + RequestedModel string `json:"RequestedModel"` + Stream bool `json:"Stream"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` + Metadata map[string]any `json:"Metadata"` +} + +type modelRouteResponse struct { + Handled bool `json:"Handled"` + TargetKind string `json:"TargetKind,omitempty"` + Target string `json:"Target,omitempty"` + TargetModel string `json:"TargetModel,omitempty"` + Reason string `json:"Reason,omitempty"` +} + +type managementRoute struct { + Method string `json:"Method"` + Path string `json:"Path"` + Menu string `json:"Menu,omitempty"` + Description string `json:"Description,omitempty"` +} + +type resourceRoute struct { + Path string `json:"Path"` + Menu string `json:"Menu,omitempty"` + Description string `json:"Description,omitempty"` +} + +type managementRegistrationResponse struct { + Routes []managementRoute `json:"routes,omitempty"` + Resources []resourceRoute `json:"resources,omitempty"` +} + +type managementRequest struct { + Method string `json:"Method"` + Path string `json:"Path"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type managementResponse struct { + StatusCode int `json:"StatusCode"` + Headers http.Header `json:"Headers"` + Body []byte `json:"Body"` +} + +type executorRequest struct { + AuthID string `json:"AuthID"` + AuthProvider string `json:"AuthProvider"` + Model string `json:"Model"` + Format string `json:"Format"` + Stream bool `json:"Stream"` + Alt string `json:"Alt"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + OriginalRequest []byte `json:"OriginalRequest"` + SourceFormat string `json:"SourceFormat"` + Payload []byte `json:"Payload"` + Metadata map[string]any `json:"Metadata"` + StorageJSON []byte `json:"StorageJSON"` + AuthMetadata map[string]any `json:"AuthMetadata"` + AuthAttributes map[string]string `json:"AuthAttributes"` +} + +type executorCallRequest struct { + executorRequest + NestedExecutorRequest executorRequest `json:"ExecutorRequest,omitempty"` + StreamID string `json:"stream_id,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type executorResponse struct { + Payload []byte `json:"Payload"` + Headers http.Header `json:"Headers,omitempty"` +} + +type executorStreamResponse struct { + Headers http.Header `json:"headers,omitempty"` +} + +type hostModelExecutionRequest struct { + EntryProtocol string `json:"entry_protocol"` + ExitProtocol string `json:"exit_protocol"` + Model string `json:"model"` + Stream bool `json:"stream"` + Body []byte `json:"body"` + Headers http.Header `json:"headers"` + Query url.Values `json:"query"` + Alt string `json:"alt,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type hostModelExecutionResponse struct { + StatusCode int `json:"status_code"` + Headers http.Header `json:"headers"` + Body []byte `json:"body"` +} + +type hostModelStreamResponse struct { + StatusCode int `json:"status_code"` + Headers http.Header `json:"headers"` + StreamID string `json:"stream_id"` +} + +type hostModelStreamReadRequest struct { + StreamID string `json:"stream_id"` +} + +type hostModelStreamReadResponse struct { + Payload []byte `json:"payload"` + Error string `json:"error"` + Done bool `json:"done"` +} + +type hostModelStreamCloseRequest struct { + StreamID string `json:"stream_id"` +} + +type hostStreamEmitRequest struct { + StreamID string `json:"stream_id"` + Payload []byte `json:"payload,omitempty"` + Error string `json:"error,omitempty"` +} + +type hostStreamCloseRequest struct { + StreamID string `json:"stream_id"` + Error string `json:"error,omitempty"` +} + +type hostLogRequest struct { + Level string `json:"level,omitempty"` + Message string `json:"message,omitempty"` + Fields map[string]any `json:"fields,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} diff --git a/cpa_governor_plugin/README.md b/cpa_governor_plugin/README.md new file mode 100644 index 0000000..2995263 --- /dev/null +++ b/cpa_governor_plugin/README.md @@ -0,0 +1,97 @@ +# CPA Governor Plugin + +CPA Governor is a self-owned CLIProxyAPI plugin for this repository. It keeps +official CPA, CPAMP, and CPA Key Policy binaries untouched while moving user +quota, usage projection, and CodexCont status into a CPA-native plugin surface. + +## Safety Mode + +The first production rollout is intentionally conservative: + +- `codexcont_enabled: true` means Governor can read CodexCont Engine health and + show status in the plugin UI. +- `codexcont_route: false` keeps real `/v1/responses` execution on the current + known-good production path. +- Turn `codexcont_route` on only after executor-level continuation has been + validated with a test key. Until then, Caddy's current CodexCont sidecar route + remains the fallback. + +## Local Test + +```powershell +cd D:\Dev\20_Software\23_Reference\llm-gateway\CodexCont\cpa_governor_plugin\go +go test ./... +``` + +## Linux Build + +The CPA production host is Linux amd64. Build a shared object with cgo: + +```bash +cd cpa_governor_plugin/go +CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags cliproxy_plugin -buildmode=c-shared -o cpa-governor.so . +``` + +The host discovers plugin files from `plugins.dir` and the platform subdirectory, +for example: + +```text +/CLIProxyAPI/plugins/linux/amd64/cpa-governor.so +``` + +If the Windows host only has a non-linux Go toolchain, build from WSL with a +temporary linux/amd64 Go toolchain instead. Keep the resulting `dist/` artifact +out of git and record its SHA256 in the Trellis task. + +## Minimal CPA Config + +```yaml +plugins: + enabled: true + dir: /CLIProxyAPI/plugins + configs: + cpa-governor: + enabled: true + priority: 20 + exclusive_auth: true + state_db_path: /CLIProxyAPI/plugin-state/cpa-governor/governor.sqlite + key_policy_state_path: /CLIProxyAPI/plugin-state/cpa-key-policy-state.json + session_secret: ${CPA_GOVERNOR_SESSION_SECRET} + codexcont_enabled: true + codexcont_route: false + codexcont_url: http://codexcont:8787 + fail_mode: fallback +``` + +Use `exclusive_auth: true` only when Governor is expected to participate in +frontend auth. Current production key policy has moved to `cpa-key-policy-plus`, +which hard-blocks disabled, disallowed-model, over-quota, and over-RPM user +keys. Governor no longer enforces request concurrency. + +## Production Routes + +The first SJC rollout uses the plugin as the unified UI and usage surface: + +- Admin page: `https://cpa-admin.konbakuyomu.us/governor/` +- User page: `https://cpa-usage.konbakuyomu.us/` + +The public API host must keep plugin/admin paths blocked. The user host should +only expose the Governor user resource and user APIs; the admin resource must +return 404 there. + +## User Key Login + +The user page accepts the full Key Policy `cpa_...` key. Native CPA `sk...` +keys and shortened previews are rejected with explanatory messages because they +are not the quota-managed user identity in this deployment. + +CPA plugin resource routes are GET-only in the current host. The user login +request therefore calls `/user/api/session` with `GET` and passes the key only +through the `X-CPA-Governor-Key` header so embedded CPAMP pages do not confuse +it with the admin shell's own `Authorization` header. Do not put user keys in +query strings. + +The CPAMP sidebar entry named `CPA Governor` and the direct +`https://cpa-admin.konbakuyomu.us/governor/` route are the same admin page. +Prefer the CPAMP sidebar for normal administration; the direct route is a +convenience/debug entrypoint, not a second system. diff --git a/cpa_governor_plugin/go/assets/admin.html b/cpa_governor_plugin/go/assets/admin.html new file mode 100644 index 0000000..2ed2f6c --- /dev/null +++ b/cpa_governor_plugin/go/assets/admin.html @@ -0,0 +1,355 @@ + + + + + + CPA Governor + + + +
+
+
+
G
+
+

CPA Governor

+

CodexCont 实时保护状态

+
+
+
+ 正在连接 + 活跃 0 + +
+
+ +
+ +
+
+

最近请求

+
+ +
+
+
+ + + + + + + + + + + + + + + +
时间用户/Key结果模型耗时命中轮末轮 reasoning续写操作
+
+
+ +
+
+ 高级日志 +
+
+
+
+ + + + diff --git a/cpa_governor_plugin/go/assets/shared.css b/cpa_governor_plugin/go/assets/shared.css new file mode 100644 index 0000000..066a7d3 --- /dev/null +++ b/cpa_governor_plugin/go/assets/shared.css @@ -0,0 +1,502 @@ +:root { + color-scheme: dark; + --bg: #111722; + --panel: #151b27; + --panel-2: #1b2230; + --panel-3: #202838; + --line: #283244; + --line-strong: #39465c; + --text: #e6eaf0; + --muted: #94a3b8; + --blue: #3b82f6; + --blue-soft: rgba(59, 130, 246, .14); + --green: #68d65f; + --green-soft: rgba(104, 214, 95, .12); + --red: #f87171; + --red-soft: rgba(248, 113, 113, .12); + --amber: #f4b942; + --amber-soft: rgba(244, 185, 66, .12); + --teal: #2bb3c5; + --teal-soft: rgba(43, 179, 197, .11); + --shadow: 0 10px 26px rgba(0, 0, 0, .18); + --mono: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace; + --sans: Inter, "Segoe UI", "Microsoft YaHei", system-ui, sans-serif; + font-family: var(--sans); +} +* { box-sizing: border-box; } +body { + margin: 0; + min-height: 100vh; + background: linear-gradient(180deg, #151b26 0%, var(--bg) 46%, #0f1520 100%); + color: var(--text); + font: 14px/1.48 var(--sans); + letter-spacing: 0; +} +button, select, input { + min-height: 38px; + border: 1px solid var(--line); + border-radius: 8px; + background: #111827; + color: var(--text); + padding: 0 12px; + font: inherit; +} +button { + display: inline-flex; + align-items: center; + justify-content: center; + gap: 8px; + cursor: pointer; + white-space: nowrap; + transition: border-color .18s ease, background .18s ease, color .18s ease; +} +button:hover, button.active, select:hover, input:focus { + border-color: var(--line-strong); + outline: none; +} +button:disabled { + cursor: wait; + opacity: .92; +} +button.primary { + border-color: rgba(59, 130, 246, .48); + background: rgba(59, 130, 246, .18); +} +button.ghost { background: #1a2230; } +button.danger-action { + border-color: rgba(248, 113, 113, .42); + background: rgba(248, 113, 113, .12); + color: #ffb8b8; +} +button.danger-action:hover { + border-color: rgba(248, 113, 113, .62); + background: rgba(248, 113, 113, .18); +} +button.compact { + min-height: 28px; + padding: 0 9px; + margin-top: 6px; + font-size: 12px; +} +.sync-button { + position: relative; + min-width: 96px; +} +.sync-button.syncing { + border-color: rgba(59, 130, 246, .5); + background: rgba(59, 130, 246, .14); + color: #c8dcff; +} +.sync-button.just-updated { + border-color: rgba(104, 214, 95, .5); + background: rgba(104, 214, 95, .12); + color: #d3f8cf; +} +.sync-button.sync-error { + border-color: rgba(248, 113, 113, .55); + background: rgba(248, 113, 113, .12); + color: #ffc7c7; +} +.sync-light { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--muted); + box-shadow: 0 0 0 0 rgba(154, 167, 184, .2); + position: relative; + flex: 0 0 auto; + transition: background .18s ease, box-shadow .18s ease; +} +.sync-button.live-ok .sync-light { + background: var(--green); + box-shadow: 0 0 10px rgba(104, 214, 95, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.live-info .sync-light { + background: var(--blue); + box-shadow: 0 0 10px rgba(59, 130, 246, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.live-warn .sync-light { + background: var(--amber); + box-shadow: 0 0 10px rgba(244, 185, 66, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.live-bad .sync-light { + background: var(--red); + box-shadow: 0 0 10px rgba(248, 113, 113, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.syncing .sync-light { + background: var(--teal); + animation: statusBlink 1s ease-in-out infinite; +} +.sync-button.just-updated .sync-light { + background: var(--green); + box-shadow: 0 0 16px rgba(97, 211, 79, .42); +} +.sync-button.sync-error .sync-light { + background: var(--red); + box-shadow: 0 0 16px rgba(255, 100, 109, .38); +} +.shell { + width: min(1680px, calc(100% - 32px)); + margin: 0 auto; + padding: 22px 0 34px; +} +.topbar { + position: relative; + min-height: 58px; + display: flex; + align-items: center; + justify-content: space-between; + gap: 16px; + padding: 0 16px; + margin-bottom: 16px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; + box-shadow: var(--shadow); +} +.brand { + display: flex; + align-items: center; + gap: 12px; + min-width: 0; +} +.brand > div { min-width: 0; } +.mark { + width: 34px; + height: 34px; + display: grid; + place-items: center; + border-radius: 8px; + background: linear-gradient(135deg, #2b8df0, #24b8cf); + color: #fff; + font-weight: 850; +} +h1, h2, h3, p { margin: 0; } +h1 { + font-size: 20px; + line-height: 1.2; + font-weight: 780; +} +h2 { font-size: 15px; font-weight: 760; } +h3 { font-size: 13px; font-weight: 760; } +.subtitle { + margin-top: 2px; + color: var(--muted); + font-size: 12px; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.toolbar, .filters { + display: flex; + align-items: center; + justify-content: flex-end; + gap: 8px; + flex-wrap: wrap; +} +.chip { + display: inline-flex; + align-items: center; + gap: 7px; + min-height: 32px; + padding: 0 10px; + border-radius: 999px; + border: 1px solid var(--line); + background: #1a2230; + color: var(--muted); + font-size: 12px; + font-weight: 720; + line-height: 1.2; + white-space: nowrap; + transition: border-color .18s ease, background .18s ease, color .18s ease, box-shadow .18s ease; +} +.dot { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--muted); + position: relative; + flex: 0 0 auto; +} +.chip.stream .dot { + animation: statusBlink 1.45s ease-in-out infinite; +} +.chip.stream .dot::after { + content: ""; + position: absolute; + inset: -5px; + border-radius: inherit; + border: 1px solid currentColor; + opacity: .55; + animation: statusPing 1.45s ease-out infinite; +} +.chip.ok { border-color: rgba(104, 214, 95, .28); background: var(--green-soft); color: #a9f2a2; } +.chip.ok .dot { background: var(--green); } +.chip.bad { border-color: rgba(248, 113, 113, .3); background: var(--red-soft); color: #ffb8b8; } +.chip.bad .dot { background: var(--red); } +.chip.warn { border-color: rgba(244, 185, 66, .3); background: var(--amber-soft); color: #ffd893; } +.chip.warn .dot { background: var(--amber); } +.chip.info { border-color: rgba(59, 130, 246, .32); background: var(--blue-soft); color: #a9c8ff; } +.chip.info .dot { background: var(--blue); } +.metrics { + display: grid; + grid-template-columns: repeat(4, minmax(150px, 1fr)); + gap: 10px; + margin-bottom: 14px; +} +.metric { + min-height: 86px; + padding: 14px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; + box-shadow: var(--shadow); + transition: border-color .2s ease, background .2s ease; +} +.metric .label { + color: var(--muted); + font-size: 12px; + font-weight: 720; +} +.metric .value { + margin-top: 6px; + font-size: 25px; + line-height: 1; + font-weight: 820; +} +.metric .hint { + margin-top: 7px; + color: var(--muted); + font-size: 12px; +} +.panel { + margin-bottom: 14px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; + box-shadow: var(--shadow); + overflow: hidden; +} +.panel-head { + min-height: 48px; + padding: 12px 14px; + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; + border-bottom: 1px solid var(--line); +} +.panel-body { padding: 14px; } +.embedded-panel { + margin-bottom: 0; + box-shadow: none; +} +.tabs { + display: flex; + gap: 8px; + margin-bottom: 14px; + overflow-x: auto; +} +.tabs button { + position: relative; + overflow: hidden; +} +.tabs button.active::after { + content: ""; + position: absolute; + left: 12px; + right: 12px; + bottom: 4px; + height: 2px; + border-radius: 999px; + background: rgba(59, 130, 246, .72); +} +.tabs button.active { + border-color: rgba(59, 130, 246, .42); + background: rgba(59, 130, 246, .16); + color: #c8dcff; +} +#content { + transition: opacity .18s ease, transform .18s ease; +} +#content.content-refreshing { + opacity: .72; + transform: translate3d(0, 3px, 0); +} +#content.view-enter { + animation: viewRise .26s ease-out both; +} +.table-wrap { overflow-x: auto; } +table { + width: 100%; + min-width: 980px; + border-collapse: collapse; + table-layout: fixed; +} +.realtime-table { + min-width: 1180px; +} +th, td { + border-bottom: 1px solid rgba(40, 52, 72, .75); + padding: 11px 12px; + text-align: left; + vertical-align: middle; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} +tbody tr { + transition: background-color .24s ease, box-shadow .24s ease, transform .2s ease; +} +tbody tr:hover { + background: rgba(148, 163, 184, .055); +} +tbody tr.data-row { + background-clip: padding-box; +} +th { + color: var(--muted); + font-size: 12px; + font-weight: 760; + background: #303442; +} +td strong { display: block; } +.strong { + display: block; + color: var(--text); + font-weight: 780; +} +.success { color: var(--green); } +.danger { color: var(--red); } +small, .muted { + color: var(--muted); + font-size: 12px; +} +.mono { font-family: var(--mono); } +.good { color: var(--green); font-weight: 760; } +.bad-text { color: var(--red); font-weight: 760; } +.blue { color: var(--blue); font-weight: 760; } +.detail-row td { + white-space: normal; + overflow: visible; + background: #111722; +} +.detail-grid { + display: grid; + grid-template-columns: repeat(4, minmax(220px, 1fr)); + gap: 10px; +} +.detail-card { + min-width: 0; + padding: 12px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; +} +.kv { + display: grid; + grid-template-columns: minmax(92px, 42%) 1fr; + gap: 7px 10px; + margin-top: 10px; +} +.kv dt { color: var(--muted); } +.kv dd { + margin: 0; + min-width: 0; + overflow-wrap: anywhere; +} +.detail-note { + margin-top: 10px; + color: var(--muted); + font-size: 12px; +} +.failure-box { + margin-top: 10px; + min-height: 46px; + max-height: 132px; + overflow: auto; + padding: 10px; + border: 1px solid rgba(40, 52, 72, .78); + border-radius: 8px; + background: #101620; + color: #d7deea; + white-space: pre-wrap; + overflow-wrap: anywhere; +} +.empty { + padding: 22px; + text-align: center; + color: var(--muted); +} +.login { + max-width: 720px; + margin: 52px auto; +} +.login-row { + display: flex; + gap: 10px; + margin-top: 14px; +} +.login input { + flex: 1; + min-width: 0; +} +#err { margin-top: 10px; color: #ffb1b6; } +details.advanced > summary { + cursor: pointer; + padding: 12px 14px; + color: var(--muted); + font-weight: 760; +} +.logs { + max-height: 320px; + overflow: auto; + padding: 0 14px 14px; +} +.log-line { + display: grid; + grid-template-columns: 92px 86px 150px 1fr; + gap: 10px; + padding: 8px 0; + border-top: 1px solid rgba(40, 52, 72, .6); + font-family: var(--mono); + font-size: 12px; +} +.hidden { display: none !important; } +@keyframes statusBlink { + 0%, 100% { transform: scale(.9); opacity: .65; } + 50% { transform: scale(1.18); opacity: 1; } +} +@keyframes statusPing { + 0% { transform: scale(.5); opacity: .55; } + 80%, 100% { transform: scale(1.8); opacity: 0; } +} +@keyframes viewRise { + 0% { opacity: .78; transform: translate3d(0, 4px, 0); } + 100% { opacity: 1; transform: translate3d(0, 0, 0); } +} +@media (prefers-reduced-motion: reduce) { + *, *::before, *::after { + animation-duration: .01ms !important; + animation-iteration-count: 1 !important; + scroll-behavior: auto !important; + transition-duration: .01ms !important; + } +} +@media (max-width: 900px) { + .metrics { grid-template-columns: repeat(2, minmax(140px, 1fr)); } + .detail-grid { grid-template-columns: 1fr; } +} +@media (max-width: 620px) { + .shell { width: min(100% - 20px, 1680px); padding-top: 12px; } + .topbar { align-items: flex-start; flex-direction: column; padding: 12px; } + .toolbar, .filters { justify-content: flex-start; } + .metrics { grid-template-columns: 1fr; } + .login-row { flex-direction: column; } + table { min-width: 900px; } + .log-line { grid-template-columns: 1fr; gap: 3px; } +} diff --git a/cpa_governor_plugin/go/assets/user.html b/cpa_governor_plugin/go/assets/user.html new file mode 100644 index 0000000..012aa45 --- /dev/null +++ b/cpa_governor_plugin/go/assets/user.html @@ -0,0 +1,624 @@ + + + + + + CPA 用量自助页 + + + +
+
+
+
U
+
+

CPA 用量自助页

+

单 Key 实时监控、额度明细和思维链保护状态

+
+
+
+ + 未登录 + + + +
+
+ + + + +
+ + + + diff --git a/cpa_governor_plugin/go/go.mod b/cpa_governor_plugin/go/go.mod new file mode 100644 index 0000000..7673945 --- /dev/null +++ b/cpa_governor_plugin/go/go.mod @@ -0,0 +1,24 @@ +module codexcont/cpa-governor-plugin + +go 1.22 + +require ( + gopkg.in/yaml.v3 v3.0.1 + modernc.org/sqlite v1.33.1 +) + +require ( + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/ncruces/go-strftime v0.1.9 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + golang.org/x/sys v0.22.0 // indirect + modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect + modernc.org/libc v1.55.3 // indirect + modernc.org/mathutil v1.6.0 // indirect + modernc.org/memory v1.8.0 // indirect + modernc.org/strutil v1.2.0 // indirect + modernc.org/token v1.1.0 // indirect +) diff --git a/cpa_governor_plugin/go/go.sum b/cpa_governor_plugin/go/go.sum new file mode 100644 index 0000000..6ea7e08 --- /dev/null +++ b/cpa_governor_plugin/go/go.sum @@ -0,0 +1,53 @@ +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= +github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic= +golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI= +golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw= +golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ= +modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ= +modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y= +modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s= +modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE= +modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ= +modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw= +modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4= +modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U= +modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w= +modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4= +modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo= +modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E= +modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU= +modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4= +modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= +modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc= +modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss= +modernc.org/sqlite v1.33.1 h1:trb6Z3YYoeM9eDL1O8do81kP+0ejv+YzgyFo+Gwy0nM= +modernc.org/sqlite v1.33.1/go.mod h1:pXV2xHxhzXZsgT/RtTFAPY6JJDEvOTcTdwADQCCWD4k= +modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA= +modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/cpa_governor_plugin/go/internal/governor/config.go b/cpa_governor_plugin/go/internal/governor/config.go new file mode 100644 index 0000000..7bd3c98 --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/config.go @@ -0,0 +1,54 @@ +package governor + +import ( + "strings" + "time" +) + +type Config struct { + Enabled bool `yaml:"enabled"` + ExclusiveAuth bool `yaml:"exclusive_auth"` + StateDBPath string `yaml:"state_db_path"` + KeyPolicyStatePath string `yaml:"key_policy_state_path"` + SessionSecret string `yaml:"session_secret"` + CodexContEnabled bool `yaml:"codexcont_enabled"` + CodexContRoute bool `yaml:"codexcont_route"` + CodexContURL string `yaml:"codexcont_url"` + FailMode string `yaml:"fail_mode"` + PollIntervalMS int `yaml:"poll_interval_ms"` +} + +func DefaultConfig() Config { + return Config{ + Enabled: true, + ExclusiveAuth: false, + StateDBPath: "cpa-governor.sqlite", + SessionSecret: "change-me", + CodexContEnabled: false, + CodexContURL: "http://codexcont:8787", + FailMode: "fallback", + PollIntervalMS: 1500, + } +} + +func (c Config) Normalize() Config { + if strings.TrimSpace(c.StateDBPath) == "" { + c.StateDBPath = DefaultConfig().StateDBPath + } + c.FailMode = strings.ToLower(strings.TrimSpace(c.FailMode)) + if c.FailMode == "" { + c.FailMode = "fallback" + } + c.CodexContURL = strings.TrimRight(strings.TrimSpace(c.CodexContURL), "/") + if c.CodexContURL == "" { + c.CodexContURL = DefaultConfig().CodexContURL + } + if c.PollIntervalMS <= 0 { + c.PollIntervalMS = 1500 + } + return c +} + +func SessionTTL() time.Duration { + return 24 * time.Hour +} diff --git a/cpa_governor_plugin/go/internal/governor/governor_test.go b/cpa_governor_plugin/go/internal/governor/governor_test.go new file mode 100644 index 0000000..cdc0106 --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/governor_test.go @@ -0,0 +1,306 @@ +package governor + +import ( + "context" + "database/sql" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +func ptr(v float64) *float64 { return &v } + +func writePolicyState(t *testing.T, dir string, rawKey string) string { + t.Helper() + path := filepath.Join(dir, "key-policy.json") + body := map[string]any{ + "keys": []map[string]any{{ + "id": "alice-key", + "name": "Alice", + "key_hash": "sha256:" + SHA256Hex(rawKey), + "enabled": true, + "rpm": 12, + "models": []map[string]any{{ + "alias": "gpt-5.5", + "target_model": "gpt-5.5", + "input_price_per_million": 5, + "output_price_per_million": 30, + "cache_read_price_per_million": 0.5, + }}, + "daily_limit_usd": 5, + "weekly_limit_usd": 30, + }}, + } + raw, err := json.Marshal(body) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, raw, 0o600); err != nil { + t.Fatal(err) + } + return path +} + +func TestKeyPolicyStateParsesSafeRecords(t *testing.T) { + dir := t.TempDir() + path := writePolicyState(t, dir, "cpa_live") + state, err := LoadKeyPolicyState(path) + if err != nil { + t.Fatal(err) + } + key, ok := state.FindByRawKey(" cpa_live ") + if !ok { + t.Fatal("raw key did not match policy state") + } + if key.ID != "alice-key" || key.Name != "Alice" || !key.Enabled { + t.Fatalf("unexpected key: %#v", key) + } + if len(key.Models) != 1 || key.Models[0] != "gpt-5.5" { + t.Fatalf("models = %#v", key.Models) + } + price, ok := PriceForModel(key.Prices, "GPT-5.5") + if !ok || price.InputPerMillion != 5 || price.OutputPerMillion != 30 || price.CacheReadPerMillion != 0.5 { + t.Fatalf("price = %#v ok=%v", price, ok) + } + safe := key.Safe() + encoded, _ := json.Marshal(safe) + if strings.Contains(string(encoded), SHA256Hex("cpa_live")) || strings.Contains(string(encoded), "cpa_live") { + t.Fatalf("safe projection leaked key material: %s", encoded) + } +} + +func TestPricingBreakdownUsesPerMillionAndCachedInput(t *testing.T) { + price := ModelPrice{ + Model: "gpt-5.5", + InputPerMillion: 5, + OutputPerMillion: 30, + CacheReadPerMillion: 0.5, + } + breakdown := CostForUsage(price, TokenUsage{ + InputTokens: 100, + CachedTokens: 20, + OutputTokens: 50, + ReasoningTokens: 30, + TotalTokens: 150, + }, "gpt-5.5") + if got := breakdown.Tokens["billable_uncached_input"]; got != 80 { + t.Fatalf("billable input = %d", got) + } + if got := breakdown.Tokens["visible_output_estimate"]; got != 20 { + t.Fatalf("visible output estimate = %d", got) + } + if breakdown.Costs["total"] <= 0 { + t.Fatalf("total cost should be positive: %#v", breakdown.Costs) + } + if breakdown.Costs["cached_input"] <= 0 { + t.Fatalf("cached input should be charged with cache read price: %#v", breakdown.Costs) + } +} + +func TestStoreUsageWindowsAndSoftReset(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "governor.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + key := KeyRecord{ + ID: "alice-key", + Name: "Alice", + KeyHash: "sha256:" + SHA256Hex("cpa_live"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa_live")), + FiveHourUSD: ptr(1), + MonthlyLimitUSD: ptr(10), + } + if err := store.UpsertKey(ctx, key); err != nil { + t.Fatal(err) + } + now := time.Now().UTC() + if err := store.InsertUsage(ctx, UsageEvent{ + RequestID: "req-old", + KeyID: "alice-key", + RequestedAt: now.Add(-2 * time.Hour), + Cost: 0.5, + }); err != nil { + t.Fatal(err) + } + if err := store.InsertUsage(ctx, UsageEvent{ + RequestID: "req-new", + KeyID: "alice-key", + RequestedAt: now.Add(-30 * time.Minute), + Cost: 0.25, + }); err != nil { + t.Fatal(err) + } + used, err := store.UsageSum(ctx, "alice-key", WindowFor(Range5H, now)) + if err != nil { + t.Fatal(err) + } + if used != 0.75 { + t.Fatalf("used before reset = %v", used) + } + if err := store.Reset(ctx, "alice-key", Range5H, now.Add(-time.Hour)); err != nil { + t.Fatal(err) + } + used, err = store.UsageSum(ctx, "alice-key", WindowFor(Range5H, now)) + if err != nil { + t.Fatal(err) + } + if used != 0.25 { + t.Fatalf("used after reset = %v", used) + } + summary, err := store.UsageSummary(ctx, "alice-key", WindowFor(Range5H, now)) + if err != nil { + t.Fatal(err) + } + if summary.Calls != 1 || summary.TotalCost != 0.25 { + t.Fatalf("summary after reset = %#v", summary) + } +} + +func TestStoreMigratesOldUsageEventsSchema(t *testing.T) { + path := filepath.Join(t.TempDir(), "governor.sqlite") + db, err := sql.Open("sqlite", path) + if err != nil { + t.Fatal(err) + } + _, err = db.Exec(`create table usage_events ( + id integer primary key autoincrement, + request_id text, + key_id text, + key_preview text, + model text, + endpoint text, + requested_at integer not null, + latency_ms integer, + failed integer not null, + failure text, + input_tokens integer, + output_tokens integer, + cached_tokens integer, + cache_read_tokens integer, + cache_creation_tokens integer, + reasoning_tokens integer, + total_tokens integer, + cost real, + cost_breakdown_json text + )`) + if err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`insert into usage_events(request_id, key_id, model, requested_at, failed, cost, cost_breakdown_json) values('old-1', 'alice-key', 'gpt-5.5', ?, 0, 0.1, '{}')`, time.Now().Unix()); err != nil { + t.Fatal(err) + } + if err := db.Close(); err != nil { + t.Fatal(err) + } + store, err := OpenStore(path) + if err != nil { + t.Fatal(err) + } + defer store.Close() + events, err := store.RecentEvents(context.Background(), "alice-key", 10) + if err != nil { + t.Fatal(err) + } + if len(events) != 1 || events[0].RequestID != "old-1" { + t.Fatalf("events = %#v", events) + } + if events[0].RequestedModel != "" || events[0].TTFTMS != 0 || events[0].StatusCode != 0 { + t.Fatalf("old row should read safe zero values: %#v", events[0]) + } +} + +func TestStoreRecentCodexSummariesFiltersByKey(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "governor.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if err := store.SaveCodexSummary(ctx, "req-a", "alice-key", "gpt-5.5", "auto_continued", map[string]any{ + "request_id": "req-a", + "protection": "auto_continued", + }); err != nil { + t.Fatal(err) + } + if err := store.SaveCodexSummary(ctx, "req-b", "bob-key", "gpt-5.5", "protected_clean", map[string]any{ + "request_id": "req-b", + "protection": "protected_clean", + }); err != nil { + t.Fatal(err) + } + alice, err := store.RecentCodexSummaries(ctx, "alice-key", 10) + if err != nil { + t.Fatal(err) + } + if len(alice) != 1 || alice[0].RequestID != "req-a" || alice[0].Protection != "auto_continued" { + t.Fatalf("alice summaries = %#v", alice) + } + all, err := store.RecentCodexSummaries(ctx, "all", 10) + if err != nil { + t.Fatal(err) + } + if len(all) != 2 { + t.Fatalf("all summaries = %#v", all) + } +} + +func TestSecurityAndRedaction(t *testing.T) { + raw := " cpa_live " + hash := SHA256Hex(raw) + if hash != SHA256Hex(strings.TrimSpace(raw)) { + t.Fatal("SHA256Hex should trim raw keys") + } + if hash != SHA256Hex("Bearer cpa_live") || hash != SHA256Hex("Authorization: Bearer cpa_live") { + t.Fatal("SHA256Hex should normalize pasted bearer prefixes") + } + token, err := SignSession(SessionPayload{KeyID: "alice", KeyHash: "sha256:" + hash, ExpiresAt: time.Now().Add(time.Hour).Unix()}, "secret") + if err != nil { + t.Fatal(err) + } + payload, ok := VerifySession(token, "secret", time.Now()) + if !ok || payload.KeyID != "alice" { + t.Fatalf("session verify failed: %#v ok=%v", payload, ok) + } + if _, ok := VerifySession(token, "wrong", time.Now()); ok { + t.Fatal("session verified with wrong secret") + } + brief := Brief("Authorization: Bearer secret and api_key=abc", 200) + if strings.Contains(brief, "secret") || strings.Contains(brief, "abc") { + t.Fatalf("secret leaked in brief: %s", brief) + } +} + +func TestSubmittedKeyHints(t *testing.T) { + cases := []struct { + name string + in string + code string + }{ + {name: "missing", in: " ", code: "missing_api_key"}, + {name: "native", in: "sk-abc", code: "native_cpa_key_not_supported"}, + {name: "preview", in: "cpa_abcd...efgh", code: "key_preview_not_usable"}, + {name: "unsupported", in: "abc", code: "unsupported_key_format"}, + {name: "short cpa", in: "Bearer cpa_live", code: "key_preview_not_usable"}, + {name: "full cpa", in: "Bearer cpa_abcdefghijklmnopqrstuvwxyz0123456789", code: "invalid_api_key"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + hint := ExplainUnmatchedSubmittedKey(tc.in) + if hint.Error != tc.code || hint.Message == "" { + t.Fatalf("hint = %#v", hint) + } + }) + } + if got := NormalizeSubmittedKey("Authorization: Bearer Bearer cpa_live "); got != "cpa_live" { + t.Fatalf("normalized key = %q", got) + } + if got := NormalizeSubmittedKey("\ufeff“Bearer cpa_live\u200b”"); got != "cpa_live" { + t.Fatalf("normalized decorated key = %q", got) + } +} diff --git a/cpa_governor_plugin/go/internal/governor/models.go b/cpa_governor_plugin/go/internal/governor/models.go new file mode 100644 index 0000000..858dfeb --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/models.go @@ -0,0 +1,384 @@ +package governor + +import ( + "encoding/json" + "fmt" + "os" + "strings" +) + +type ModelPrice struct { + Model string `json:"model"` + TargetModel string `json:"target_model,omitempty"` + Provider string `json:"provider,omitempty"` + InputPerMillion float64 `json:"input_per_million"` + OutputPerMillion float64 `json:"output_per_million"` + CacheReadPerMillion float64 `json:"cache_read_per_million"` + CacheCreationPerMillion float64 `json:"cache_creation_per_million"` +} + +type KeyRecord struct { + ID string `json:"id"` + Name string `json:"name"` + KeyHash string `json:"key_hash"` + Enabled bool `json:"enabled"` + Preview string `json:"preview"` + RPM int `json:"rpm,omitempty"` + Concurrency int `json:"concurrency,omitempty"` + Models []string `json:"models"` + Prices map[string]ModelPrice `json:"prices,omitempty"` + DailyLimitUSD *float64 `json:"daily_limit_usd,omitempty"` + WeeklyLimitUSD *float64 `json:"weekly_limit_usd,omitempty"` + FiveHourUSD *float64 `json:"five_hour_usd,omitempty"` + MonthlyLimitUSD *float64 `json:"monthly_limit_usd,omitempty"` +} + +func (k KeyRecord) Safe() map[string]any { + return map[string]any{ + "id": k.ID, + "name": k.Name, + "enabled": k.Enabled, + "preview": k.Preview, + "rpm": k.RPM, + "concurrency": k.Concurrency, + "models": append([]string(nil), k.Models...), + "limits": map[string]any{ + "five_hour_usd": k.FiveHourUSD, + "daily_usd": k.DailyLimitUSD, + "weekly_usd": k.WeeklyLimitUSD, + "monthly_usd": k.MonthlyLimitUSD, + }, + "pricing": map[string]any{ + "models": k.Prices, + }, + } +} + +type KeyPolicyState struct { + Keys []KeyRecord +} + +func LoadKeyPolicyState(path string) (KeyPolicyState, error) { + raw, err := os.ReadFile(path) + if err != nil { + return KeyPolicyState{}, err + } + var data any + if err := json.Unmarshal(raw, &data); err != nil { + return KeyPolicyState{}, err + } + keys := extractKeys(data) + out := make([]KeyRecord, 0, len(keys)) + for _, rawKey := range keys { + if key, ok := parseKey(rawKey); ok { + out = append(out, key) + } + } + return KeyPolicyState{Keys: out}, nil +} + +func (s KeyPolicyState) FindByRawKey(rawKey string) (KeyRecord, bool) { + hash := SHA256Hex(rawKey) + return s.FindByRawHash(hash) +} + +func (s KeyPolicyState) FindByRawHash(hash string) (KeyRecord, bool) { + normalized, err := NormalizeHash(hash) + if err != nil { + return KeyRecord{}, false + } + for _, key := range s.Keys { + keyHash, err := NormalizeHash(key.KeyHash) + if err == nil && keyHash == normalized { + return key, true + } + } + return KeyRecord{}, false +} + +func extractKeys(data any) []map[string]any { + if arr, ok := data.([]any); ok { + return mapsFromArray(arr) + } + obj, ok := data.(map[string]any) + if !ok { + return nil + } + for _, path := range [][]string{{"keys"}, {"state", "keys"}, {"data", "keys"}, {"config", "keys"}} { + var cur any = obj + for _, part := range path { + m, ok := cur.(map[string]any) + if !ok { + cur = nil + break + } + cur = m[part] + } + if arr, ok := cur.([]any); ok { + return mapsFromArray(arr) + } + } + return nil +} + +func mapsFromArray(arr []any) []map[string]any { + out := make([]map[string]any, 0, len(arr)) + for _, item := range arr { + if m, ok := item.(map[string]any); ok { + out = append(out, m) + } + } + return out +} + +func parseKey(raw map[string]any) (KeyRecord, bool) { + rawHash := firstString(raw, "key_hash", "keyHash", "hash", "api_key_hash", "apiKeyHash") + if rawHash == "" { + return KeyRecord{}, false + } + normalized, err := NormalizeHash(rawHash) + if err != nil { + return KeyRecord{}, false + } + id := firstString(raw, "id", "key_id", "keyId") + if id == "" { + id = HashPreview(normalized) + } + name := firstString(raw, "name", "label", "alias", "description") + if name == "" { + name = id + } + enabled, hasEnabled := firstBool(raw, "enabled", "is_enabled", "isEnabled") + disabled, _ := firstBool(raw, "disabled", "is_disabled", "isDisabled") + if !hasEnabled { + enabled = !disabled + } + modelItems := asList(firstAny(raw, "models", "allowed_models", "allowedModels", "model_allowlist", "modelAllowlist", "aliases")) + models := parseModels(modelItems) + prices := parsePrices(raw, modelItems) + return KeyRecord{ + ID: strings.TrimSpace(id), + Name: strings.TrimSpace(name), + KeyHash: "sha256:" + normalized, + Enabled: enabled && !disabled, + Preview: firstNonEmpty(firstString(raw, "preview", "key_preview", "keyPreview"), HashPreview(normalized)), + RPM: firstInt(raw, "rpm", "rpm_limit", "rpmLimit", "rpm_per_minute", "rpmPerMinute"), + Concurrency: firstInt(raw, "concurrency", "concurrency_limit", "concurrencyLimit", "max_concurrent", "maxConcurrent"), + Models: models, + Prices: prices, + DailyLimitUSD: firstFloatPtr(raw, "daily_limit_usd", "dailyLimitUsd", "daily_limit", "dailyLimit", "daily_usd", "dailyUsd"), + WeeklyLimitUSD: firstFloatPtr(raw, "weekly_limit_usd", "weeklyLimitUsd", "weekly_limit", "weeklyLimit", "weekly_usd", "weeklyUsd"), + FiveHourUSD: firstFloatPtr(raw, "five_hour_limit_usd", "fiveHourLimitUsd", "five_hour_usd", "fiveHourUsd", "5h_limit_usd"), + MonthlyLimitUSD: firstFloatPtr(raw, "monthly_limit_usd", "monthlyLimitUsd", "monthly_usd", "monthlyUsd", "month_limit_usd"), + }, true +} + +func parseModels(items []any) []string { + seen := map[string]bool{} + var out []string + for _, item := range items { + name := "" + if m, ok := item.(map[string]any); ok { + name = modelName(m) + } else { + name = strings.TrimSpace(toString(item)) + } + if name != "" && !seen[name] { + seen[name] = true + out = append(out, name) + } + } + return out +} + +func parsePrices(raw map[string]any, modelItems []any) map[string]ModelPrice { + out := map[string]ModelPrice{} + for _, item := range modelItems { + if m, ok := item.(map[string]any); ok { + if p, ok := parsePriceEntry(m, ""); ok { + out[p.Model] = p + } + } + } + priceRaw := firstAny(raw, "model_prices", "modelPrices", "prices") + switch v := priceRaw.(type) { + case map[string]any: + for name, item := range v { + if m, ok := item.(map[string]any); ok { + if p, ok := parsePriceEntry(m, name); ok { + out[p.Model] = p + } + } + } + case []any: + for _, item := range v { + if m, ok := item.(map[string]any); ok { + if p, ok := parsePriceEntry(m, ""); ok { + out[p.Model] = p + } + } + } + } + return out +} + +func parsePriceEntry(raw map[string]any, defaultModel string) (ModelPrice, bool) { + model := modelName(raw) + if model == "" { + model = strings.TrimSpace(defaultModel) + } + if model == "" { + return ModelPrice{}, false + } + price := ModelPrice{ + Model: model, + TargetModel: firstString(raw, "target_model", "targetModel", "upstream_model", "upstreamModel"), + Provider: firstString(raw, "provider", "type"), + InputPerMillion: firstFloat(raw, "input_price_per_million", "inputPricePerMillion", "input", "prompt", "prompt_price_per_million"), + OutputPerMillion: firstFloat(raw, "output_price_per_million", "outputPricePerMillion", "output", "completion", "completion_price_per_million"), + CacheReadPerMillion: firstFloat(raw, "cache_read_price_per_million", "cacheReadPricePerMillion", "cache_price_per_million", "cachePricePerMillion", "cache_read", "cacheRead", "cache"), + CacheCreationPerMillion: firstFloat(raw, "cache_creation_price_per_million", "cacheCreationPricePerMillion", "cache_write_price_per_million", "cacheWritePricePerMillion", "cache_creation", "cacheCreation", "cache_write", "cacheWrite"), + } + if price.InputPerMillion <= 0 && price.OutputPerMillion <= 0 && price.CacheReadPerMillion <= 0 && price.CacheCreationPerMillion <= 0 { + return ModelPrice{}, false + } + return price, true +} + +func modelName(raw map[string]any) string { + return firstString(raw, "alias", "model", "name", "id", "target_model", "targetModel", "upstream_model", "upstreamModel") +} + +func firstAny(raw map[string]any, names ...string) any { + for _, name := range names { + if value, ok := raw[name]; ok { + return value + } + } + return nil +} + +func firstString(raw map[string]any, names ...string) string { + for _, name := range names { + if value, ok := raw[name]; ok { + text := strings.TrimSpace(toString(value)) + if text != "" { + return text + } + } + } + return "" +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func firstBool(raw map[string]any, names ...string) (bool, bool) { + for _, name := range names { + if value, ok := raw[name]; ok { + switch v := value.(type) { + case bool: + return v, true + case string: + switch strings.ToLower(strings.TrimSpace(v)) { + case "true", "1", "yes", "enabled": + return true, true + case "false", "0", "no", "disabled": + return false, true + } + } + } + } + return false, false +} + +func firstInt(raw map[string]any, names ...string) int { + for _, name := range names { + if value, ok := raw[name]; ok { + switch v := value.(type) { + case float64: + return int(v) + case int: + return v + case string: + var n int + if err := json.NewDecoder(strings.NewReader(v)).Decode(&n); err == nil { + return n + } + } + } + } + return 0 +} + +func firstFloat(raw map[string]any, names ...string) float64 { + ptr := firstFloatPtr(raw, names...) + if ptr == nil { + return 0 + } + return *ptr +} + +func firstFloatPtr(raw map[string]any, names ...string) *float64 { + for _, name := range names { + if value, ok := raw[name]; ok { + switch v := value.(type) { + case float64: + return &v + case int: + f := float64(v) + return &f + case string: + var f float64 + if err := json.NewDecoder(strings.NewReader(v)).Decode(&f); err == nil { + return &f + } + } + } + } + return nil +} + +func asList(value any) []any { + switch v := value.(type) { + case []any: + return v + case []string: + out := make([]any, len(v)) + for i := range v { + out[i] = v[i] + } + return out + case string: + parts := strings.Split(v, ",") + out := make([]any, 0, len(parts)) + for _, part := range parts { + if text := strings.TrimSpace(part); text != "" { + out = append(out, text) + } + } + return out + default: + return nil + } +} + +func toString(value any) string { + switch v := value.(type) { + case string: + return v + case json.Number: + return v.String() + default: + if value == nil { + return "" + } + return strings.TrimSpace(fmt.Sprint(value)) + } +} diff --git a/cpa_governor_plugin/go/internal/governor/pricing.go b/cpa_governor_plugin/go/internal/governor/pricing.go new file mode 100644 index 0000000..06d5942 --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/pricing.go @@ -0,0 +1,102 @@ +package governor + +import "strings" + +const perMillion = 1_000_000.0 + +type TokenUsage struct { + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CachedTokens int64 `json:"cached_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheCreationTokens int64 `json:"cache_creation_tokens"` + ReasoningTokens int64 `json:"reasoning_tokens"` + TotalTokens int64 `json:"total_tokens"` +} + +type CostBreakdown struct { + Source string `json:"source"` + Model string `json:"model"` + Prices ModelPrice `json:"prices"` + Tokens map[string]int64 `json:"tokens"` + Costs map[string]float64 `json:"costs"` +} + +func PriceForModel(prices map[string]ModelPrice, model string) (ModelPrice, bool) { + model = strings.TrimSpace(model) + if model == "" { + return ModelPrice{}, false + } + if price, ok := prices[model]; ok { + return price, true + } + lower := strings.ToLower(model) + for name, price := range prices { + if strings.ToLower(name) == lower { + return price, true + } + } + return ModelPrice{}, false +} + +func CostForUsage(price ModelPrice, usage TokenUsage, model string) CostBreakdown { + input := max64(usage.InputTokens, 0) + output := max64(usage.OutputTokens, 0) + cached := max64(usage.CachedTokens, 0) + cacheRead := max64(usage.CacheReadTokens, 0) + cacheCreation := max64(usage.CacheCreationTokens, 0) + reasoning := max64(usage.ReasoningTokens, 0) + billableInput := max64(input-cached, 0) + cacheReadPrice := price.CacheReadPerMillion + if cacheReadPrice <= 0 { + cacheReadPrice = price.InputPerMillion + } + cacheCreationPrice := price.CacheCreationPerMillion + if cacheCreationPrice <= 0 { + cacheCreationPrice = price.InputPerMillion + } + inputCost := float64(billableInput) * price.InputPerMillion / perMillion + cachedCost := float64(cached) * cacheReadPrice / perMillion + cacheReadCost := float64(cacheRead) * cacheReadPrice / perMillion + cacheCreationCost := float64(cacheCreation) * cacheCreationPrice / perMillion + outputCost := float64(output) * price.OutputPerMillion / perMillion + total := inputCost + cachedCost + cacheReadCost + cacheCreationCost + outputCost + totalTokens := usage.TotalTokens + if totalTokens <= 0 { + totalTokens = input + output + } + if strings.TrimSpace(model) == "" { + model = price.Model + } + return CostBreakdown{ + Source: "governor_price_book", + Model: model, + Prices: price, + Tokens: map[string]int64{ + "input": input, + "billable_uncached_input": billableInput, + "cached_input": cached, + "cache_read": cacheRead, + "cache_creation": cacheCreation, + "output": output, + "reasoning": reasoning, + "visible_output_estimate": max64(output-reasoning, 0), + "total": totalTokens, + }, + Costs: map[string]float64{ + "input": inputCost, + "cached_input": cachedCost, + "cache_read": cacheReadCost, + "cache_creation": cacheCreationCost, + "output": outputCost, + "total": total, + }, + } +} + +func max64(a, b int64) int64 { + if a > b { + return a + } + return b +} diff --git a/cpa_governor_plugin/go/internal/governor/quota.go b/cpa_governor_plugin/go/internal/governor/quota.go new file mode 100644 index 0000000..2c8f5b4 --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/quota.go @@ -0,0 +1,66 @@ +package governor + +import ( + "strings" + "time" +) + +const ( + Range5H = "5h" + Range24H = "24h" + Range7D = "7d" + RangeMonth = "month" +) + +type Window struct { + Name string `json:"name"` + From time.Time `json:"from"` + To time.Time `json:"to"` +} + +func WindowFor(rangeName string, now time.Time) Window { + now = now.UTC() + switch strings.ToLower(strings.TrimSpace(rangeName)) { + case Range5H: + return Window{Name: Range5H, From: now.Add(-5 * time.Hour), To: now} + case Range7D: + return Window{Name: Range7D, From: now.Add(-7 * 24 * time.Hour), To: now} + case RangeMonth: + loc := time.FixedZone("Asia/Shanghai", 8*60*60) + local := now.In(loc) + start := time.Date(local.Year(), local.Month(), 1, 0, 0, 0, 0, loc) + return Window{Name: RangeMonth, From: start.UTC(), To: now} + default: + return Window{Name: Range24H, From: now.Add(-24 * time.Hour), To: now} + } +} + +type QuotaDecision struct { + Allowed bool `json:"allowed"` + Reason string `json:"reason,omitempty"` + UsedUSD float64 `json:"used_usd"` + LimitUSD *float64 `json:"limit_usd,omitempty"` +} + +func CheckLimit(used float64, limit *float64) QuotaDecision { + if limit == nil || *limit <= 0 { + return QuotaDecision{Allowed: true, UsedUSD: used} + } + if used >= *limit { + return QuotaDecision{Allowed: false, Reason: "quota_exceeded", UsedUSD: used, LimitUSD: limit} + } + return QuotaDecision{Allowed: true, UsedUSD: used, LimitUSD: limit} +} + +func ModelAllowed(allowed []string, model string) bool { + if len(allowed) == 0 { + return true + } + model = strings.TrimSpace(model) + for _, item := range allowed { + if strings.EqualFold(strings.TrimSpace(item), model) { + return true + } + } + return false +} diff --git a/cpa_governor_plugin/go/internal/governor/redaction.go b/cpa_governor_plugin/go/internal/governor/redaction.go new file mode 100644 index 0000000..ac5608f --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/redaction.go @@ -0,0 +1,35 @@ +package governor + +import ( + "regexp" + "strings" +) + +var ( + bearerRe = regexp.MustCompile(`(?i)\bBearer\s+[A-Za-z0-9._~+/=-]+`) + secretRe = regexp.MustCompile(`(?i)\b(api[_-]?key|access[_-]?token|refresh[_-]?token|authorization|cookie|secret)\s*[:=]\s*['"]?[^'"\s,;]+`) + spaceRe = regexp.MustCompile(`\s+`) +) + +func RedactString(value string) string { + value = bearerRe.ReplaceAllString(value, "Bearer [REDACTED]") + value = secretRe.ReplaceAllStringFunc(value, func(match string) string { + parts := strings.FieldsFunc(match, func(r rune) bool { return r == ':' || r == '=' }) + if len(parts) == 0 { + return "[REDACTED]" + } + return strings.TrimSpace(parts[0]) + "=[REDACTED]" + }) + return value +} + +func Brief(value string, limit int) string { + value = spaceRe.ReplaceAllString(strings.TrimSpace(RedactString(value)), " ") + if limit <= 0 || len(value) <= limit { + return value + } + if limit <= 3 { + return value[:limit] + } + return strings.TrimSpace(value[:limit-3]) + "..." +} diff --git a/cpa_governor_plugin/go/internal/governor/security.go b/cpa_governor_plugin/go/internal/governor/security.go new file mode 100644 index 0000000..318e486 --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/security.go @@ -0,0 +1,131 @@ +package governor + +import ( + "crypto/hmac" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/json" + "errors" + "regexp" + "strings" + "time" + "unicode" +) + +func SHA256Hex(value string) string { + sum := sha256.Sum256([]byte(NormalizeSubmittedKey(value))) + return hex.EncodeToString(sum[:]) +} + +var bearerPrefixPattern = regexp.MustCompile(`(?i)^\s*(authorization\s*:\s*)?(bearer\s+)+`) + +// NormalizeSubmittedKey accepts the common clipboard shapes users paste into +// the self-service portal, while keeping hashing deterministic. +func NormalizeSubmittedKey(value string) string { + text := strings.TrimSpace(value) + text = strings.Map(func(r rune) rune { + if unicode.Is(unicode.Cf, r) { + return -1 + } + return r + }, text) + text = strings.TrimSpace(strings.Trim(text, `"'`+"`"+`“”‘’「」『』<>`)) + text = bearerPrefixPattern.ReplaceAllString(text, "") + return strings.TrimSpace(strings.Trim(text, `"'`+"`"+`“”‘’「」『』<>`)) +} + +type SubmittedKeyHint struct { + Error string + Message string +} + +func ExplainUnmatchedSubmittedKey(value string) SubmittedKeyHint { + key := NormalizeSubmittedKey(value) + lower := strings.ToLower(key) + switch { + case key == "": + return SubmittedKeyHint{Error: "missing_api_key", Message: "请粘贴完整的 cpa_ 开头用户 Key。"} + case strings.HasPrefix(lower, "sk-") || strings.HasPrefix(lower, "sk_"): + return SubmittedKeyHint{Error: "native_cpa_key_not_supported", Message: "这是 CPA 原生 sk Key，不能登录用量自助页。请使用 Key Policy 创建时弹窗里的完整 cpa_ 用户 Key。"} + case strings.Contains(key, "...") || strings.Contains(key, "…"): + return SubmittedKeyHint{Error: "key_preview_not_usable", Message: "你粘贴的是缩略预览，不是完整 Key。Key Policy 创建或轮换时弹窗里的完整 cpa_ Key 才能登录。"} + case !strings.HasPrefix(lower, "cpa_"): + return SubmittedKeyHint{Error: "unsupported_key_format", Message: "用量自助页只接受 Key Policy 的完整 cpa_ 用户 Key。"} + case len(key) < 40: + return SubmittedKeyHint{Error: "key_preview_not_usable", Message: "这个 cpa_ Key 太短，像是列表里的预览，不是完整 Key。请在 Key Policy 里点击“轮换”，复制弹窗中新生成的完整 Key。"} + default: + return SubmittedKeyHint{Error: "invalid_api_key", Message: "这个 cpa_ Key 没有匹配到当前 Key Policy 记录。请确认粘贴的是创建或轮换弹窗里的完整 Key；列表里的 cpa_xxx...xxx 只是预览，旧 Key 关闭弹窗后无法找回，需要在 Key Policy 里轮换生成新的完整 Key。"} + } +} + +func NormalizeHash(value string) (string, error) { + text := strings.TrimSpace(value) + text = strings.TrimPrefix(text, "sha256:") + if len(text) != 64 { + return "", errors.New("invalid hash length") + } + _, err := hex.DecodeString(text) + if err != nil { + return "", err + } + return strings.ToLower(text), nil +} + +func HashPreview(value string) string { + normalized, err := NormalizeHash(value) + if err != nil { + normalized = SHA256Hex(value) + } + if len(normalized) <= 16 { + return normalized + } + return normalized[:8] + "..." + normalized[len(normalized)-6:] +} + +type SessionPayload struct { + KeyID string `json:"key_id"` + KeyHash string `json:"key_hash"` + ExpiresAt int64 `json:"expires_at"` +} + +func SignSession(payload SessionPayload, secret string) (string, error) { + if strings.TrimSpace(secret) == "" { + return "", errors.New("session secret is required") + } + raw, err := json.Marshal(payload) + if err != nil { + return "", err + } + body := base64.RawURLEncoding.EncodeToString(raw) + mac := hmac.New(sha256.New, []byte(secret)) + _, _ = mac.Write([]byte(body)) + sig := base64.RawURLEncoding.EncodeToString(mac.Sum(nil)) + return body + "." + sig, nil +} + +func VerifySession(token, secret string, now time.Time) (SessionPayload, bool) { + parts := strings.Split(token, ".") + if len(parts) != 2 || strings.TrimSpace(secret) == "" { + return SessionPayload{}, false + } + mac := hmac.New(sha256.New, []byte(secret)) + _, _ = mac.Write([]byte(parts[0])) + want := mac.Sum(nil) + got, err := base64.RawURLEncoding.DecodeString(parts[1]) + if err != nil || !hmac.Equal(got, want) { + return SessionPayload{}, false + } + raw, err := base64.RawURLEncoding.DecodeString(parts[0]) + if err != nil { + return SessionPayload{}, false + } + var payload SessionPayload + if err := json.Unmarshal(raw, &payload); err != nil { + return SessionPayload{}, false + } + if payload.ExpiresAt > 0 && now.Unix() > payload.ExpiresAt { + return SessionPayload{}, false + } + return payload, true +} diff --git a/cpa_governor_plugin/go/internal/governor/store.go b/cpa_governor_plugin/go/internal/governor/store.go new file mode 100644 index 0000000..f8d5b15 --- /dev/null +++ b/cpa_governor_plugin/go/internal/governor/store.go @@ -0,0 +1,648 @@ +package governor + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "os" + "path/filepath" + "time" + + _ "modernc.org/sqlite" +) + +type Store struct { + db *sql.DB +} + +type UsageEvent struct { + RequestID string `json:"request_id"` + KeyID string `json:"key_id"` + KeyPreview string `json:"key_preview"` + Model string `json:"model"` + RequestedModel string `json:"requested_model,omitempty"` + ActualModel string `json:"actual_model,omitempty"` + Provider string `json:"provider,omitempty"` + ExecutorType string `json:"executor_type,omitempty"` + Endpoint string `json:"endpoint,omitempty"` + RequestedAt time.Time `json:"requested_at"` + LatencyMS int64 `json:"latency_ms"` + TTFTMS int64 `json:"ttft_ms,omitempty"` + ReasoningEffort string `json:"reasoning_effort,omitempty"` + ServiceTier string `json:"service_tier,omitempty"` + StatusCode int `json:"status_code,omitempty"` + Failed bool `json:"failed"` + Failure string `json:"failure,omitempty"` + Usage TokenUsage `json:"usage"` + Cost float64 `json:"cost"` + CostBreakdown CostBreakdown `json:"cost_breakdown"` +} + +type UsageSummary struct { + Calls int64 `json:"calls"` + Failed int64 `json:"failed"` + TotalCost float64 `json:"total_cost"` + Usage TokenUsage `json:"usage"` +} + +type CodexSummary struct { + RequestID string `json:"request_id"` + KeyID string `json:"key_id,omitempty"` + Model string `json:"model,omitempty"` + Protection string `json:"protection,omitempty"` + Summary map[string]any `json:"summary"` + UpdatedAt time.Time `json:"updated_at"` +} + +func OpenStore(path string) (*Store, error) { + if path == "" { + path = "cpa-governor.sqlite" + } + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil && filepath.Dir(path) != "." { + return nil, err + } + db, err := sql.Open("sqlite", path) + if err != nil { + return nil, err + } + store := &Store{db: db} + if err := store.EnsureSchema(context.Background()); err != nil { + _ = db.Close() + return nil, err + } + return store, nil +} + +func (s *Store) Close() error { + if s == nil || s.db == nil { + return nil + } + return s.db.Close() +} + +func (s *Store) EnsureSchema(ctx context.Context) error { + stmts := []string{ + `pragma journal_mode=wal`, + `create table if not exists keys ( + id text primary key, + name text not null, + key_hash text not null unique, + enabled integer not null, + preview text, + rpm integer, + concurrency integer, + models_json text, + prices_json text, + five_hour_limit_usd real, + daily_limit_usd real, + weekly_limit_usd real, + monthly_limit_usd real, + updated_at integer not null + )`, + `create table if not exists reset_watermarks ( + key_id text not null, + window text not null, + reset_at integer not null, + primary key(key_id, window) + )`, + `create table if not exists usage_events ( + id integer primary key autoincrement, + request_id text, + key_id text, + key_preview text, + model text, + requested_model text, + actual_model text, + provider text, + executor_type text, + endpoint text, + requested_at integer not null, + latency_ms integer, + ttft_ms integer, + reasoning_effort text, + service_tier text, + status_code integer, + failed integer not null, + failure text, + input_tokens integer, + output_tokens integer, + cached_tokens integer, + cache_read_tokens integer, + cache_creation_tokens integer, + reasoning_tokens integer, + total_tokens integer, + cost real, + cost_breakdown_json text + )`, + `create table if not exists codexcont_summaries ( + request_id text primary key, + key_id text, + model text, + protection text, + summary_json text, + updated_at integer not null + )`, + `create table if not exists audit_log ( + id integer primary key autoincrement, + timestamp integer not null, + actor text, + action text not null, + target text, + detail_json text + )`, + `create table if not exists settings ( + key text primary key, + value text not null, + updated_at integer not null + )`, + } + for _, stmt := range stmts { + if _, err := s.db.ExecContext(ctx, stmt); err != nil { + return err + } + } + if err := s.ensureColumns(ctx, "usage_events", map[string]string{ + "requested_model": "text default ''", + "actual_model": "text default ''", + "provider": "text default ''", + "executor_type": "text default ''", + "ttft_ms": "integer default 0", + "reasoning_effort": "text default ''", + "service_tier": "text default ''", + "status_code": "integer default 0", + }); err != nil { + return err + } + return nil +} + +func (s *Store) ensureColumns(ctx context.Context, table string, columns map[string]string) error { + rows, err := s.db.QueryContext(ctx, fmt.Sprintf(`pragma table_info(%s)`, table)) + if err != nil { + return err + } + defer rows.Close() + existing := map[string]bool{} + for rows.Next() { + var cid int + var name, typ string + var notNull int + var defaultValue sql.NullString + var pk int + if err := rows.Scan(&cid, &name, &typ, ¬Null, &defaultValue, &pk); err != nil { + return err + } + existing[name] = true + } + if err := rows.Err(); err != nil { + return err + } + for name, typ := range columns { + if existing[name] { + continue + } + if _, err := s.db.ExecContext(ctx, fmt.Sprintf(`alter table %s add column %s %s`, table, name, typ)); err != nil { + return err + } + } + return nil +} + +func (s *Store) UpsertKey(ctx context.Context, key KeyRecord) error { + models, _ := json.Marshal(key.Models) + prices, _ := json.Marshal(key.Prices) + _, err := s.db.ExecContext( + ctx, + `insert into keys( + id, name, key_hash, enabled, preview, rpm, concurrency, models_json, prices_json, + five_hour_limit_usd, daily_limit_usd, weekly_limit_usd, monthly_limit_usd, updated_at + ) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + on conflict(id) do update set + name=excluded.name, + key_hash=excluded.key_hash, + enabled=excluded.enabled, + preview=excluded.preview, + rpm=excluded.rpm, + concurrency=excluded.concurrency, + models_json=excluded.models_json, + prices_json=excluded.prices_json, + five_hour_limit_usd=coalesce(keys.five_hour_limit_usd, excluded.five_hour_limit_usd), + daily_limit_usd=excluded.daily_limit_usd, + weekly_limit_usd=excluded.weekly_limit_usd, + monthly_limit_usd=coalesce(keys.monthly_limit_usd, excluded.monthly_limit_usd), + updated_at=excluded.updated_at`, + key.ID, + key.Name, + key.KeyHash, + boolInt(key.Enabled), + key.Preview, + key.RPM, + key.Concurrency, + string(models), + string(prices), + key.FiveHourUSD, + key.DailyLimitUSD, + key.WeeklyLimitUSD, + key.MonthlyLimitUSD, + time.Now().Unix(), + ) + return err +} + +func (s *Store) ImportKeys(ctx context.Context, state KeyPolicyState) error { + seen := make(map[string]bool, len(state.Keys)) + for _, key := range state.Keys { + seen[key.ID] = true + if err := s.UpsertKey(ctx, key); err != nil { + return err + } + } + if err := s.syncDeletedKeys(ctx, seen); err != nil { + return err + } + return nil +} + +func (s *Store) syncDeletedKeys(ctx context.Context, seen map[string]bool) error { + rows, err := s.db.QueryContext(ctx, `select id from keys`) + if err != nil { + return err + } + var stale []string + for rows.Next() { + var id string + if err := rows.Scan(&id); err != nil { + _ = rows.Close() + return err + } + if !seen[id] { + stale = append(stale, id) + } + } + if err := rows.Close(); err != nil { + return err + } + for _, id := range stale { + if _, err := s.db.ExecContext(ctx, `delete from keys where id=?`, id); err != nil { + return err + } + } + return nil +} + +func (s *Store) ListKeys(ctx context.Context) ([]KeyRecord, error) { + rows, err := s.db.QueryContext(ctx, `select id, name, key_hash, enabled, preview, rpm, concurrency, models_json, prices_json, + five_hour_limit_usd, daily_limit_usd, weekly_limit_usd, monthly_limit_usd from keys order by name collate nocase`) + if err != nil { + return nil, err + } + defer rows.Close() + var out []KeyRecord + for rows.Next() { + var key KeyRecord + var enabled int + var modelsJSON, pricesJSON string + var fiveHour, daily, weekly, monthly sql.NullFloat64 + if err := rows.Scan( + &key.ID, &key.Name, &key.KeyHash, &enabled, &key.Preview, &key.RPM, &key.Concurrency, + &modelsJSON, &pricesJSON, &fiveHour, &daily, &weekly, &monthly, + ); err != nil { + return nil, err + } + key.Enabled = enabled != 0 + key.FiveHourUSD = nullFloatPtr(fiveHour) + key.DailyLimitUSD = nullFloatPtr(daily) + key.WeeklyLimitUSD = nullFloatPtr(weekly) + key.MonthlyLimitUSD = nullFloatPtr(monthly) + _ = json.Unmarshal([]byte(modelsJSON), &key.Models) + _ = json.Unmarshal([]byte(pricesJSON), &key.Prices) + out = append(out, key) + } + return out, rows.Err() +} + +func (s *Store) FindKeyByHash(ctx context.Context, hash string) (KeyRecord, bool, error) { + keys, err := s.ListKeys(ctx) + if err != nil { + return KeyRecord{}, false, err + } + normalized, err := NormalizeHash(hash) + if err != nil { + return KeyRecord{}, false, nil + } + for _, key := range keys { + keyHash, err := NormalizeHash(key.KeyHash) + if err == nil && keyHash == normalized { + return key, true, nil + } + } + return KeyRecord{}, false, nil +} + +func (s *Store) SetLimits(ctx context.Context, id string, fiveHour, monthly *float64) error { + res, err := s.db.ExecContext(ctx, `update keys set five_hour_limit_usd=?, monthly_limit_usd=?, updated_at=? where id=?`, fiveHour, monthly, time.Now().Unix(), id) + if err != nil { + return err + } + if affected, _ := res.RowsAffected(); affected == 0 { + return fmt.Errorf("unknown key: %s", id) + } + return s.Audit(ctx, "admin", "set_limits", id, map[string]any{"five_hour_usd": fiveHour, "monthly_usd": monthly}) +} + +func (s *Store) Reset(ctx context.Context, id, window string, at time.Time) error { + _, err := s.db.ExecContext(ctx, `insert into reset_watermarks(key_id, window, reset_at) values(?, ?, ?) + on conflict(key_id, window) do update set reset_at=excluded.reset_at`, id, window, at.Unix()) + if err != nil { + return err + } + return s.Audit(ctx, "admin", "reset_usage", id, map[string]any{"window": window, "reset_at": at.Unix()}) +} + +func (s *Store) InsertUsage(ctx context.Context, event UsageEvent) error { + if event.RequestedAt.IsZero() { + event.RequestedAt = time.Now() + } + breakdown, _ := json.Marshal(event.CostBreakdown) + _, err := s.db.ExecContext( + ctx, + `insert into usage_events( + request_id, key_id, key_preview, model, requested_model, actual_model, provider, executor_type, + endpoint, requested_at, latency_ms, ttft_ms, reasoning_effort, service_tier, status_code, failed, failure, + input_tokens, output_tokens, cached_tokens, cache_read_tokens, cache_creation_tokens, + reasoning_tokens, total_tokens, cost, cost_breakdown_json + ) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + event.RequestID, event.KeyID, event.KeyPreview, event.Model, event.RequestedModel, event.ActualModel, + event.Provider, event.ExecutorType, event.Endpoint, event.RequestedAt.Unix(), + event.LatencyMS, event.TTFTMS, event.ReasoningEffort, event.ServiceTier, event.StatusCode, + boolInt(event.Failed), event.Failure, + event.Usage.InputTokens, event.Usage.OutputTokens, event.Usage.CachedTokens, event.Usage.CacheReadTokens, + event.Usage.CacheCreationTokens, event.Usage.ReasoningTokens, event.Usage.TotalTokens, + event.Cost, string(breakdown), + ) + return err +} + +func (s *Store) UsageSum(ctx context.Context, keyID string, window Window) (float64, error) { + var total sql.NullFloat64 + from := window.From.Unix() + if resetAt, ok := s.ResetAt(ctx, keyID, window.Name); ok && resetAt > from { + from = resetAt + } + args := []any{from, window.To.Unix()} + query := `select coalesce(sum(cost), 0) from usage_events where requested_at >= ? and requested_at <= ?` + if keyID != "" && keyID != "all" { + query += ` and key_id = ?` + args = append(args, keyID) + } + if err := s.db.QueryRowContext(ctx, query, args...).Scan(&total); err != nil { + return 0, err + } + if !total.Valid { + return 0, nil + } + return total.Float64, nil +} + +func (s *Store) ResetAt(ctx context.Context, keyID, window string) (int64, bool) { + if s == nil || s.db == nil || keyID == "" || window == "" || keyID == "all" { + return 0, false + } + var resetAt sql.NullInt64 + if err := s.db.QueryRowContext(ctx, `select reset_at from reset_watermarks where key_id=? and window=?`, keyID, window).Scan(&resetAt); err != nil { + return 0, false + } + return resetAt.Int64, resetAt.Valid +} + +func (s *Store) UsageSummary(ctx context.Context, keyID string, window Window) (UsageSummary, error) { + from := window.From.Unix() + if resetAt, ok := s.ResetAt(ctx, keyID, window.Name); ok && resetAt > from { + from = resetAt + } + args := []any{from, window.To.Unix()} + query := `select + count(*), + coalesce(sum(case when failed != 0 then 1 else 0 end), 0), + coalesce(sum(cost), 0), + coalesce(sum(input_tokens), 0), + coalesce(sum(output_tokens), 0), + coalesce(sum(cached_tokens), 0), + coalesce(sum(cache_read_tokens), 0), + coalesce(sum(cache_creation_tokens), 0), + coalesce(sum(reasoning_tokens), 0), + coalesce(sum(total_tokens), 0) + from usage_events where requested_at >= ? and requested_at <= ?` + if keyID != "" && keyID != "all" { + query += ` and key_id = ?` + args = append(args, keyID) + } + var summary UsageSummary + if err := s.db.QueryRowContext(ctx, query, args...).Scan( + &summary.Calls, + &summary.Failed, + &summary.TotalCost, + &summary.Usage.InputTokens, + &summary.Usage.OutputTokens, + &summary.Usage.CachedTokens, + &summary.Usage.CacheReadTokens, + &summary.Usage.CacheCreationTokens, + &summary.Usage.ReasoningTokens, + &summary.Usage.TotalTokens, + ); err != nil { + return UsageSummary{}, err + } + return summary, nil +} + +func (s *Store) RecentEvents(ctx context.Context, keyID string, limit int) ([]UsageEvent, error) { + if limit <= 0 || limit > 200 { + limit = 100 + } + query := `select coalesce(request_id, ''), coalesce(key_id, ''), coalesce(key_preview, ''), coalesce(model, ''), + coalesce(requested_model, ''), coalesce(actual_model, ''), coalesce(provider, ''), coalesce(executor_type, ''), + coalesce(endpoint, ''), requested_at, coalesce(latency_ms, 0), coalesce(ttft_ms, 0), + coalesce(reasoning_effort, ''), coalesce(service_tier, ''), coalesce(status_code, 0), coalesce(failed, 0), coalesce(failure, ''), + coalesce(input_tokens, 0), coalesce(output_tokens, 0), coalesce(cached_tokens, 0), coalesce(cache_read_tokens, 0), + coalesce(cache_creation_tokens, 0), coalesce(reasoning_tokens, 0), coalesce(total_tokens, 0), coalesce(cost, 0), coalesce(cost_breakdown_json, '{}') + from usage_events` + args := []any{} + if keyID != "" && keyID != "all" { + query += ` where key_id = ?` + args = append(args, keyID) + } + query += ` order by requested_at desc, id desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []UsageEvent + for rows.Next() { + var event UsageEvent + var ts int64 + var failed int + var breakdown string + if err := rows.Scan( + &event.RequestID, &event.KeyID, &event.KeyPreview, &event.Model, &event.RequestedModel, &event.ActualModel, + &event.Provider, &event.ExecutorType, &event.Endpoint, &ts, &event.LatencyMS, &event.TTFTMS, + &event.ReasoningEffort, &event.ServiceTier, &event.StatusCode, &failed, &event.Failure, + &event.Usage.InputTokens, &event.Usage.OutputTokens, &event.Usage.CachedTokens, &event.Usage.CacheReadTokens, + &event.Usage.CacheCreationTokens, &event.Usage.ReasoningTokens, &event.Usage.TotalTokens, &event.Cost, &breakdown, + ); err != nil { + return nil, err + } + event.RequestedAt = time.Unix(ts, 0) + event.Failed = failed != 0 + _ = json.Unmarshal([]byte(breakdown), &event.CostBreakdown) + out = append(out, event) + } + return out, rows.Err() +} + +func (s *Store) RecentEventsWindow(ctx context.Context, keyID string, window Window, limit int) ([]UsageEvent, error) { + if limit <= 0 || limit > 200 { + limit = 100 + } + from := window.From.Unix() + if resetAt, ok := s.ResetAt(ctx, keyID, window.Name); ok && resetAt > from { + from = resetAt + } + query := `select coalesce(request_id, ''), coalesce(key_id, ''), coalesce(key_preview, ''), coalesce(model, ''), + coalesce(requested_model, ''), coalesce(actual_model, ''), coalesce(provider, ''), coalesce(executor_type, ''), + coalesce(endpoint, ''), requested_at, coalesce(latency_ms, 0), coalesce(ttft_ms, 0), + coalesce(reasoning_effort, ''), coalesce(service_tier, ''), coalesce(status_code, 0), coalesce(failed, 0), coalesce(failure, ''), + coalesce(input_tokens, 0), coalesce(output_tokens, 0), coalesce(cached_tokens, 0), coalesce(cache_read_tokens, 0), + coalesce(cache_creation_tokens, 0), coalesce(reasoning_tokens, 0), coalesce(total_tokens, 0), coalesce(cost, 0), coalesce(cost_breakdown_json, '{}') + from usage_events where requested_at >= ? and requested_at <= ?` + args := []any{from, window.To.Unix()} + if keyID != "" && keyID != "all" { + query += ` and key_id = ?` + args = append(args, keyID) + } + query += ` order by requested_at desc, id desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []UsageEvent + for rows.Next() { + var event UsageEvent + var ts int64 + var failed int + var breakdown string + if err := rows.Scan( + &event.RequestID, &event.KeyID, &event.KeyPreview, &event.Model, &event.RequestedModel, &event.ActualModel, + &event.Provider, &event.ExecutorType, &event.Endpoint, &ts, &event.LatencyMS, &event.TTFTMS, + &event.ReasoningEffort, &event.ServiceTier, &event.StatusCode, &failed, &event.Failure, + &event.Usage.InputTokens, &event.Usage.OutputTokens, &event.Usage.CachedTokens, &event.Usage.CacheReadTokens, + &event.Usage.CacheCreationTokens, &event.Usage.ReasoningTokens, &event.Usage.TotalTokens, &event.Cost, &breakdown, + ); err != nil { + return nil, err + } + event.RequestedAt = time.Unix(ts, 0) + event.Failed = failed != 0 + _ = json.Unmarshal([]byte(breakdown), &event.CostBreakdown) + out = append(out, event) + } + return out, rows.Err() +} + +func (s *Store) SaveCodexSummary(ctx context.Context, requestID, keyID, model, protection string, summary any) error { + raw, _ := json.Marshal(summary) + _, err := s.db.ExecContext(ctx, `insert into codexcont_summaries(request_id, key_id, model, protection, summary_json, updated_at) + values(?, ?, ?, ?, ?, ?) + on conflict(request_id) do update set key_id=excluded.key_id, model=excluded.model, + protection=excluded.protection, summary_json=excluded.summary_json, updated_at=excluded.updated_at`, + requestID, keyID, model, protection, string(raw), time.Now().Unix()) + return err +} + +func (s *Store) RecentCodexSummaries(ctx context.Context, keyID string, limit int) ([]CodexSummary, error) { + if limit <= 0 || limit > 200 { + limit = 100 + } + query := `select request_id, key_id, model, protection, summary_json, updated_at from codexcont_summaries` + args := []any{} + if keyID != "" && keyID != "all" { + query += ` where key_id = ?` + args = append(args, keyID) + } + query += ` order by updated_at desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []CodexSummary + for rows.Next() { + var item CodexSummary + var raw string + var ts int64 + if err := rows.Scan(&item.RequestID, &item.KeyID, &item.Model, &item.Protection, &raw, &ts); err != nil { + return nil, err + } + item.UpdatedAt = time.Unix(ts, 0) + _ = json.Unmarshal([]byte(raw), &item.Summary) + if item.Summary == nil { + item.Summary = map[string]any{} + } + out = append(out, item) + } + return out, rows.Err() +} + +func (s *Store) Audit(ctx context.Context, actor, action, target string, detail any) error { + raw, _ := json.Marshal(detail) + _, err := s.db.ExecContext(ctx, `insert into audit_log(timestamp, actor, action, target, detail_json) values(?, ?, ?, ?, ?)`, + time.Now().Unix(), actor, action, target, string(raw)) + return err +} + +func (s *Store) LoadSettings(ctx context.Context) (map[string]string, error) { + rows, err := s.db.QueryContext(ctx, `select key, value from settings`) + if err != nil { + return nil, err + } + defer rows.Close() + out := map[string]string{} + for rows.Next() { + var key, value string + if err := rows.Scan(&key, &value); err != nil { + return nil, err + } + out[key] = value + } + return out, rows.Err() +} + +func (s *Store) SaveSettings(ctx context.Context, values map[string]string) error { + for key, value := range values { + if _, err := s.db.ExecContext(ctx, `insert into settings(key, value, updated_at) values(?, ?, ?) + on conflict(key) do update set value=excluded.value, updated_at=excluded.updated_at`, + key, value, time.Now().Unix()); err != nil { + return err + } + } + return s.Audit(ctx, "admin", "set_settings", "governor", values) +} + +func boolInt(value bool) int { + if value { + return 1 + } + return 0 +} + +func nullFloatPtr(value sql.NullFloat64) *float64 { + if !value.Valid { + return nil + } + v := value.Float64 + return &v +} diff --git a/cpa_governor_plugin/go/main.go b/cpa_governor_plugin/go/main.go new file mode 100644 index 0000000..ab1f54f --- /dev/null +++ b/cpa_governor_plugin/go/main.go @@ -0,0 +1,1609 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/url" + "os" + "sort" + "strconv" + "strings" + "sync" + "time" + + "codexcont/cpa-governor-plugin/internal/governor" + _ "embed" + "gopkg.in/yaml.v3" +) + +const pluginID = "cpa-governor" + +//go:embed assets/admin.html +var adminHTMLTemplate string + +//go:embed assets/user.html +var userHTMLTemplate string + +//go:embed assets/shared.css +var sharedCSSTemplate string + +type envelope struct { + OK bool `json:"ok"` + Result json.RawMessage `json:"result,omitempty"` + Error *envelopeError `json:"error,omitempty"` +} + +type envelopeError struct { + Code string `json:"code"` + Message string `json:"message"` +} + +type lifecycleRequest struct { + ConfigYAML []byte `json:"config_yaml"` +} + +type registration struct { + SchemaVersion uint32 `json:"schema_version"` + Metadata metadata `json:"metadata"` + Capabilities capabilities `json:"capabilities"` +} + +type metadata struct { + Name string `json:"Name"` + Version string `json:"Version"` + Author string `json:"Author"` + GitHubRepository string `json:"GitHubRepository"` + ConfigFields []configField `json:"ConfigFields"` +} + +type capabilities struct { + FrontendAuthProvider bool `json:"frontend_auth_provider"` + FrontendAuthProviderExclusive bool `json:"frontend_auth_provider_exclusive"` + ModelRouter bool `json:"model_router"` + Executor bool `json:"executor"` + ExecutorModelScope string `json:"executor_model_scope"` + ExecutorInputFormats []string `json:"executor_input_formats"` + ExecutorOutputFormats []string `json:"executor_output_formats"` + UsagePlugin bool `json:"usage_plugin"` + ManagementAPI bool `json:"management_api"` +} + +type runtimeState struct { + mu sync.RWMutex + cfg governor.Config + store *governor.Store + keyState governor.KeyPolicyState + keyStatePath string + keyStateModTime time.Time + keyStateLastCheck time.Time + rpmBuckets map[string][]time.Time +} + +var state = runtimeState{ + rpmBuckets: map[string][]time.Time{}, +} + +func main() { runPreviewIfRequested() } + +func runPreviewIfRequested() { + addr := strings.TrimSpace(os.Getenv("CPA_GOVERNOR_PREVIEW_ADDR")) + if addr == "" { + return + } + cfg := governor.DefaultConfig() + previewDir, err := os.MkdirTemp("", "cpa-governor-preview-*") + if err != nil { + panic(err) + } + cfg.StateDBPath = previewDir + "/governor.sqlite" + cfg.SessionSecret = "preview-secret" + cfg.CodexContEnabled = true + cfg.CodexContURL = "http://" + addr + store, err := governor.OpenStore(cfg.StateDBPath) + if err != nil { + panic(err) + } + previewRawKey := "cpa_preview_abcdefghijklmnopqrstuvwxyz0123456789AB" + previewKey := governor.KeyRecord{ + ID: "preview-key", + Name: "演示用户", + KeyHash: "sha256:" + governor.SHA256Hex(previewRawKey), + Enabled: true, + Preview: governor.HashPreview(governor.SHA256Hex(previewRawKey)), + RPM: 60, + Concurrency: 2, + Models: []string{"gpt-5.5", "gpt-5.4"}, + FiveHourUSD: floatPtr(2), + DailyLimitUSD: floatPtr(8), + WeeklyLimitUSD: floatPtr(40), + MonthlyLimitUSD: floatPtr(120), + Prices: map[string]governor.ModelPrice{ + "gpt-5.5": {Model: "gpt-5.5", InputPerMillion: 5, OutputPerMillion: 30, CacheReadPerMillion: 0.5}, + }, + } + _ = store.UpsertKey(context.Background(), previewKey) + _ = store.InsertUsage(context.Background(), governor.UsageEvent{ + RequestID: "req-preview-a", + KeyID: previewKey.ID, + KeyPreview: previewKey.Preview, + Model: "gpt-5.5", + RequestedModel: "gpt-5.5", + ActualModel: "gpt-5.5", + Provider: "openai", + ExecutorType: "codex", + Endpoint: "/v1/responses", + RequestedAt: time.Now().Add(-3 * time.Minute), + LatencyMS: 14320, + TTFTMS: 1180, + ReasoningEffort: "high", + ServiceTier: "default", + StatusCode: 200, + Usage: governor.TokenUsage{ + InputTokens: 120000, + CachedTokens: 103000, + OutputTokens: 2100, + ReasoningTokens: 516, + TotalTokens: 122100, + }, + Cost: 0.151, + CostBreakdown: governor.CostForUsage(previewKey.Prices["gpt-5.5"], governor.TokenUsage{ + InputTokens: 120000, + CachedTokens: 103000, + OutputTokens: 2100, + ReasoningTokens: 516, + TotalTokens: 122100, + }, "gpt-5.5"), + }) + _ = store.SaveCodexSummary(context.Background(), "req-preview-a", previewKey.ID, "gpt-5.5", "auto_continued", map[string]any{ + "request_id": "req-preview-a", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": time.Now().Add(-2 * time.Minute).Format(time.RFC3339), + "updated_at": time.Now().Add(-90 * time.Second).Format(time.RFC3339), + "duration_ms": 5570, + "status": "completed", + "protection": "auto_continued", + "key_identity": previewKey.Safe(), + "latest_round": 2, + "latest_reasoning_tokens": 181, + "first_truncation_round": 1, + "first_truncation_reasoning_tokens": 516, + "first_truncation_decision": "continue", + "continuation_count": 1, + "stopped_reason": "completed", + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 516, "decision": "continue", "truncation_match": true}, {"round": 2, "reasoning_tokens": 181, "decision": "clean", "truncation_match": false}}, + }) + state.mu.Lock() + state.cfg = cfg + state.store = store + state.keyState = governor.KeyPolicyState{Keys: []governor.KeyRecord{previewKey}} + state.rpmBuckets = map[string][]time.Time{} + state.mu.Unlock() + mux := http.NewServeMux() + mux.HandleFunc("/v0/resource/plugins/cpa-governor/admin", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "text/html; charset=utf-8") + _, _ = w.Write([]byte(adminHTML())) + }) + mux.HandleFunc("/v0/resource/plugins/cpa-governor/user", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "text/html; charset=utf-8") + _, _ = w.Write([]byte(userHTML())) + }) + writePreviewStatus := func(w http.ResponseWriter) { + w.Header().Set("content-type", "application/json; charset=utf-8") + _ = json.NewEncoder(w).Encode(map[string]any{ + "ok": true, + "counters": map[string]any{ + "total_requests": 4, + "active_requests": 17, + "continuations": 1, + "truncation_hits": 1, + "failures": 0, + }, + "last_error": nil, + }) + } + writePreviewRequests := func(w http.ResponseWriter) { + w.Header().Set("content-type", "application/json; charset=utf-8") + now := time.Now().UTC() + identity := previewKey.Safe() + _ = json.NewEncoder(w).Encode(map[string]any{"requests": []map[string]any{ + { + "request_id": "req-preview-a", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": now.Add(-4 * time.Minute).Format(time.RFC3339), + "updated_at": now.Add(-3 * time.Minute).Format(time.RFC3339), + "duration_ms": 5570, + "status": "completed", + "protection": "auto_continued", + "key_identity": identity, + "latest_round": 2, + "latest_reasoning_tokens": 181, + "first_truncation_round": 1, + "first_truncation_reasoning_tokens": 516, + "continuation_count": 1, + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 516, "decision": "continue", "truncation_match": true}, {"round": 2, "reasoning_tokens": 181, "decision": "clean", "truncation_match": false}}, + }, + { + "request_id": "req-preview-stale", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": now.Add(-45 * time.Minute).Format(time.RFC3339), + "updated_at": now.Add(-44 * time.Minute).Format(time.RFC3339), + "status": "processing", + "protection": "processing", + "key_identity": identity, + "latest_round": 1, + "latest_reasoning_tokens": 140, + "continuation_count": 0, + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 140, "decision": "clean", "truncation_match": false}}, + }, + { + "request_id": "req-preview-live", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": now.Add(-30 * time.Second).Format(time.RFC3339), + "updated_at": now.Add(-5 * time.Second).Format(time.RFC3339), + "status": "processing", + "protection": "processing", + "key_identity": identity, + "latest_round": 1, + "latest_reasoning_tokens": 140, + "continuation_count": 0, + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 140, "decision": "clean", "truncation_match": false}}, + }, + }}) + } + mux.HandleFunc("/governor/codexcont/admin/status", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewStatus(w) + }) + mux.HandleFunc("/admin/status", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewStatus(w) + }) + mux.HandleFunc("/governor/codexcont/admin/requests", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewRequests(w) + }) + mux.HandleFunc("/admin/requests", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewRequests(w) + }) + mux.HandleFunc("/governor/codexcont/admin/logs/stream", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "text/event-stream") + w.Header().Set("cache-control", "no-cache") + _, _ = w.Write([]byte("event: ready\ndata: {\"ok\":true}\n\n")) + _, _ = w.Write([]byte("event: log\ndata: {\"ts\":\"2026-07-02T02:09:59Z\",\"level\":\"info\",\"event\":\"round_decision\",\"message\":\"preview round decision\",\"fields\":{\"request_id\":\"req-preview-a\"}}\n\n")) + }) + mux.HandleFunc("/engine/healthz", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "application/json; charset=utf-8") + _, _ = w.Write([]byte(`{"ok":true,"mode":"preview"}`)) + }) + mux.HandleFunc("/favicon.ico", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.WriteHeader(http.StatusNoContent) + }) + mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + rawReq, _ := json.Marshal(managementRequest{ + Method: r.Method, + Path: r.URL.Path, + Headers: r.Header, + Query: r.URL.Query(), + }) + rawResp, _ := managementHandle(rawReq) + var env envelope + _ = json.Unmarshal(rawResp, &env) + var resp managementResponse + _ = json.Unmarshal(env.Result, &resp) + for key, values := range resp.Headers { + for _, value := range values { + w.Header().Add(key, value) + } + } + if resp.StatusCode != 0 { + w.WriteHeader(resp.StatusCode) + } + _, _ = w.Write(resp.Body) + }) + if err := http.ListenAndServe(addr, mux); err != nil { + panic(err) + } +} + +func floatPtr(value float64) *float64 { return &value } + +func shutdownPlugin() { + state.mu.Lock() + defer state.mu.Unlock() + if state.store != nil { + _ = state.store.Close() + state.store = nil + } +} + +func handleMethod(method string, request []byte) ([]byte, error) { + switch method { + case methodPluginRegister, methodPluginReconfigure: + if err := configure(request); err != nil { + return nil, err + } + return okEnvelope(pluginRegistration()) + case methodFrontendAuthIdentifier: + return okEnvelope(map[string]string{"identifier": pluginID}) + case methodFrontendAuthAuthenticate: + return frontendAuth(request) + case methodModelRoute: + return routeModel(request) + case methodExecutorIdentifier: + return okEnvelope(map[string]string{"identifier": pluginID}) + case methodExecutorExecute: + return executorExecute(request) + case methodExecutorExecuteStream: + return executorExecuteStream(request) + case methodExecutorCountTokens: + return okEnvelope(executorResponse{Payload: []byte(`{"input_tokens":0}`)}) + case methodUsageHandle: + return usageHandle(request) + case methodManagementRegister: + return managementRegister() + case methodManagementHandle: + return managementHandle(request) + default: + return errorEnvelope("unknown_method", "unknown method: "+method), nil + } +} + +func configure(raw []byte) error { + cfg := governor.DefaultConfig() + if len(raw) > 0 { + var req lifecycleRequest + if err := json.Unmarshal(raw, &req); err != nil { + return err + } + if len(req.ConfigYAML) > 0 { + if err := yaml.Unmarshal(req.ConfigYAML, &cfg); err != nil { + return err + } + } + } + cfg = cfg.Normalize() + store, err := governor.OpenStore(cfg.StateDBPath) + if err != nil { + return err + } + cfg = applyStoredSettings(cfg, store) + var keyState governor.KeyPolicyState + if strings.TrimSpace(cfg.KeyPolicyStatePath) != "" { + loaded, err := governor.LoadKeyPolicyState(cfg.KeyPolicyStatePath) + if err == nil { + keyState = loaded + _ = store.ImportKeys(context.Background(), loaded) + } + } + state.mu.Lock() + old := state.store + state.cfg = cfg + state.store = store + state.keyState = keyState + state.keyStatePath = strings.TrimSpace(cfg.KeyPolicyStatePath) + state.keyStateModTime = time.Time{} + state.keyStateLastCheck = time.Time{} + state.mu.Unlock() + if old != nil { + _ = old.Close() + } + _ = refreshKeyPolicyState(true) + return nil +} + +func applyStoredSettings(cfg governor.Config, store *governor.Store) governor.Config { + if store == nil { + return cfg + } + settings, err := store.LoadSettings(context.Background()) + if err != nil { + return cfg + } + if value, ok := settings["codexcont_enabled"]; ok { + if parsed, err := strconv.ParseBool(value); err == nil { + cfg.CodexContEnabled = parsed + } + } + if value := strings.TrimSpace(settings["codexcont_url"]); value != "" { + cfg.CodexContURL = strings.TrimRight(value, "/") + } + if value := strings.TrimSpace(settings["fail_mode"]); value != "" { + cfg.FailMode = strings.ToLower(value) + } + return cfg.Normalize() +} + +func pluginRegistration() registration { + cfg := loadedConfig() + return registration{ + SchemaVersion: schemaVersion, + Metadata: metadata{ + Name: "cpa-governor", + Version: "0.1.0", + Author: "konbakuyomu/CodexCont", + GitHubRepository: "https://local/CodexCont", + ConfigFields: []configField{ + {Name: "enabled", Type: configBoolean, Description: "Enable Governor key authentication and management surfaces."}, + {Name: "exclusive_auth", Type: configBoolean, Description: "When true, Governor frontend auth is exclusive."}, + {Name: "state_db_path", Type: configString, Description: "SQLite path for Governor state."}, + {Name: "key_policy_state_path", Type: configString, Description: "Optional CPA Key Policy state JSON path to import/mirror."}, + {Name: "session_secret", Type: configString, Description: "Secret used to sign user portal sessions."}, + {Name: "codexcont_enabled", Type: configBoolean, Description: "Enable CodexCont engine status integration."}, + {Name: "codexcont_route", Type: configBoolean, Description: "Route protected Responses requests to Governor executor. Keep false until cutover validation."}, + {Name: "codexcont_url", Type: configString, Description: "Internal CodexCont engine base URL."}, + {Name: "fail_mode", Type: configEnum, EnumValues: []string{"fallback", "fail_closed"}, Description: "Behavior when engine is unavailable."}, + }, + }, + Capabilities: capabilities{ + FrontendAuthProvider: true, + FrontendAuthProviderExclusive: cfg.ExclusiveAuth, + ModelRouter: true, + Executor: true, + ExecutorModelScope: "static", + ExecutorInputFormats: []string{"openai", "responses"}, + ExecutorOutputFormats: []string{"openai", "responses"}, + UsagePlugin: true, + ManagementAPI: true, + }, + } +} + +func loadedConfig() governor.Config { + state.mu.RLock() + defer state.mu.RUnlock() + if state.cfg.StateDBPath == "" { + return governor.DefaultConfig() + } + return state.cfg +} + +func loadedStore() *governor.Store { + state.mu.RLock() + defer state.mu.RUnlock() + return state.store +} + +func frontendAuth(raw []byte) ([]byte, error) { + var req frontendAuthRequest + if err := json.Unmarshal(raw, &req); err != nil { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + cfg := loadedConfig() + if !cfg.Enabled { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + key := bearer(req.Headers.Get("Authorization")) + if key == "" { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + record, ok := findKeyByRaw(key) + if !ok || !record.Enabled { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + model := requestedModelFromBody(req.Body) + if model != "" && !governor.ModelAllowed(record.Models, model) { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + if !allowRPM(record) { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + if !allowQuota(record) { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + return okEnvelope(frontendAuthResponse{ + Authenticated: true, + Principal: record.ID, + Metadata: map[string]string{ + "provider": "cpa-governor", + "key_id": record.ID, + "key_name": record.Name, + "preview": record.Preview, + }, + }) +} + +func findKeyByRaw(rawKey string) (governor.KeyRecord, bool) { + _ = refreshKeyPolicyState(false) + if key, ok := lookupKeyByRaw(rawKey); ok { + return key, true + } + if strings.HasPrefix(strings.ToLower(governor.NormalizeSubmittedKey(rawKey)), "cpa_") { + _ = refreshKeyPolicyState(true) + return lookupKeyByRaw(rawKey) + } + return governor.KeyRecord{}, false +} + +func lookupKeyByRaw(rawKey string) (governor.KeyRecord, bool) { + state.mu.RLock() + keyState := state.keyState + store := state.store + state.mu.RUnlock() + if key, ok := keyState.FindByRawKey(rawKey); ok { + return key, true + } + if store != nil { + key, ok, err := store.FindKeyByHash(context.Background(), governor.SHA256Hex(rawKey)) + if err == nil && ok { + return key, true + } + } + return governor.KeyRecord{}, false +} + +func refreshKeyPolicyState(force bool) error { + state.mu.RLock() + path := strings.TrimSpace(state.keyStatePath) + lastMod := state.keyStateModTime + store := state.store + state.mu.RUnlock() + if path == "" || store == nil { + return nil + } + now := time.Now() + info, err := os.Stat(path) + if err != nil { + state.mu.Lock() + state.keyStateLastCheck = now + state.mu.Unlock() + return err + } + if !force && !info.ModTime().After(lastMod) { + state.mu.Lock() + state.keyStateLastCheck = now + state.mu.Unlock() + return nil + } + loaded, err := governor.LoadKeyPolicyState(path) + if err != nil { + state.mu.Lock() + state.keyStateLastCheck = now + state.mu.Unlock() + return err + } + if err := store.ImportKeys(context.Background(), loaded); err != nil { + return err + } + state.mu.Lock() + state.keyState = loaded + state.keyStateModTime = info.ModTime() + state.keyStateLastCheck = now + state.mu.Unlock() + return nil +} + +func routeModel(raw []byte) ([]byte, error) { + var req modelRouteRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + cfg := loadedConfig() + if !cfg.Enabled || !cfg.CodexContRoute { + return okEnvelope(modelRouteResponse{Handled: false}) + } + if !isResponsesRequest(req.SourceFormat, req.Body) { + return okEnvelope(modelRouteResponse{Handled: false}) + } + return okEnvelope(modelRouteResponse{ + Handled: true, + TargetKind: routeTargetSelf, + Reason: "cpa_governor_codexcont_engine_enabled", + }) +} + +func isResponsesRequest(source string, body []byte) bool { + source = strings.ToLower(strings.TrimSpace(source)) + if strings.Contains(source, "response") || source == "openai" { + return true + } + return strings.Contains(string(body), `"stream"`) && strings.Contains(string(body), `"model"`) +} + +func requestedModelFromBody(body []byte) string { + if len(body) == 0 { + return "" + } + var raw map[string]any + if err := json.Unmarshal(body, &raw); err != nil { + return "" + } + if text, ok := raw["model"].(string); ok { + return strings.TrimSpace(text) + } + return "" +} + +func allowRPM(key governor.KeyRecord) bool { + if key.RPM <= 0 { + return true + } + state.mu.Lock() + defer state.mu.Unlock() + now := time.Now() + cutoff := now.Add(-time.Minute) + bucket := state.rpmBuckets[key.ID] + kept := bucket[:0] + for _, ts := range bucket { + if ts.After(cutoff) { + kept = append(kept, ts) + } + } + if len(kept) >= key.RPM { + state.rpmBuckets[key.ID] = kept + return false + } + state.rpmBuckets[key.ID] = append(kept, now) + return true +} + +func allowQuota(key governor.KeyRecord) bool { + store := loadedStore() + if store == nil { + return true + } + ctx := context.Background() + now := time.Now() + limits := []struct { + name string + limit *float64 + }{ + {governor.Range5H, key.FiveHourUSD}, + {governor.Range24H, key.DailyLimitUSD}, + {governor.Range7D, key.WeeklyLimitUSD}, + {governor.RangeMonth, key.MonthlyLimitUSD}, + } + for _, item := range limits { + used, err := store.UsageSum(ctx, key.ID, governor.WindowFor(item.name, now)) + if err != nil { + continue + } + if !governor.CheckLimit(used, item.limit).Allowed { + return false + } + } + return true +} + +func executorUnavailable() ([]byte, error) { + cfg := loadedConfig() + if cfg.CodexContRoute && cfg.FailMode == "fail_closed" { + return errorEnvelope("codexcont_executor_pending", "CPA Governor protected executor is not enabled in this build"), nil + } + return errorEnvelope("codexcont_executor_pending", "CPA Governor protected executor is in passive mode; disable codexcont_enabled to route through CPA provider path"), nil +} + +func executorExecute(raw []byte) ([]byte, error) { + var req executorCallRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + cfg := loadedConfig() + if !cfg.CodexContRoute { + return executorUnavailable() + } + payload := req.ExecutorRequest.Payload + if len(payload) == 0 { + payload = req.ExecutorRequest.OriginalRequest + } + result, err := callHost(methodHostModelExecute, hostModelExecutionRequest{ + EntryProtocol: firstNonEmpty(req.ExecutorRequest.SourceFormat, "openai"), + ExitProtocol: firstNonEmpty(req.ExecutorRequest.Format, "openai"), + Model: req.ExecutorRequest.Model, + Stream: false, + Body: payload, + Headers: cloneHeader(req.ExecutorRequest.Headers), + Query: cloneValues(req.ExecutorRequest.Query), + Alt: req.ExecutorRequest.Alt, + HostCallbackID: req.HostCallbackID, + }) + if err != nil { + return errorEnvelope("host_model_execute_error", err.Error()), nil + } + var resp hostModelExecutionResponse + if err := json.Unmarshal(result, &resp); err != nil { + return errorEnvelope("host_model_execute_decode_error", err.Error()), nil + } + if resp.StatusCode >= 400 { + return errorEnvelope("host_model_execute_http_error", fmt.Sprintf("upstream returned %d", resp.StatusCode)), nil + } + return okEnvelope(executorResponse{Payload: resp.Body, Headers: resp.Headers}) +} + +func executorExecuteStream(raw []byte) ([]byte, error) { + var req executorCallRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + cfg := loadedConfig() + if !cfg.CodexContRoute { + return executorUnavailable() + } + if strings.TrimSpace(req.StreamID) == "" { + return errorEnvelope("stream_id_required", "stream_id is required for executor.execute_stream"), nil + } + payload := req.ExecutorRequest.Payload + if len(payload) == 0 { + payload = req.ExecutorRequest.OriginalRequest + } + result, err := callHost(methodHostModelExecuteStream, hostModelExecutionRequest{ + EntryProtocol: firstNonEmpty(req.ExecutorRequest.SourceFormat, "openai"), + ExitProtocol: firstNonEmpty(req.ExecutorRequest.Format, "openai"), + Model: req.ExecutorRequest.Model, + Stream: true, + Body: payload, + Headers: cloneHeader(req.ExecutorRequest.Headers), + Query: cloneValues(req.ExecutorRequest.Query), + Alt: req.ExecutorRequest.Alt, + HostCallbackID: req.HostCallbackID, + }) + if err != nil { + return errorEnvelope("host_model_stream_error", err.Error()), nil + } + var resp hostModelStreamResponse + if err := json.Unmarshal(result, &resp); err != nil { + return errorEnvelope("host_model_stream_decode_error", err.Error()), nil + } + if resp.StatusCode >= 400 { + return errorEnvelope("host_model_stream_http_error", fmt.Sprintf("upstream returned %d", resp.StatusCode)), nil + } + if resp.StreamID == "" { + return errorEnvelope("host_model_stream_empty", "host returned empty stream id"), nil + } + go forwardHostStream(req.StreamID, resp.StreamID) + return okEnvelope(executorStreamResponse{Headers: resp.Headers}) +} + +func usageHandle(raw []byte) ([]byte, error) { + var rec usageRecord + if err := json.Unmarshal(raw, &rec); err != nil { + return nil, err + } + _ = refreshKeyPolicyState(false) + store := loadedStore() + if store == nil { + return okEnvelope(map[string]any{}) + } + keys, _ := store.ListKeys(context.Background()) + keyByID := map[string]governor.KeyRecord{} + for _, key := range keys { + keyByID[key.ID] = key + } + key := keyByID[rec.APIKey] + if key.ID == "" { + key = keyByID[rec.Source] + } + usage := governor.TokenUsage{ + InputTokens: rec.Detail.InputTokens, + OutputTokens: rec.Detail.OutputTokens, + CachedTokens: rec.Detail.CachedTokens, + CacheReadTokens: rec.Detail.CacheReadTokens, + CacheCreationTokens: rec.Detail.CacheCreationTokens, + ReasoningTokens: rec.Detail.ReasoningTokens, + TotalTokens: rec.Detail.TotalTokens, + } + var cost float64 + var breakdown governor.CostBreakdown + if key.ID != "" { + if price, ok := governor.PriceForModel(key.Prices, firstNonEmpty(rec.Alias, rec.Model)); ok { + breakdown = governor.CostForUsage(price, usage, firstNonEmpty(rec.Alias, rec.Model)) + cost = breakdown.Costs["total"] + } + } + event := governor.UsageEvent{ + RequestID: firstNonEmpty(rec.ResponseHeaders.Get("x-request-id"), rec.ResponseHeaders.Get("x-openai-request-id")), + KeyID: key.ID, + KeyPreview: key.Preview, + Model: firstNonEmpty(rec.Alias, rec.Model), + RequestedModel: firstNonEmpty(rec.Alias, rec.Model), + ActualModel: rec.Model, + Provider: rec.Provider, + ExecutorType: rec.ExecutorType, + Endpoint: rec.Source, + RequestedAt: rec.RequestedAt, + LatencyMS: rec.Latency.Milliseconds(), + TTFTMS: rec.TTFT.Milliseconds(), + ReasoningEffort: rec.ReasoningEffort, + ServiceTier: rec.ServiceTier, + StatusCode: rec.Failure.StatusCode, + Failed: rec.Failed, + Failure: governor.Brief(rec.Failure.Body, 600), + Usage: usage, + Cost: cost, + CostBreakdown: breakdown, + } + _ = store.InsertUsage(context.Background(), event) + return okEnvelope(map[string]any{}) +} + +func managementRegister() ([]byte, error) { + resp := managementRegistrationResponse{ + Routes: []managementRoute{ + {Method: http.MethodGet, Path: "/plugins/cpa-governor/keys"}, + {Method: http.MethodPut, Path: "/plugins/cpa-governor/keys/limits"}, + {Method: http.MethodPost, Path: "/plugins/cpa-governor/keys/reset"}, + {Method: http.MethodGet, Path: "/plugins/cpa-governor/events"}, + {Method: http.MethodGet, Path: "/plugins/cpa-governor/codexcont"}, + {Method: http.MethodPut, Path: "/plugins/cpa-governor/codexcont"}, + }, + Resources: []resourceRoute{ + {Path: "/admin", Menu: "CPA Governor", Description: "Governor admin dashboard"}, + {Path: "/admin/api/keys"}, + {Path: "/admin/api/keys/limits"}, + {Path: "/admin/api/keys/reset"}, + {Path: "/admin/api/events"}, + {Path: "/admin/api/codexcont"}, + {Path: "/user", Description: "Self-service usage dashboard"}, + {Path: "/user/api/session"}, + {Path: "/user/api/me"}, + {Path: "/user/api/usage"}, + {Path: "/user/api/events"}, + {Path: "/user/api/codexcont"}, + }, + } + return okEnvelope(resp) +} + +func managementHandle(raw []byte) ([]byte, error) { + var req managementRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + path := strings.TrimSpace(req.Path) + switch { + case path == "/v0/resource/plugins/cpa-governor/admin" || path == "/admin": + return managementHTML(adminHTML()) + case path == "/v0/resource/plugins/cpa-governor/user" || path == "/user": + return managementHTML(userHTML()) + case strings.HasSuffix(path, "/admin/api/keys"): + return adminKeys(req) + case strings.HasSuffix(path, "/admin/api/keys/limits"): + return adminSetLimits(req) + case strings.HasSuffix(path, "/admin/api/keys/reset"): + return adminReset(req) + case strings.HasSuffix(path, "/admin/api/events"): + return adminEvents(req) + case strings.HasSuffix(path, "/admin/api/codexcont"): + return adminCodexCont(req) + case strings.Contains(path, "/user/api/session"): + return userSession(req) + case strings.Contains(path, "/user/api/me"): + return userMe(req) + case strings.Contains(path, "/user/api/usage"): + return userUsage(req) + case strings.Contains(path, "/user/api/events"): + return userEvents(req) + case strings.Contains(path, "/user/api/codexcont"): + return userCodexCont(req) + case strings.HasSuffix(path, "/plugins/cpa-governor/keys"): + return adminKeys(req) + case strings.HasSuffix(path, "/plugins/cpa-governor/keys/limits"): + return adminSetLimits(req) + case strings.HasSuffix(path, "/plugins/cpa-governor/keys/reset"): + return adminReset(req) + case strings.HasSuffix(path, "/plugins/cpa-governor/events"): + return adminEvents(req) + case strings.HasSuffix(path, "/plugins/cpa-governor/codexcont"): + return adminCodexCont(req) + default: + return jsonResponse(http.StatusNotFound, map[string]any{"ok": false, "error": "not_found"}) + } +} + +func adminKeys(_ managementRequest) ([]byte, error) { + _ = refreshKeyPolicyState(false) + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + keys, err := store.ListKeys(context.Background()) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + safe := make([]map[string]any, 0, len(keys)) + now := time.Now() + for _, key := range keys { + row := key.Safe() + row["usage"] = usageWindows(context.Background(), store, key.ID, now) + safe = append(safe, row) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "keys": safe, "codexcont": codexcontStatus()}) +} + +func adminSetLimits(req managementRequest) ([]byte, error) { + var body struct { + Limits []struct { + ID string `json:"id"` + FiveHourUSD *float64 `json:"five_hour_usd"` + MonthlyUSD *float64 `json:"monthly_usd"` + } `json:"limits"` + } + if err := json.Unmarshal(req.Body, &body); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "invalid_json"}) + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + for _, item := range body.Limits { + if strings.TrimSpace(item.ID) == "" { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "missing_key_id"}) + } + if err := store.SetLimits(context.Background(), item.ID, item.FiveHourUSD, item.MonthlyUSD); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": err.Error()}) + } + } + return adminKeys(req) +} + +func adminReset(req managementRequest) ([]byte, error) { + var body struct { + ID string `json:"id"` + Window string `json:"window"` + } + if err := json.Unmarshal(req.Body, &body); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "invalid_json"}) + } + if body.Window == "" { + body.Window = "all" + } + windows := []string{body.Window} + if body.Window == "all" { + windows = []string{governor.Range5H, governor.Range24H, governor.Range7D, governor.RangeMonth} + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + for _, window := range windows { + if err := store.Reset(context.Background(), body.ID, window, time.Now()); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": err.Error()}) + } + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true}) +} + +func adminEvents(req managementRequest) ([]byte, error) { + keyID := req.Query.Get("key_id") + if keyID == "" { + keyID = "all" + } + return eventsResponseFromRequest(req, keyID) +} + +func adminCodexCont(req managementRequest) ([]byte, error) { + if req.Method == http.MethodPut || (req.Method == http.MethodGet && req.Query.Get("action") == "save") { + var body struct { + Enabled *bool `json:"enabled"` + URL string `json:"url"` + FailMode string `json:"fail_mode"` + } + if req.Method == http.MethodGet { + if rawEnabled := strings.TrimSpace(req.Query.Get("enabled")); rawEnabled != "" { + if parsed, err := strconv.ParseBool(rawEnabled); err == nil { + body.Enabled = &parsed + } + } + body.URL = req.Query.Get("url") + body.FailMode = req.Query.Get("fail_mode") + } else { + _ = json.Unmarshal(req.Body, &body) + } + store := loadedStore() + state.mu.Lock() + if body.Enabled != nil { + state.cfg.CodexContEnabled = *body.Enabled + } + if strings.TrimSpace(body.URL) != "" { + state.cfg.CodexContURL = strings.TrimRight(strings.TrimSpace(body.URL), "/") + } + if strings.TrimSpace(body.FailMode) != "" { + state.cfg.FailMode = strings.ToLower(strings.TrimSpace(body.FailMode)) + } + state.cfg = state.cfg.Normalize() + cfg := state.cfg + state.mu.Unlock() + if store != nil { + settings := map[string]string{ + "codexcont_enabled": strconv.FormatBool(cfg.CodexContEnabled), + "codexcont_url": cfg.CodexContURL, + "fail_mode": cfg.FailMode, + } + if err := store.SaveSettings(context.Background(), settings); err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "save_settings_failed"}) + } + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "codexcont": codexcontStatus()}) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "codexcont": codexcontStatus()}) +} + +func userSession(req managementRequest) ([]byte, error) { + key := userSubmittedKey(req) + if key == "" { + hint := governor.ExplainUnmatchedSubmittedKey(key) + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": hint.Error, "message": hint.Message}) + } + record, ok := findKeyByRaw(key) + if !ok { + hint := governor.ExplainUnmatchedSubmittedKey(key) + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": hint.Error, "message": hint.Message}) + } + if !record.Enabled { + return jsonResponse(http.StatusForbidden, map[string]any{"ok": false, "error": "api_key_disabled", "message": "这个 Key 当前已禁用，请联系管理员。"}) + } + cfg := loadedConfig() + token, err := governor.SignSession(governor.SessionPayload{ + KeyID: record.ID, + KeyHash: record.KeyHash, + ExpiresAt: time.Now().Add(governor.SessionTTL()).Unix(), + }, cfg.SessionSecret) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "session_error"}) + } + body, err := json.Marshal(map[string]any{"ok": true, "me": record.Safe()}) + if err != nil { + return nil, err + } + resp := managementResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{ + "Content-Type": []string{"application/json; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + "Set-Cookie": []string{fmt.Sprintf("cpa_governor_session=%s; Path=/; HttpOnly; SameSite=Lax; Max-Age=86400", token)}, + }, + Body: body, + } + return okEnvelope(resp) +} + +func userSubmittedKey(req managementRequest) string { + raw := firstNonEmpty( + headerFirst(req.Headers, "X-CPA-Governor-Key"), + headerFirst(req.Headers, "X-CPA-User-Key"), + ) + if raw == "" { + raw = bearer(headerFirst(req.Headers, "Authorization")) + } + return governor.NormalizeSubmittedKey(raw) +} + +func userMe(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated"}) + } + row := key.Safe() + if store := loadedStore(); store != nil { + row["usage"] = usageWindows(context.Background(), store, key.ID, time.Now()) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "me": row}) +} + +func userUsage(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated"}) + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + rangeName := req.Query.Get("range") + if rangeName == "" { + rangeName = governor.Range24H + } + summary, err := store.UsageSummary(context.Background(), key.ID, governor.WindowFor(rangeName, time.Now())) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + success := summary.Calls - summary.Failed + successRate := 0.0 + if summary.Calls > 0 { + successRate = float64(success) / float64(summary.Calls) + } + return jsonResponse(http.StatusOK, map[string]any{ + "ok": true, + "range": rangeName, + "limits": key.Safe()["limits"], + "summary": map[string]any{ + "calls": summary.Calls, + "success": success, + "failed": summary.Failed, + "success_rate": successRate, + "total_cost": summary.TotalCost, + "usage": summary.Usage, + }, + }) +} + +func userEvents(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated"}) + } + return eventsResponseFromRequest(req, key.ID) +} + +func userCodexCont(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated"}) + } + limit := 80 + if rawLimit := strings.TrimSpace(req.Query.Get("limit")); rawLimit != "" { + if parsed, err := strconv.Atoi(rawLimit); err == nil { + limit = parsed + } + } + if limit <= 0 || limit > 200 { + limit = 80 + } + requests, source := codexRequestsForKey(key, limit) + sortCodexSummariesNewestFirst(requests) + return jsonResponse(http.StatusOK, map[string]any{ + "ok": true, + "codexcont": codexcontStatus(), + "requests": requests, + "source": source, + }) +} + +func eventsResponse(keyID string, limit int) ([]byte, error) { + return eventsResponseWithRange(keyID, "", limit) +} + +func eventsResponseFromRequest(req managementRequest, keyID string) ([]byte, error) { + limit := 100 + if rawLimit := strings.TrimSpace(req.Query.Get("limit")); rawLimit != "" { + if parsed, err := strconv.Atoi(rawLimit); err == nil { + limit = parsed + } + } + return eventsResponseWithRange(keyID, req.Query.Get("range"), limit) +} + +func eventsResponseWithRange(keyID string, rangeName string, limit int) ([]byte, error) { + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + var events []governor.UsageEvent + var err error + if strings.TrimSpace(rangeName) == "" { + events, err = store.RecentEvents(context.Background(), keyID, limit) + } else { + events, err = store.RecentEventsWindow(context.Background(), keyID, governor.WindowFor(rangeName, time.Now()), limit) + } + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "events": events}) +} + +func usageWindows(ctx context.Context, store *governor.Store, keyID string, now time.Time) map[string]float64 { + out := map[string]float64{} + for _, name := range []string{governor.Range5H, governor.Range24H, governor.Range7D, governor.RangeMonth} { + value, _ := store.UsageSum(ctx, keyID, governor.WindowFor(name, now)) + out[name] = value + } + return out +} + +func codexRequestsForKey(key governor.KeyRecord, limit int) ([]map[string]any, string) { + if requests, ok := fetchCodexContRequests(key, limit); ok { + return requests, "codexcont_admin" + } + store := loadedStore() + if store == nil { + return []map[string]any{}, "unavailable" + } + items, err := store.RecentCodexSummaries(context.Background(), key.ID, limit) + if err != nil { + return []map[string]any{}, "store_error" + } + out := make([]map[string]any, 0, len(items)) + for _, item := range items { + safe := safeCodexSummary(item.Summary, key) + if safe == nil { + continue + } + out = append(out, safe) + } + sortCodexSummariesNewestFirst(out) + return out, "governor_store" +} + +func fetchCodexContRequests(key governor.KeyRecord, limit int) ([]map[string]any, bool) { + cfg := loadedConfig() + if !cfg.CodexContEnabled { + return nil, false + } + base := strings.TrimRight(strings.TrimSpace(cfg.CodexContURL), "/") + if base == "" { + return nil, false + } + parsed, err := url.Parse(base + "/admin/requests?limit=" + strconv.Itoa(limit)) + if err != nil || parsed.Scheme == "" || parsed.Host == "" { + return nil, false + } + client := http.Client{Timeout: 1200 * time.Millisecond} + resp, err := client.Get(parsed.String()) + if err != nil { + return nil, false + } + defer resp.Body.Close() + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return nil, false + } + var body struct { + Requests []map[string]any `json:"requests"` + } + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { + return nil, false + } + out := make([]map[string]any, 0, len(body.Requests)) + for _, req := range body.Requests { + safe := safeCodexSummary(req, key) + if safe == nil { + continue + } + out = append(out, safe) + } + sortCodexSummariesNewestFirst(out) + return out, true +} + +func sortCodexSummariesNewestFirst(items []map[string]any) { + sort.SliceStable(items, func(i, j int) bool { + left, leftOK := codexSummaryDisplayTime(items[i]) + right, rightOK := codexSummaryDisplayTime(items[j]) + if leftOK != rightOK { + return leftOK + } + if !leftOK { + return false + } + return left.After(right) + }) +} + +func codexSummaryDisplayTime(req map[string]any) (time.Time, bool) { + for _, field := range []string{"started_at", "updated_at", "ended_at"} { + if parsed, ok := parseCodexSummaryTime(req[field]); ok { + return parsed, true + } + } + return time.Time{}, false +} + +func parseCodexSummaryTime(value any) (time.Time, bool) { + switch v := value.(type) { + case time.Time: + if v.IsZero() { + return time.Time{}, false + } + return v, true + case string: + raw := strings.TrimSpace(v) + if raw == "" { + return time.Time{}, false + } + for _, layout := range []string{time.RFC3339Nano, time.RFC3339, "2006-01-02 15:04:05"} { + if parsed, err := time.Parse(layout, raw); err == nil { + return parsed, true + } + } + return time.Time{}, false + case json.Number: + if asInt, err := v.Int64(); err == nil { + return unixLikeTime(asInt) + } + if asFloat, err := v.Float64(); err == nil { + return unixLikeTime(int64(asFloat)) + } + case float64: + return unixLikeTime(int64(v)) + case int64: + return unixLikeTime(v) + case int: + return unixLikeTime(int64(v)) + } + return time.Time{}, false +} + +func unixLikeTime(raw int64) (time.Time, bool) { + if raw <= 0 { + return time.Time{}, false + } + if raw > 1_000_000_000_000 { + return time.UnixMilli(raw), true + } + return time.Unix(raw, 0), true +} + +func safeCodexSummary(req map[string]any, key governor.KeyRecord) map[string]any { + if req == nil { + return nil + } + identity, _ := req["key_identity"].(map[string]any) + if !codexIdentityMatches(identity, key) { + return nil + } + fields := []string{ + "request_id", "model", "path", "started_at", "updated_at", "ended_at", + "duration_ms", "status", "protection", "latest_round", + "latest_reasoning_tokens", "first_truncation_round", + "first_truncation_reasoning_tokens", "first_truncation_decision", + "continuation_count", "stopped_reason", "failure_reason", + "passthrough_reason", "rounds", + } + out := map[string]any{} + for _, field := range fields { + if value, ok := req[field]; ok { + out[field] = value + } + } + out["key_identity"] = key.Safe() + return out +} + +func codexIdentityMatches(identity map[string]any, key governor.KeyRecord) bool { + if strings.TrimSpace(key.ID) == "" || identity == nil { + return false + } + id := strings.TrimSpace(fmt.Sprint(identity["id"])) + if id != "" && id == key.ID { + return true + } + preview := strings.TrimSpace(fmt.Sprint(identity["preview"])) + return preview != "" && preview == key.Preview +} + +func forwardHostStream(targetStreamID string, sourceStreamID string) { + defer func() { + _, _ = callHost(methodHostModelStreamClose, hostModelStreamCloseRequest{StreamID: sourceStreamID}) + _, _ = callHost(methodHostStreamClose, hostStreamCloseRequest{StreamID: targetStreamID}) + }() + for { + result, err := callHost(methodHostModelStreamRead, hostModelStreamReadRequest{StreamID: sourceStreamID}) + if err != nil { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Error: governor.Brief(err.Error(), 400), + }) + return + } + var chunk hostModelStreamReadResponse + if err := json.Unmarshal(result, &chunk); err != nil { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Error: "decode host stream chunk: " + governor.Brief(err.Error(), 300), + }) + return + } + if len(chunk.Payload) > 0 { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Payload: chunk.Payload, + }) + } + if chunk.Error != "" { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Error: governor.Brief(chunk.Error, 400), + }) + return + } + if chunk.Done { + return + } + } +} + +func keyFromSession(req managementRequest) (governor.KeyRecord, bool) { + _ = refreshKeyPolicyState(false) + cookie := headerFirst(req.Headers, "Cookie") + token := "" + for _, part := range strings.Split(cookie, ";") { + part = strings.TrimSpace(part) + if strings.HasPrefix(part, "cpa_governor_session=") { + token = strings.TrimPrefix(part, "cpa_governor_session=") + break + } + } + cfg := loadedConfig() + payload, ok := governor.VerifySession(token, cfg.SessionSecret, time.Now()) + if !ok { + return governor.KeyRecord{}, false + } + store := loadedStore() + if store == nil { + return governor.KeyRecord{}, false + } + keys, err := store.ListKeys(context.Background()) + if err != nil { + return governor.KeyRecord{}, false + } + for _, key := range keys { + if key.ID == payload.KeyID && key.Enabled { + return key, true + } + } + return governor.KeyRecord{}, false +} + +func codexcontStatus() map[string]any { + cfg := loadedConfig() + status := map[string]any{ + "enabled": cfg.CodexContEnabled, + "route": cfg.CodexContRoute, + "url": cfg.CodexContURL, + "fail_mode": cfg.FailMode, + "mode": "passive_until_executor_cutover", + } + if cfg.CodexContEnabled { + health := probeCodexContHealth(cfg.CodexContURL) + for key, value := range health { + status[key] = value + } + } + return status +} + +func probeCodexContHealth(baseURL string) map[string]any { + out := map[string]any{ + "health_ok": false, + } + parsed, err := url.Parse(strings.TrimRight(baseURL, "/") + "/engine/healthz") + if err != nil || parsed.Scheme == "" || parsed.Host == "" { + out["health_error"] = "invalid_engine_url" + return out + } + client := http.Client{Timeout: 800 * time.Millisecond} + resp, err := client.Get(parsed.String()) + if err != nil { + out["health_error"] = governor.Brief(err.Error(), 200) + return out + } + defer resp.Body.Close() + out["health_status"] = resp.StatusCode + if resp.StatusCode >= 200 && resp.StatusCode < 300 { + out["health_ok"] = true + } + return out +} + +func bearer(value string) string { + value = strings.TrimSpace(value) + if strings.HasPrefix(strings.ToLower(value), "bearer ") { + return strings.TrimSpace(value[7:]) + } + return "" +} + +func headerFirst(headers http.Header, name string) string { + if headers == nil { + return "" + } + if value := strings.TrimSpace(headers.Get(name)); value != "" { + return value + } + for key, values := range headers { + if !strings.EqualFold(key, name) { + continue + } + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + } + return "" +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func cloneHeader(headers http.Header) http.Header { + if headers == nil { + return nil + } + cloned := make(http.Header, len(headers)) + for key, values := range headers { + cloned[key] = append([]string(nil), values...) + } + return cloned +} + +func cloneValues(values map[string][]string) map[string][]string { + if values == nil { + return nil + } + cloned := make(map[string][]string, len(values)) + for key, items := range values { + cloned[key] = append([]string(nil), items...) + } + return cloned +} + +func okEnvelope(v any) ([]byte, error) { + raw, err := json.Marshal(v) + if err != nil { + return nil, err + } + return json.Marshal(envelope{OK: true, Result: raw}) +} + +func errorEnvelope(code, message string) []byte { + raw, _ := json.Marshal(envelope{OK: false, Error: &envelopeError{Code: code, Message: message}}) + return raw +} + +func jsonResponse(status int, v any) ([]byte, error) { + body, err := json.Marshal(v) + if err != nil { + return nil, err + } + return okEnvelope(managementResponse{ + StatusCode: status, + Headers: http.Header{ + "Content-Type": []string{"application/json; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + }, + Body: body, + }) +} + +func managementHTML(html string) ([]byte, error) { + return okEnvelope(managementResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{ + "Content-Type": []string{"text/html; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + }, + Body: []byte(html), + }) +} + +var hostCall = func(method string, payload any) (json.RawMessage, error) { + _ = payload + return nil, fmt.Errorf("host callback %s is unavailable", method) +} + +func callHost(method string, payload any) (json.RawMessage, error) { + return hostCall(method, payload) +} + +func adminHTML() string { + return renderHTML(adminHTMLTemplate) +} + +func userHTML() string { + return renderHTML(userHTMLTemplate) +} + +func sharedCSS() string { + return sharedCSSTemplate +} + +func renderHTML(tpl string) string { + return strings.ReplaceAll(tpl, "{{CSS}}", sharedCSS()) +} diff --git a/cpa_governor_plugin/go/main_test.go b/cpa_governor_plugin/go/main_test.go new file mode 100644 index 0000000..44afb05 --- /dev/null +++ b/cpa_governor_plugin/go/main_test.go @@ -0,0 +1,603 @@ +package main + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "net/url" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "codexcont/cpa-governor-plugin/internal/governor" +) + +func configureTestState(t *testing.T) governor.KeyRecord { + t.Helper() + store, err := governor.OpenStore(filepath.Join(t.TempDir(), "governor.sqlite")) + if err != nil { + t.Fatal(err) + } + key := governor.KeyRecord{ + ID: "alice-key", + Name: "Alice", + KeyHash: "sha256:" + governor.SHA256Hex("cpa_live"), + Enabled: true, + Preview: governor.HashPreview(governor.SHA256Hex("cpa_live")), + RPM: 60, + Concurrency: 2, + Models: []string{"gpt-5.5"}, + Prices: map[string]governor.ModelPrice{ + "gpt-5.5": { + Model: "gpt-5.5", + InputPerMillion: 5, + OutputPerMillion: 30, + CacheReadPerMillion: 0.5, + }, + }, + } + if err := store.UpsertKey(context.Background(), key); err != nil { + t.Fatal(err) + } + state.mu.Lock() + state.cfg = governor.DefaultConfig() + state.cfg.SessionSecret = "secret" + state.store = store + state.keyState = governor.KeyPolicyState{} + state.rpmBuckets = map[string][]time.Time{} + state.mu.Unlock() + t.Cleanup(func() { + state.mu.Lock() + if state.store != nil { + _ = state.store.Close() + state.store = nil + } + state.mu.Unlock() + }) + return key +} + +func unwrapEnvelope(t *testing.T, raw []byte, out any) { + t.Helper() + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + if !env.OK { + t.Fatalf("envelope error: %#v", env.Error) + } + if err := json.Unmarshal(env.Result, out); err != nil { + t.Fatal(err) + } +} + +func unwrapManagementResponse(t *testing.T, raw []byte) managementResponse { + t.Helper() + var resp managementResponse + unwrapEnvelope(t, raw, &resp) + return resp +} + +func TestPluginRegistrationUsesLocalABI(t *testing.T) { + raw, err := handleMethod(methodPluginRegister, nil) + if err != nil { + t.Fatal(err) + } + var reg registration + unwrapEnvelope(t, raw, ®) + if reg.SchemaVersion != schemaVersion || !reg.Capabilities.ManagementAPI || !reg.Capabilities.UsagePlugin { + t.Fatalf("registration = %#v", reg) + } + if len(reg.Metadata.ConfigFields) == 0 { + t.Fatal("config fields should be exposed") + } + + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var payload map[string]any + if err := json.Unmarshal(env.Result, &payload); err != nil { + t.Fatal(err) + } + caps, _ := payload["capabilities"].(map[string]any) + for _, key := range []string{"frontend_auth_provider", "model_router", "executor", "usage_plugin", "management_api"} { + if caps[key] != true { + t.Fatalf("capability %s missing or false in RPC registration: %s", key, string(env.Result)) + } + } + if _, ok := caps["FrontendAuthProvider"]; ok { + t.Fatalf("unexpected Go-style capability field in RPC registration: %s", string(env.Result)) + } +} + +func TestManagementRegisterUsesCPARPCSchema(t *testing.T) { + raw, err := managementRegister() + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + if !env.OK { + t.Fatalf("envelope error: %#v", env.Error) + } + var payload map[string]any + if err := json.Unmarshal(env.Result, &payload); err != nil { + t.Fatal(err) + } + if _, ok := payload["routes"]; !ok { + t.Fatalf("missing lowercase routes field: %s", string(env.Result)) + } + if _, ok := payload["resources"]; !ok { + t.Fatalf("missing lowercase resources field: %s", string(env.Result)) + } + if _, ok := payload["Routes"]; ok { + t.Fatalf("unexpected uppercase Routes field: %s", string(env.Result)) + } + if _, ok := payload["Resources"]; ok { + t.Fatalf("unexpected uppercase Resources field: %s", string(env.Result)) + } + resources, ok := payload["resources"].([]any) + if !ok { + t.Fatalf("resources should be an array: %#v", payload["resources"]) + } + userResourceSeen := false + for _, item := range resources { + resource, _ := item.(map[string]any) + if resource["Path"] != "/user" { + continue + } + userResourceSeen = true + if menu, _ := resource["Menu"].(string); strings.TrimSpace(menu) != "" { + t.Fatalf("user resource should stay routable but hidden from CPAMP sidebar menu: %#v", resource) + } + } + if !userResourceSeen { + t.Fatal("user resource should remain registered for the dedicated cpa-usage host") + } +} + +func TestFrontendAuthAcceptsManagedKeyAndRejectsDisallowedModel(t *testing.T) { + configureTestState(t) + authBody, _ := json.Marshal(frontendAuthRequest{ + Headers: http.Header{"Authorization": []string{"Bearer cpa_live"}}, + Body: []byte(`{"model":"gpt-5.5","stream":true}`), + }) + raw, err := frontendAuth(authBody) + if err != nil { + t.Fatal(err) + } + var resp frontendAuthResponse + unwrapEnvelope(t, raw, &resp) + if !resp.Authenticated || resp.Principal != "alice-key" { + t.Fatalf("auth response = %#v", resp) + } + disallowed, _ := json.Marshal(frontendAuthRequest{ + Headers: http.Header{"Authorization": []string{"Bearer cpa_live"}}, + Body: []byte(`{"model":"other-model","stream":true}`), + }) + raw, err = frontendAuth(disallowed) + if err != nil { + t.Fatal(err) + } + unwrapEnvelope(t, raw, &resp) + if resp.Authenticated { + t.Fatalf("disallowed model authenticated: %#v", resp) + } +} + +func TestUserSessionNormalizesPastedBearerKey(t *testing.T) { + configureTestState(t) + raw, err := userSession(managementRequest{ + Headers: http.Header{"Authorization": []string{"Bearer Authorization: Bearer Bearer cpa_live "}}, + }) + if err != nil { + t.Fatal(err) + } + resp := unwrapManagementResponse(t, raw) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d body=%s", resp.StatusCode, string(resp.Body)) + } + if got := resp.Headers.Get("set-cookie"); got == "" { + t.Fatal("session cookie was not set") + } + if got := resp.Headers.Get("Cache-Control"); got != "no-store" { + t.Fatalf("cache-control = %q", got) + } + var body map[string]any + if err := json.Unmarshal(resp.Body, &body); err != nil { + t.Fatal(err) + } + if body["ok"] != true { + t.Fatalf("body = %#v", body) + } +} + +func TestUserSessionPrefersDedicatedHeaderOverAuthorization(t *testing.T) { + configureTestState(t) + raw, err := userSession(managementRequest{ + Headers: http.Header{ + "Authorization": []string{"Bearer cpamp-admin-token"}, + "X-CPA-Governor-Key": []string{"Authorization: Bearer cpa_live"}, + }, + }) + if err != nil { + t.Fatal(err) + } + resp := unwrapManagementResponse(t, raw) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d body=%s", resp.StatusCode, string(resp.Body)) + } + if got := resp.Headers.Get("set-cookie"); got == "" { + t.Fatal("session cookie was not set") + } + if got := resp.Headers.Get("Cache-Control"); got != "no-store" { + t.Fatalf("cache-control = %q", got) + } +} + +func TestUserSessionExplainsNativeAndPreviewKeys(t *testing.T) { + configureTestState(t) + cases := []struct { + name string + key string + code string + }{ + {name: "native sk", key: "sk-test", code: "native_cpa_key_not_supported"}, + {name: "preview", key: "cpa_abcd...efgh", code: "key_preview_not_usable"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + raw, err := userSession(managementRequest{ + Headers: http.Header{"Authorization": []string{"Bearer " + tc.key}}, + }) + if err != nil { + t.Fatal(err) + } + resp := unwrapManagementResponse(t, raw) + if resp.StatusCode != http.StatusUnauthorized { + t.Fatalf("status = %d body=%s", resp.StatusCode, string(resp.Body)) + } + var body map[string]any + if err := json.Unmarshal(resp.Body, &body); err != nil { + t.Fatal(err) + } + if body["error"] != tc.code || body["message"] == "" { + t.Fatalf("body = %#v", body) + } + }) + } +} + +func TestUserHTMLSessionUsesGETResourceRoute(t *testing.T) { + html := userHTML() + if !strings.Contains(html, `api("/session"`) || !strings.Contains(html, `"X-CPA-Governor-Key": raw`) { + t.Fatal("user login should call the GET-only resource session route with the dedicated user-key header") + } + if strings.Contains(html, "Authorization:'Bearer '+raw") || strings.Contains(html, `Authorization:"Bearer "+raw`) { + t.Fatal("CPAMP embeds plugin pages behind its own auth; user login must use a dedicated key header") + } + if strings.Contains(strings.ToLower(html), "method:'post'") || strings.Contains(strings.ToLower(html), `method:"post"`) { + t.Fatal("CPA resource routes are GET-only; user login must not use POST") + } +} + +func TestAdminHTMLIsReadOnlyCodexContDashboard(t *testing.T) { + html := adminHTML() + for _, want := range []string{ + `/governor/codexcont/admin`, + `EventSource(BASE + "/logs/stream")`, + `最近请求`, + `高级日志`, + `命中轮`, + `末轮 reasoning`, + `AbortController`, + `cancelSnapshot`, + `reconnectAll`, + `sync-button`, + `applySyncLight`, + `refreshMinimumDelay`, + `activeProcessingCount`, + `PROCESSING_STALE_MS`, + `live-bad`, + } { + if !strings.Contains(html, want) { + t.Fatalf("admin dashboard missing %q", want) + } + } + for _, forbidden := range []string{ + `Key 管理`, + `>请求明细`, + `保存 CodexCont`, + `action:'save'`, + `action:"save"`, + `ccEnabled`, + `ccUrl`, + `ccFail`, + `c.active_requests`, + } { + if strings.Contains(html, forbidden) { + t.Fatalf("admin dashboard should be read-only and not contain %q", forbidden) + } + } +} + +func TestUserHTMLHasTwoTabsAndNoSpinner(t *testing.T) { + html := userHTML() + if !strings.Contains(html, `data-tab="usage"`) || !strings.Contains(html, `额度与明细`) { + t.Fatal("user page should expose the quota/details tab") + } + if !strings.Contains(html, `data-tab="codex"`) || !strings.Contains(html, `思维链保护`) { + t.Fatal("user page should expose the protection tab") + } + if strings.Contains(html, `data-tab="requests"`) || strings.Contains(html, `>请求明细`) { + t.Fatal("request details should be merged into the quota/details tab, not a third tab") + } + for _, want := range []string{ + `setRefreshState`, + `markRefreshStart`, + `just-updated`, + `refreshActive`, + `switchTab`, + `AbortController`, + `cancelRefresh`, + `visibilitychange`, + `applySyncLight`, + `live-bad`, + `sortProtectionItems`, + `activeProcessingCount`, + `PROCESSING_STALE_MS`, + `id="active-chip"`, + `setInterval(() => refreshActive(false), 5000)`, + `单 Key 实时监控`, + } { + if !strings.Contains(html, want) { + t.Fatalf("user page should keep realtime refresh animation hook %q", want) + } + } + if strings.Contains(html, `load(true)`) || strings.Contains(html, `setInterval(() => load(false), 3000)`) { + t.Fatal("user page should not use the old blocking tab switch or 3s full reload loop") + } + css := sharedCSS() + for _, want := range []string{ + `.sync-button.syncing`, + `.sync-button.just-updated`, + `.sync-button.live-ok .sync-light`, + `.sync-button.live-bad .sync-light`, + `@keyframes statusBlink`, + `@keyframes statusPing`, + } { + if !strings.Contains(css, want) { + t.Fatalf("shared css should keep restrained realtime status style %q", want) + } + } + for _, forbidden := range []string{ + ".spin", "spin ", "rotate(", ".metric::after", + `@keyframes syncSweep`, + `@keyframes liveSweep`, + `@keyframes metricBump`, + `@keyframes rowFresh`, + `.sync-button::after`, + `.topbar::after`, + `.metrics.cards-updated .metric`, + } { + if strings.Contains(css, forbidden) { + t.Fatalf("custom pages should not use old spinner/sweep/bump effects: %q", forbidden) + } + } +} + +func TestAdminCodexContGETSavePersistsSettings(t *testing.T) { + configureTestState(t) + raw, err := adminCodexCont(managementRequest{ + Method: http.MethodGet, + Query: url.Values{ + "action": []string{"save"}, + "enabled": []string{"true"}, + "url": []string{"http://codexcont:8787/"}, + "fail_mode": []string{"fail_closed"}, + }, + }) + if err != nil { + t.Fatal(err) + } + resp := unwrapManagementResponse(t, raw) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d body=%s", resp.StatusCode, string(resp.Body)) + } + cfg := loadedConfig() + if !cfg.CodexContEnabled || cfg.CodexContURL != "http://codexcont:8787" || cfg.FailMode != "fail_closed" { + t.Fatalf("cfg = %#v", cfg) + } + settings, err := loadedStore().LoadSettings(context.Background()) + if err != nil { + t.Fatal(err) + } + if settings["codexcont_enabled"] != "true" || settings["codexcont_url"] != "http://codexcont:8787" || settings["fail_mode"] != "fail_closed" { + t.Fatalf("settings = %#v", settings) + } +} + +func TestUserCodexContFiltersToCurrentKey(t *testing.T) { + key := configureTestState(t) + otherPreview := governor.HashPreview(governor.SHA256Hex("cpa_bob")) + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/engine/healthz" { + w.Header().Set("content-type", "application/json") + _, _ = w.Write([]byte(`{"ok":true}`)) + return + } + if r.URL.Path != "/admin/requests" { + t.Fatalf("unexpected path %s", r.URL.Path) + } + w.Header().Set("content-type", "application/json") + _, _ = w.Write([]byte(`{"requests":[ + {"request_id":"alice-old","model":"gpt-5.5","protection":"protected_clean","started_at":"2026-07-02T10:00:00Z","key_identity":{"known":true,"id":"alice-key","name":"Alice","preview":"` + key.Preview + `"},"latest_reasoning_tokens":120,"continuation_count":0}, + {"request_id":"bob-1","model":"gpt-5.5","protection":"protected_clean","key_identity":{"known":true,"id":"bob-key","name":"Bob","preview":"` + otherPreview + `"},"latest_reasoning_tokens":120,"continuation_count":0}, + {"request_id":"unknown-1","model":"gpt-5.5","protection":"protected_clean","key_identity":{"known":false,"preview":"nope"}}, + {"request_id":"alice-new","model":"gpt-5.5","protection":"auto_continued","started_at":"2026-07-02T11:00:00Z","key_identity":{"known":true,"id":"alice-key","name":"Alice","preview":"` + key.Preview + `"},"latest_reasoning_tokens":181,"continuation_count":1} + ]}`)) + })) + defer srv.Close() + + cfg := loadedConfig() + cfg.CodexContEnabled = true + cfg.CodexContURL = srv.URL + state.mu.Lock() + state.cfg = cfg + state.mu.Unlock() + + token, err := governor.SignSession(governor.SessionPayload{ + KeyID: key.ID, + KeyHash: key.KeyHash, + ExpiresAt: time.Now().Add(time.Hour).Unix(), + }, cfg.SessionSecret) + if err != nil { + t.Fatal(err) + } + raw, err := userCodexCont(managementRequest{ + Headers: http.Header{"Cookie": []string{"cpa_governor_session=" + token}}, + Query: url.Values{"limit": []string{"20"}}, + }) + if err != nil { + t.Fatal(err) + } + resp := unwrapManagementResponse(t, raw) + if resp.StatusCode != http.StatusOK { + t.Fatalf("status = %d body=%s", resp.StatusCode, string(resp.Body)) + } + var body struct { + OK bool `json:"ok"` + Source string `json:"source"` + Requests []map[string]any `json:"requests"` + } + if err := json.Unmarshal(resp.Body, &body); err != nil { + t.Fatal(err) + } + if !body.OK || body.Source != "codexcont_admin" { + t.Fatalf("body = %#v", body) + } + if len(body.Requests) != 2 || body.Requests[0]["request_id"] != "alice-new" || body.Requests[1]["request_id"] != "alice-old" { + t.Fatalf("requests were not filtered to current key: %#v", body.Requests) + } + encoded, _ := json.Marshal(body.Requests) + if strings.Contains(string(encoded), "bob-1") || strings.Contains(string(encoded), "unknown-1") { + t.Fatalf("other users leaked into codexcont response: %s", encoded) + } +} + +func TestUsageHandleStoresCost(t *testing.T) { + configureTestState(t) + rec := usageRecord{ + Provider: "openai", + ExecutorType: "codex", + Model: "gpt-5.5-real", + Alias: "gpt-5.5", + APIKey: "alice-key", + Source: "/v1/responses", + ReasoningEffort: "high", + ServiceTier: "default", + RequestedAt: time.Now(), + Latency: 1500 * time.Millisecond, + TTFT: 220 * time.Millisecond, + Failure: usageFailure{ + StatusCode: 200, + }, + ResponseHeaders: http.Header{"X-Request-Id": []string{"req-usage-1"}}, + Detail: usageDetail{ + InputTokens: 100, + CachedTokens: 20, + OutputTokens: 50, + ReasoningTokens: 30, + TotalTokens: 150, + }, + } + rawRec, _ := json.Marshal(rec) + if _, err := usageHandle(rawRec); err != nil { + t.Fatal(err) + } + events, err := loadedStore().RecentEvents(context.Background(), "alice-key", 10) + if err != nil { + t.Fatal(err) + } + if len(events) != 1 || events[0].Cost <= 0 || events[0].Usage.ReasoningTokens != 30 { + t.Fatalf("events = %#v", events) + } + got := events[0] + if got.RequestID != "req-usage-1" || got.Model != "gpt-5.5" || got.ActualModel != "gpt-5.5-real" { + t.Fatalf("model/request fields not projected: %#v", got) + } + if got.Provider != "openai" || got.ExecutorType != "codex" || got.Endpoint != "/v1/responses" { + t.Fatalf("source fields not projected: %#v", got) + } + if got.ReasoningEffort != "high" || got.ServiceTier != "default" || got.TTFTMS != 220 || got.StatusCode != 200 { + t.Fatalf("realtime detail fields not projected: %#v", got) + } +} + +func TestRefreshKeyPolicyStateImportsNewKeys(t *testing.T) { + dir := t.TempDir() + store, err := governor.OpenStore(filepath.Join(dir, "governor.sqlite")) + if err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, "policy.json") + writePolicy := func(id, rawKey string) { + t.Helper() + body := map[string]any{"keys": []map[string]any{{ + "id": id, + "name": id, + "key_hash": "sha256:" + governor.SHA256Hex(rawKey), + "enabled": true, + }}} + raw, err := json.Marshal(body) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, raw, 0o600); err != nil { + t.Fatal(err) + } + } + writePolicy("alice", "cpa_alice") + + state.mu.Lock() + state.cfg = governor.DefaultConfig() + state.store = store + state.keyState = governor.KeyPolicyState{} + state.keyStatePath = path + state.keyStateModTime = time.Time{} + state.keyStateLastCheck = time.Time{} + state.rpmBuckets = map[string][]time.Time{} + state.mu.Unlock() + t.Cleanup(func() { + state.mu.Lock() + if state.store != nil { + _ = state.store.Close() + state.store = nil + } + state.mu.Unlock() + }) + + if err := refreshKeyPolicyState(true); err != nil { + t.Fatal(err) + } + if key, ok := findKeyByRaw("cpa_alice"); !ok || key.ID != "alice" { + t.Fatalf("alice not imported: %#v ok=%v", key, ok) + } + + writePolicy("bob", "cpa_bob") + if err := refreshKeyPolicyState(true); err != nil { + t.Fatal(err) + } + if key, ok := findKeyByRaw("cpa_bob"); !ok || key.ID != "bob" { + t.Fatalf("bob not imported after refresh: %#v ok=%v", key, ok) + } + if key, ok := findKeyByRaw("cpa_alice"); ok { + t.Fatalf("deleted alice should not remain in Governor mirror: %#v", key) + } +} diff --git a/cpa_governor_plugin/go/plugin_export.go b/cpa_governor_plugin/go/plugin_export.go new file mode 100644 index 0000000..f0db1d4 --- /dev/null +++ b/cpa_governor_plugin/go/plugin_export.go @@ -0,0 +1,166 @@ +//go:build cliproxy_plugin + +package main + +/* +#include +#include + +typedef struct { + void* ptr; + size_t len; +} cliproxy_buffer; + +typedef int (*cliproxy_host_call_fn)(void*, const char*, const uint8_t*, size_t, cliproxy_buffer*); +typedef void (*cliproxy_host_free_fn)(void*, size_t); + +typedef struct { + uint32_t abi_version; + void* host_ctx; + cliproxy_host_call_fn call; + cliproxy_host_free_fn free_buffer; +} cliproxy_host_api; + +typedef int (*cliproxy_plugin_call_fn)(char*, uint8_t*, size_t, cliproxy_buffer*); +typedef void (*cliproxy_plugin_free_fn)(void*, size_t); +typedef void (*cliproxy_plugin_shutdown_fn)(void); + +typedef struct { + uint32_t abi_version; + cliproxy_plugin_call_fn call; + cliproxy_plugin_free_fn free_buffer; + cliproxy_plugin_shutdown_fn shutdown; +} cliproxy_plugin_api; + +extern int cliproxyPluginCall(char*, uint8_t*, size_t, cliproxy_buffer*); +extern void cliproxyPluginFree(void*, size_t); +extern void cliproxyPluginShutdown(void); + +static const cliproxy_host_api* stored_host; + +static void store_host_api(const cliproxy_host_api* host) { + stored_host = host; +} + +static int call_host_api(const char* method, const uint8_t* request, size_t request_len, cliproxy_buffer* response) { + if (stored_host == NULL || stored_host->call == NULL) { + return 1; + } + return stored_host->call(stored_host->host_ctx, method, request, request_len, response); +} + +static void free_host_buffer(void* ptr, size_t len) { + if (stored_host != NULL && stored_host->free_buffer != NULL && ptr != NULL) { + stored_host->free_buffer(ptr, len); + } +} +*/ +import "C" + +import ( + "encoding/json" + "fmt" + "unsafe" +) + +//export cliproxy_plugin_init +func cliproxy_plugin_init(host *C.cliproxy_host_api, plugin *C.cliproxy_plugin_api) C.int { + if plugin == nil { + return 1 + } + C.store_host_api(host) + hostCall = cgoHostCall + plugin.abi_version = C.uint32_t(abiVersion) + plugin.call = C.cliproxy_plugin_call_fn(C.cliproxyPluginCall) + plugin.free_buffer = C.cliproxy_plugin_free_fn(C.cliproxyPluginFree) + plugin.shutdown = C.cliproxy_plugin_shutdown_fn(C.cliproxyPluginShutdown) + return 0 +} + +//export cliproxyPluginCall +func cliproxyPluginCall(method *C.char, request *C.uint8_t, requestLen C.size_t, response *C.cliproxy_buffer) C.int { + if response != nil { + response.ptr = nil + response.len = 0 + } + if method == nil { + writeCResponse(response, errorEnvelope("invalid_method", "method is required")) + return 1 + } + var requestBytes []byte + if request != nil && requestLen > 0 { + requestBytes = C.GoBytes(unsafe.Pointer(request), C.int(requestLen)) + } + raw, errHandle := handleMethod(C.GoString(method), requestBytes) + if errHandle != nil { + writeCResponse(response, errorEnvelope("plugin_error", errHandle.Error())) + return 1 + } + writeCResponse(response, raw) + return 0 +} + +//export cliproxyPluginFree +func cliproxyPluginFree(ptr unsafe.Pointer, _ C.size_t) { + if ptr != nil { + C.free(ptr) + } +} + +//export cliproxyPluginShutdown +func cliproxyPluginShutdown() { + shutdownPlugin() +} + +func writeCResponse(response *C.cliproxy_buffer, raw []byte) { + if response == nil || len(raw) == 0 { + return + } + ptr := C.CBytes(raw) + if ptr == nil { + return + } + response.ptr = ptr + response.len = C.size_t(len(raw)) +} + +func cgoHostCall(method string, payload any) (json.RawMessage, error) { + rawPayload, err := json.Marshal(payload) + if err != nil { + return nil, err + } + cMethod := C.CString(method) + defer C.free(unsafe.Pointer(cMethod)) + var response C.cliproxy_buffer + var requestPtr *C.uint8_t + if len(rawPayload) > 0 { + cPayload := C.CBytes(rawPayload) + if cPayload == nil { + return nil, fmt.Errorf("allocate host callback") + } + defer C.free(cPayload) + requestPtr = (*C.uint8_t)(cPayload) + } + callCode := C.call_host_api(cMethod, requestPtr, C.size_t(len(rawPayload)), &response) + var rawResponse []byte + if response.ptr != nil && response.len > 0 { + rawResponse = C.GoBytes(response.ptr, C.int(response.len)) + } + if response.ptr != nil { + C.free_host_buffer(response.ptr, response.len) + } + if callCode != 0 || len(rawResponse) == 0 { + return nil, fmt.Errorf("host callback %s returned code=%d", method, int(callCode)) + } + var env envelope + if err := json.Unmarshal(rawResponse, &env); err != nil { + return nil, err + } + if !env.OK { + if env.Error != nil { + return nil, fmt.Errorf("%s: %s", env.Error.Code, env.Error.Message) + } + return nil, fmt.Errorf("host callback failed") + } + return env.Result, nil +} diff --git a/cpa_governor_plugin/go/plugin_types.go b/cpa_governor_plugin/go/plugin_types.go new file mode 100644 index 0000000..991e717 --- /dev/null +++ b/cpa_governor_plugin/go/plugin_types.go @@ -0,0 +1,231 @@ +package main + +import ( + "net/http" + "net/url" + "time" +) + +const ( + abiVersion uint32 = 1 + schemaVersion uint32 = 1 + + methodPluginRegister = "plugin.register" + methodPluginReconfigure = "plugin.reconfigure" + methodFrontendAuthIdentifier = "frontend_auth.identifier" + methodFrontendAuthAuthenticate = "frontend_auth.authenticate" + methodModelRoute = "model.route" + methodExecutorIdentifier = "executor.identifier" + methodExecutorExecute = "executor.execute" + methodExecutorExecuteStream = "executor.execute_stream" + methodExecutorCountTokens = "executor.count_tokens" + methodUsageHandle = "usage.handle" + methodManagementRegister = "management.register" + methodManagementHandle = "management.handle" + methodHostModelExecute = "host.model.execute" + methodHostModelExecuteStream = "host.model.execute_stream" + methodHostModelStreamRead = "host.model.stream_read" + methodHostModelStreamClose = "host.model.stream_close" + methodHostStreamEmit = "host.stream.emit" + methodHostStreamClose = "host.stream.close" +) + +const ( + configString = "string" + configBoolean = "boolean" + configEnum = "enum" + + routeTargetSelf = "self" +) + +type configField struct { + Name string `json:"Name"` + Type string `json:"Type"` + EnumValues []string `json:"EnumValues,omitempty"` + Description string `json:"Description"` +} + +type frontendAuthRequest struct { + Method string `json:"Method"` + Path string `json:"Path"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` +} + +type frontendAuthResponse struct { + Authenticated bool `json:"Authenticated"` + Principal string `json:"Principal,omitempty"` + Metadata map[string]string `json:"Metadata,omitempty"` +} + +type modelRouteRequest struct { + PluginID string `json:"PluginID"` + SourceFormat string `json:"SourceFormat"` + RequestedModel string `json:"RequestedModel"` + Stream bool `json:"Stream"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` + Metadata map[string]any `json:"Metadata"` + AvailableProviders []string `json:"AvailableProviders"` +} + +type modelRouteResponse struct { + Handled bool `json:"Handled"` + TargetKind string `json:"TargetKind,omitempty"` + Target string `json:"Target,omitempty"` + TargetModel string `json:"TargetModel,omitempty"` + Reason string `json:"Reason,omitempty"` +} + +type managementRegistrationResponse struct { + Routes []managementRoute `json:"routes,omitempty"` + Resources []resourceRoute `json:"resources,omitempty"` +} + +type managementRoute struct { + Method string `json:"Method"` + Path string `json:"Path"` + Menu string `json:"Menu,omitempty"` + Description string `json:"Description,omitempty"` +} + +type resourceRoute struct { + Path string `json:"Path"` + Menu string `json:"Menu,omitempty"` + Description string `json:"Description,omitempty"` +} + +type managementRequest struct { + Method string `json:"Method"` + Path string `json:"Path"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type managementResponse struct { + StatusCode int `json:"StatusCode"` + Headers http.Header `json:"Headers"` + Body []byte `json:"Body"` +} + +type executorResponse struct { + Payload []byte `json:"Payload"` + Headers http.Header `json:"Headers,omitempty"` +} + +type usageRecord struct { + Provider string `json:"Provider"` + ExecutorType string `json:"ExecutorType"` + Model string `json:"Model"` + Alias string `json:"Alias"` + APIKey string `json:"APIKey"` + AuthID string `json:"AuthID"` + AuthIndex string `json:"AuthIndex"` + AuthType string `json:"AuthType"` + Source string `json:"Source"` + ReasoningEffort string `json:"ReasoningEffort"` + ServiceTier string `json:"ServiceTier"` + RequestedAt time.Time `json:"RequestedAt"` + Latency time.Duration `json:"Latency"` + TTFT time.Duration `json:"TTFT"` + Failed bool `json:"Failed"` + Failure usageFailure `json:"Failure"` + Detail usageDetail `json:"Detail"` + ResponseHeaders http.Header `json:"ResponseHeaders"` +} + +type executorRequest struct { + AuthID string `json:"AuthID"` + AuthProvider string `json:"AuthProvider"` + Model string `json:"Model"` + Format string `json:"Format"` + Stream bool `json:"Stream"` + Alt string `json:"Alt"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + OriginalRequest []byte `json:"OriginalRequest"` + SourceFormat string `json:"SourceFormat"` + Payload []byte `json:"Payload"` + Metadata map[string]any `json:"Metadata"` + StorageJSON []byte `json:"StorageJSON"` + AuthMetadata map[string]any `json:"AuthMetadata"` + AuthAttributes map[string]string `json:"AuthAttributes"` +} + +type executorCallRequest struct { + ExecutorRequest executorRequest `json:"ExecutorRequest"` + StreamID string `json:"stream_id,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type executorStreamResponse struct { + Headers http.Header `json:"headers,omitempty"` +} + +type hostModelExecutionRequest struct { + EntryProtocol string `json:"entry_protocol"` + ExitProtocol string `json:"exit_protocol"` + Model string `json:"model"` + Stream bool `json:"stream"` + Body []byte `json:"body"` + Headers http.Header `json:"headers"` + Query url.Values `json:"query"` + Alt string `json:"alt,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type hostModelExecutionResponse struct { + StatusCode int `json:"status_code"` + Headers http.Header `json:"headers"` + Body []byte `json:"body"` +} + +type hostModelStreamResponse struct { + StatusCode int `json:"status_code"` + Headers http.Header `json:"headers"` + StreamID string `json:"stream_id"` +} + +type hostModelStreamReadRequest struct { + StreamID string `json:"stream_id"` +} + +type hostModelStreamReadResponse struct { + Payload []byte `json:"payload"` + Error string `json:"error"` + Done bool `json:"done"` +} + +type hostModelStreamCloseRequest struct { + StreamID string `json:"stream_id"` +} + +type hostStreamEmitRequest struct { + StreamID string `json:"stream_id"` + Payload []byte `json:"payload,omitempty"` + Error string `json:"error,omitempty"` +} + +type hostStreamCloseRequest struct { + StreamID string `json:"stream_id"` + Error string `json:"error,omitempty"` +} + +type usageFailure struct { + StatusCode int `json:"StatusCode"` + Body string `json:"Body"` +} + +type usageDetail struct { + InputTokens int64 `json:"InputTokens"` + OutputTokens int64 `json:"OutputTokens"` + ReasoningTokens int64 `json:"ReasoningTokens"` + CachedTokens int64 `json:"CachedTokens"` + CacheReadTokens int64 `json:"CacheReadTokens"` + CacheCreationTokens int64 `json:"CacheCreationTokens"` + TotalTokens int64 `json:"TotalTokens"` +} diff --git a/cpa_key_policy_plus_plugin/README.md b/cpa_key_policy_plus_plugin/README.md new file mode 100644 index 0000000..bac3798 --- /dev/null +++ b/cpa_key_policy_plus_plugin/README.md @@ -0,0 +1,74 @@ +# CPA Key Policy Plus Plugin + +`cpa-key-policy-plus` is the self-owned replacement for the old Key Policy +plugin. It owns ordinary user `cpa_` keys, per-key limits, usage projection, +soft resets, model allowlists, RPM policy, and rolling quota windows. + +It does not modify CPA, CPAMP, or the old Key Policy source/image. + +Concurrency and Codex active-window limits were intentionally retired because +normal Codex conversations can trip them too easily. Old database fields remain +for schema compatibility, but new saves force them to `0` and frontend auth does +not enforce them. + +## Safety Boundary + +Plus is safe to deploy as the exclusive user-key auth and quota authority. The +public `/v1/responses` route should stay on the current known-good CodexCont +sidecar path until the separate `cpa-codexcont-executor` plugin has passed a +controlled server validation. The current CodexCont Engine API is +status/summary only. + +## Local Test + +```powershell +cd D:\Dev\20_Software\23_Reference\llm-gateway\CodexCont\cpa_key_policy_plus_plugin\go +go test ./... +``` + +## Linux Build + +```bash +cd cpa_key_policy_plus_plugin/go +CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -tags cliproxy_plugin -buildmode=c-shared -o cpa-key-policy-plus.so . +``` + +The production CPA host loads platform artifacts from the configured plugin +directory, for example: + +```text +/CLIProxyAPI/plugins/linux/amd64/cpa-key-policy-plus.so +``` + +## Minimal CPA Config + +```yaml +plugins: + enabled: true + dir: /CLIProxyAPI/plugins + configs: + cpa-key-policy-plus: + enabled: true + priority: 10 + exclusive_auth: true + state_db_path: /CLIProxyAPI/plugin-state/cpa-key-policy-plus/policyplus.sqlite + key_policy_state_path: /CLIProxyAPI/plugin-state/cpa-key-policy-state.json + legacy_quota_db_path: /CLIProxyAPI/plugin-state/cpa-usage-portal/usage-portal.sqlite + governor_state_db_path: /CLIProxyAPI/plugin-state/cpa-governor/governor.sqlite + codex_summary_db_path: /CLIProxyAPI/plugin-state/cpa-codexcont-executor/executor.sqlite + session_secret: ${CPA_KEY_POLICY_PLUS_SESSION_SECRET} + codexcont_enabled: true + codexcont_route: false + codexcont_url: http://codexcont:8787 + fail_mode: fallback +``` + +Disable the old `cpa-key-policy` only after Plus loads and current `cpa_` keys +authenticate successfully. + +## User Page + +`cpa-usage.konbakuyomu.us` should expose only the Plus user resource and +`/user/api/*` compatibility routes. User login uses the full `cpa_` key in the +`X-CPA-Key-Policy-Plus-Key` header. Native `sk...` keys and shortened previews +are rejected. diff --git a/cpa_key_policy_plus_plugin/go/assets/admin.html b/cpa_key_policy_plus_plugin/go/assets/admin.html new file mode 100644 index 0000000..cb5b08f --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/assets/admin.html @@ -0,0 +1,834 @@ + + + + + + CPA Key Policy+ + + + + +
+
+
+
K+
+
+

CPA Key Policy+

+

Key 策略层：读取 CPA 原生 Key，只编辑 RPM、模型、价格和额度窗口

+
+
+
+ 准备同步 + + +
+
+ +
+ +
+
+
+
+

Key 策略

+

Key 的新增、删除、复制和别名在 CPA/CPAMP 管理；这里仅维护 Plus 策略。

+
+ +
+
+ + + + + + + + + + + + + +
来源/别名状态额度概览RPM模型/价格
加载中...
+
+
+ + +
+
+ + + + + + diff --git a/cpa_key_policy_plus_plugin/go/assets/shared.css b/cpa_key_policy_plus_plugin/go/assets/shared.css new file mode 100644 index 0000000..aaf9aca --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/assets/shared.css @@ -0,0 +1,553 @@ +:root { + color-scheme: dark; + --bg: #111722; + --panel: #151b27; + --panel-2: #1b2230; + --panel-3: #202838; + --line: #283244; + --line-strong: #39465c; + --text: #e6eaf0; + --muted: #94a3b8; + --blue: #3b82f6; + --blue-soft: rgba(59, 130, 246, .14); + --green: #68d65f; + --green-soft: rgba(104, 214, 95, .12); + --red: #f87171; + --red-soft: rgba(248, 113, 113, .12); + --amber: #f4b942; + --amber-soft: rgba(244, 185, 66, .12); + --teal: #2bb3c5; + --teal-soft: rgba(43, 179, 197, .11); + --shadow: 0 10px 26px rgba(0, 0, 0, .18); + --mono: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace; + --sans: Inter, "Segoe UI", "Microsoft YaHei", system-ui, sans-serif; + font-family: var(--sans); +} +* { box-sizing: border-box; } +body { + margin: 0; + min-height: 100vh; + background: linear-gradient(180deg, #151b26 0%, var(--bg) 46%, #0f1520 100%); + color: var(--text); + font: 14px/1.48 var(--sans); + letter-spacing: 0; + overflow-x: hidden; +} +button, select, input { + min-height: 38px; + border: 1px solid var(--line); + border-radius: 8px; + background: #111827; + color: var(--text); + padding: 0 12px; + font: inherit; +} +button { + display: inline-flex; + align-items: center; + justify-content: center; + gap: 8px; + cursor: pointer; + white-space: nowrap; + transition: border-color .18s ease, background .18s ease, color .18s ease; +} +button:hover, button.active, select:hover, input:focus { + border-color: var(--line-strong); + outline: none; +} +button:disabled { + cursor: wait; + opacity: .92; +} +button.primary { + border-color: rgba(59, 130, 246, .48); + background: rgba(59, 130, 246, .18); +} +button.ghost { background: #1a2230; } +button.danger-action { + border-color: rgba(248, 113, 113, .42); + background: rgba(248, 113, 113, .12); + color: #ffb8b8; +} +button.danger-action:hover { + border-color: rgba(248, 113, 113, .62); + background: rgba(248, 113, 113, .18); +} +button.compact { + min-height: 28px; + padding: 0 9px; + margin-top: 6px; + font-size: 12px; +} +.sync-button { + position: relative; + min-width: 96px; +} +.sync-button.syncing { + border-color: rgba(59, 130, 246, .5); + background: rgba(59, 130, 246, .14); + color: #c8dcff; +} +.sync-button.just-updated { + border-color: rgba(104, 214, 95, .5); + background: rgba(104, 214, 95, .12); + color: #d3f8cf; +} +.sync-button.sync-error { + border-color: rgba(248, 113, 113, .55); + background: rgba(248, 113, 113, .12); + color: #ffc7c7; +} +.sync-light { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--muted); + box-shadow: 0 0 0 0 rgba(154, 167, 184, .2); + position: relative; + flex: 0 0 auto; + transition: background .18s ease, box-shadow .18s ease; +} +.sync-button.live-ok .sync-light { + background: var(--green); + box-shadow: 0 0 10px rgba(104, 214, 95, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.live-info .sync-light { + background: var(--blue); + box-shadow: 0 0 10px rgba(59, 130, 246, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.live-warn .sync-light { + background: var(--amber); + box-shadow: 0 0 10px rgba(244, 185, 66, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.live-bad .sync-light { + background: var(--red); + box-shadow: 0 0 10px rgba(248, 113, 113, .28); + animation: statusBlink 1.45s ease-in-out infinite; +} +.sync-button.syncing .sync-light { + background: var(--teal); + animation: statusBlink 1s ease-in-out infinite; +} +.sync-button.just-updated .sync-light { + background: var(--green); + box-shadow: 0 0 16px rgba(97, 211, 79, .42); +} +.sync-button.sync-error .sync-light { + background: var(--red); + box-shadow: 0 0 16px rgba(255, 100, 109, .38); +} +.shell { + width: min(1680px, calc(100% - 32px)); + margin: 0 auto; + padding: 22px 0 34px; +} +.topbar { + position: relative; + min-height: 58px; + display: flex; + align-items: center; + justify-content: space-between; + gap: 16px; + padding: 0 16px; + margin-bottom: 16px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; + box-shadow: var(--shadow); +} +.brand { + display: flex; + align-items: center; + gap: 12px; + min-width: 0; +} +.brand > div { min-width: 0; } +.mark { + width: 34px; + height: 34px; + display: grid; + place-items: center; + border-radius: 8px; + background: linear-gradient(135deg, #2b8df0, #24b8cf); + color: #fff; + font-weight: 850; +} +h1, h2, h3, p { margin: 0; } +h1 { + font-size: 20px; + line-height: 1.2; + font-weight: 780; +} +h2 { font-size: 15px; font-weight: 760; } +h3 { font-size: 13px; font-weight: 760; } +.subtitle { + margin-top: 2px; + color: var(--muted); + font-size: 12px; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.toolbar, .filters { + display: flex; + align-items: center; + justify-content: flex-end; + gap: 8px; + flex-wrap: wrap; +} +.chip { + display: inline-flex; + align-items: center; + gap: 7px; + min-height: 32px; + padding: 0 10px; + border-radius: 999px; + border: 1px solid var(--line); + background: #1a2230; + color: var(--muted); + font-size: 12px; + font-weight: 720; + line-height: 1.2; + white-space: nowrap; + transition: border-color .18s ease, background .18s ease, color .18s ease, box-shadow .18s ease; +} +.dot { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--muted); + position: relative; + flex: 0 0 auto; +} +.chip.stream .dot { + animation: statusBlink 1.45s ease-in-out infinite; +} +.chip.stream .dot::after { + content: ""; + position: absolute; + inset: -5px; + border-radius: inherit; + border: 1px solid currentColor; + opacity: .55; + animation: statusPing 1.45s ease-out infinite; +} +.chip.ok { border-color: rgba(104, 214, 95, .28); background: var(--green-soft); color: #a9f2a2; } +.chip.ok .dot { background: var(--green); } +.chip.bad { border-color: rgba(248, 113, 113, .3); background: var(--red-soft); color: #ffb8b8; } +.chip.bad .dot { background: var(--red); } +.chip.warn { border-color: rgba(244, 185, 66, .3); background: var(--amber-soft); color: #ffd893; } +.chip.warn .dot { background: var(--amber); } +.chip.info { border-color: rgba(59, 130, 246, .32); background: var(--blue-soft); color: #a9c8ff; } +.chip.info .dot { background: var(--blue); } +.metrics { + display: grid; + grid-template-columns: repeat(4, minmax(150px, 1fr)); + gap: 10px; + margin-bottom: 14px; +} +.metric { + min-height: 86px; + padding: 14px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; + box-shadow: var(--shadow); + transition: border-color .2s ease, background .2s ease; +} +.metric .label { + color: var(--muted); + font-size: 12px; + font-weight: 720; +} +.metric .value { + margin-top: 6px; + font-size: 25px; + line-height: 1; + font-weight: 820; +} +.metric .hint { + margin-top: 7px; + color: var(--muted); + font-size: 12px; +} +.panel { + margin-bottom: 14px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; + box-shadow: var(--shadow); + overflow: hidden; +} +.panel-head { + min-height: 48px; + padding: 12px 14px; + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; + border-bottom: 1px solid var(--line); +} +.panel-body { padding: 14px; } +.embedded-panel { + margin-bottom: 0; + box-shadow: none; +} +.tabs { + display: flex; + gap: 8px; + margin-bottom: 14px; + overflow-x: auto; +} +.tabs button { + position: relative; + overflow: hidden; +} +.tabs button.active::after { + content: ""; + position: absolute; + left: 12px; + right: 12px; + bottom: 4px; + height: 2px; + border-radius: 999px; + background: rgba(59, 130, 246, .72); +} +.tabs button.active { + border-color: rgba(59, 130, 246, .42); + background: rgba(59, 130, 246, .16); + color: #c8dcff; +} +#content { + transition: opacity .18s ease, transform .18s ease; +} +#content.content-refreshing { + opacity: .72; + transform: translate3d(0, 3px, 0); +} +#content.view-enter { + animation: viewRise .26s ease-out both; +} +.table-wrap { overflow-x: auto; } +table { + width: 100%; + min-width: 980px; + border-collapse: collapse; + table-layout: fixed; +} +.realtime-table { + min-width: 1180px; +} +th, td { + border-bottom: 1px solid rgba(40, 52, 72, .75); + padding: 11px 12px; + text-align: left; + vertical-align: middle; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} +tbody tr { + transition: background-color .24s ease, box-shadow .24s ease, transform .2s ease; +} +tbody tr:hover { + background: rgba(148, 163, 184, .055); +} +tbody tr.data-row { + background-clip: padding-box; +} +th { + color: var(--muted); + font-size: 12px; + font-weight: 760; + background: #303442; +} +td strong { display: block; } +.strong { + display: block; + color: var(--text); + font-weight: 780; +} +.success { color: var(--green); } +.danger { color: var(--red); } +small, .muted { + color: var(--muted); + font-size: 12px; +} +.mono { font-family: var(--mono); } +.good { color: var(--green); font-weight: 760; } +.bad-text { color: var(--red); font-weight: 760; } +.blue { color: var(--blue); font-weight: 760; } +.detail-row td { + white-space: normal; + overflow: visible; + background: #111722; +} +.detail-grid { + display: grid; + grid-template-columns: repeat(4, minmax(220px, 1fr)); + gap: 10px; +} +.detail-card { + min-width: 0; + padding: 12px; + border: 1px solid var(--line); + border-radius: 8px; + background: #151b27; +} +.kv { + display: grid; + grid-template-columns: minmax(92px, 42%) 1fr; + gap: 7px 10px; + margin-top: 10px; +} +.kv dt { color: var(--muted); } +.kv dd { + margin: 0; + min-width: 0; + overflow-wrap: anywhere; +} +.detail-note { + margin-top: 10px; + color: var(--muted); + font-size: 12px; +} +.failure-box { + margin-top: 10px; + min-height: 46px; + max-height: 132px; + overflow: auto; + padding: 10px; + border: 1px solid rgba(40, 52, 72, .78); + border-radius: 8px; + background: #101620; + color: #d7deea; + white-space: pre-wrap; + overflow-wrap: anywhere; +} +.notice { + margin: 10px 14px; + padding: 10px; + border: 1px solid rgba(244, 185, 66, .26); + border-radius: 8px; + background: var(--amber-soft); + color: #ffd893; + font-size: 12px; +} +.empty { + padding: 22px; + text-align: center; + color: var(--muted); +} +.key-cell input { min-width: 170px; } +input.narrow { width: 82px; min-width: 74px; } +textarea { + width: 220px; + min-height: 38px; + margin: 2px 0; + padding: 8px 10px; + border: 1px solid var(--line); + border-radius: 7px; + background: #0f151f; + color: var(--text); + font-family: var(--mono); + font-size: 12px; + resize: vertical; +} +.check { + display: inline-flex; + align-items: center; + gap: 6px; + color: var(--text); + white-space: nowrap; +} +.actions { + display: flex; + flex-wrap: wrap; + gap: 6px; + min-width: 150px; +} +button.mini { + min-height: 28px; + padding: 4px 8px; + font-size: 12px; +} +.logbox { + margin: 0; + max-height: 260px; + overflow: auto; + padding: 14px; + color: var(--muted); +} +.login { + max-width: 720px; + margin: 52px auto; +} +.login-row { + display: flex; + gap: 10px; + margin-top: 14px; +} +.login input { + flex: 1; + min-width: 0; +} +#err { margin-top: 10px; color: #ffb1b6; } +details.advanced > summary { + cursor: pointer; + padding: 12px 14px; + color: var(--muted); + font-weight: 760; +} +.logs { + max-height: 320px; + overflow: auto; + padding: 0 14px 14px; +} +.log-line { + display: grid; + grid-template-columns: 92px 86px 150px 1fr; + gap: 10px; + padding: 8px 0; + border-top: 1px solid rgba(40, 52, 72, .6); + font-family: var(--mono); + font-size: 12px; +} +.hidden { display: none !important; } +@keyframes statusBlink { + 0%, 100% { transform: scale(.9); opacity: .65; } + 50% { transform: scale(1.18); opacity: 1; } +} +@keyframes statusPing { + 0% { transform: scale(.5); opacity: .55; } + 80%, 100% { transform: scale(1.8); opacity: 0; } +} +@keyframes viewRise { + 0% { opacity: .78; transform: translate3d(0, 4px, 0); } + 100% { opacity: 1; transform: translate3d(0, 0, 0); } +} +@media (prefers-reduced-motion: reduce) { + *, *::before, *::after { + animation-duration: .01ms !important; + animation-iteration-count: 1 !important; + scroll-behavior: auto !important; + transition-duration: .01ms !important; + } +} +@media (max-width: 900px) { + .metrics { grid-template-columns: repeat(2, minmax(140px, 1fr)); } + .detail-grid { grid-template-columns: 1fr; } +} +@media (max-width: 620px) { + .shell { width: min(1680px, calc(100% - 20px)); padding-top: 12px; } + .topbar { align-items: flex-start; flex-direction: column; padding: 12px; } + .topbar .brand { width: 100%; max-width: 100%; } + .toolbar, .filters { justify-content: flex-start; } + .metrics { grid-template-columns: 1fr; } + .login-row { flex-direction: column; } + table { min-width: 900px; } + .log-line { grid-template-columns: 1fr; gap: 3px; } +} diff --git a/cpa_key_policy_plus_plugin/go/assets/user.html b/cpa_key_policy_plus_plugin/go/assets/user.html new file mode 100644 index 0000000..a06e28a --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/assets/user.html @@ -0,0 +1,670 @@ + + + + + + CPA 用量自助页 + + + +
+
+
+
U
+
+

CPA 用量自助页

+

单 Key 实时监控、额度明细和思维链保护状态

+
+
+
+ 未登录 + + + +
+
+ + + + +
+ + + + diff --git a/cpa_key_policy_plus_plugin/go/go.mod b/cpa_key_policy_plus_plugin/go/go.mod new file mode 100644 index 0000000..632ad2a --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/go.mod @@ -0,0 +1,24 @@ +module codexcont/cpa-key-policy-plus-plugin + +go 1.22 + +require ( + gopkg.in/yaml.v3 v3.0.1 + modernc.org/sqlite v1.33.1 +) + +require ( + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/ncruces/go-strftime v0.1.9 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + golang.org/x/sys v0.22.0 // indirect + modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect + modernc.org/libc v1.55.3 // indirect + modernc.org/mathutil v1.6.0 // indirect + modernc.org/memory v1.8.0 // indirect + modernc.org/strutil v1.2.0 // indirect + modernc.org/token v1.1.0 // indirect +) diff --git a/cpa_key_policy_plus_plugin/go/go.sum b/cpa_key_policy_plus_plugin/go/go.sum new file mode 100644 index 0000000..6ea7e08 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/go.sum @@ -0,0 +1,53 @@ +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= +github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic= +golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI= +golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw= +golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ= +modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ= +modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y= +modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s= +modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE= +modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ= +modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw= +modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4= +modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U= +modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w= +modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4= +modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo= +modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E= +modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU= +modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4= +modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= +modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc= +modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss= +modernc.org/sqlite v1.33.1 h1:trb6Z3YYoeM9eDL1O8do81kP+0ejv+YzgyFo+Gwy0nM= +modernc.org/sqlite v1.33.1/go.mod h1:pXV2xHxhzXZsgT/RtTFAPY6JJDEvOTcTdwADQCCWD4k= +modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA= +modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/config.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/config.go new file mode 100644 index 0000000..d2b7b74 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/config.go @@ -0,0 +1,85 @@ +package policyplus + +import ( + "strings" + "time" +) + +type Config struct { + Enabled bool `yaml:"enabled"` + ExclusiveAuth bool `yaml:"exclusive_auth"` + StateDBPath string `yaml:"state_db_path"` + KeyPolicyStatePath string `yaml:"key_policy_state_path"` + LegacyQuotaDBPath string `yaml:"legacy_quota_db_path"` + GovernorStateDBPath string `yaml:"governor_state_db_path"` + CodexSummaryDBPath string `yaml:"codex_summary_db_path"` + NativeKeysConfigPath string `yaml:"native_keys_config_path"` + CPAMPAliasDBPath string `yaml:"cpamp_alias_db_path"` + CPAMPAliasDBPaths string `yaml:"cpamp_alias_db_paths"` + SessionSecret string `yaml:"session_secret"` + CodexContEnabled bool `yaml:"codexcont_enabled"` + CodexContRoute bool `yaml:"codexcont_route"` + CodexContURL string `yaml:"codexcont_url"` + FailMode string `yaml:"fail_mode"` + PollIntervalMS int `yaml:"poll_interval_ms"` +} + +func DefaultConfig() Config { + return Config{ + Enabled: true, + ExclusiveAuth: true, + StateDBPath: "cpa-policyplus.sqlite", + SessionSecret: "change-me", + CodexContEnabled: false, + CodexContURL: "http://codexcont:8787", + FailMode: "fallback", + PollIntervalMS: 1500, + } +} + +func (c Config) Normalize() Config { + if strings.TrimSpace(c.StateDBPath) == "" { + c.StateDBPath = DefaultConfig().StateDBPath + } + c.FailMode = strings.ToLower(strings.TrimSpace(c.FailMode)) + if c.FailMode == "" { + c.FailMode = "fallback" + } + c.CodexContURL = strings.TrimRight(strings.TrimSpace(c.CodexContURL), "/") + if c.CodexContURL == "" { + c.CodexContURL = DefaultConfig().CodexContURL + } + c.CodexSummaryDBPath = strings.TrimSpace(c.CodexSummaryDBPath) + c.NativeKeysConfigPath = strings.TrimSpace(c.NativeKeysConfigPath) + c.CPAMPAliasDBPath = strings.TrimSpace(c.CPAMPAliasDBPath) + c.CPAMPAliasDBPaths = strings.TrimSpace(c.CPAMPAliasDBPaths) + if c.PollIntervalMS <= 0 { + c.PollIntervalMS = 1500 + } + return c +} + +func (c Config) AliasDBPaths() []string { + c = c.Normalize() + seen := map[string]bool{} + var out []string + add := func(value string) { + for _, part := range strings.FieldsFunc(value, func(r rune) bool { + return r == ',' || r == ';' || r == '\n' || r == '\r' || r == '\t' + }) { + part = strings.TrimSpace(part) + if part == "" || seen[part] { + continue + } + seen[part] = true + out = append(out, part) + } + } + add(c.CPAMPAliasDBPath) + add(c.CPAMPAliasDBPaths) + return out +} + +func SessionTTL() time.Duration { + return 24 * time.Hour +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/models.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/models.go new file mode 100644 index 0000000..3e8f8db --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/models.go @@ -0,0 +1,714 @@ +package policyplus + +import ( + "encoding/json" + "fmt" + "os" + "regexp" + "strings" +) + +type ModelPrice struct { + Model string `json:"model"` + TargetModel string `json:"target_model,omitempty"` + Provider string `json:"provider,omitempty"` + InputPerMillion float64 `json:"input_per_million"` + OutputPerMillion float64 `json:"output_per_million"` + CacheReadPerMillion float64 `json:"cache_read_per_million"` + CacheCreationPerMillion float64 `json:"cache_creation_per_million"` +} + +type ModelOption struct { + ID string `json:"id"` + DisplayName string `json:"display_name,omitempty"` + Type string `json:"type,omitempty"` + OwnedBy string `json:"owned_by,omitempty"` + Source string `json:"source,omitempty"` + Known bool `json:"known"` +} + +type KeyRecord struct { + ID string `json:"id"` + Name string `json:"name"` + KeyHash string `json:"key_hash"` + Enabled bool `json:"enabled"` + Preview string `json:"preview"` + RPM int `json:"rpm,omitempty"` + Concurrency int `json:"concurrency,omitempty"` + MaxActiveSessions int `json:"max_active_sessions,omitempty"` + Models []string `json:"models"` + Prices map[string]ModelPrice `json:"prices,omitempty"` + DailyLimitUSD *float64 `json:"daily_limit_usd,omitempty"` + WeeklyLimitUSD *float64 `json:"weekly_limit_usd,omitempty"` + FiveHourUSD *float64 `json:"five_hour_usd,omitempty"` + MonthlyLimitUSD *float64 `json:"monthly_limit_usd,omitempty"` + Archived bool `json:"archived,omitempty"` + ArchivedAt int64 `json:"archived_at,omitempty"` + Source string `json:"source,omitempty"` + SourcePresent bool `json:"source_present"` + Alias string `json:"alias,omitempty"` + InheritedFrom string `json:"inherited_from,omitempty"` + InheritConflict bool `json:"inherit_conflict,omitempty"` + Hidden bool `json:"hidden,omitempty"` + LastEnabled bool `json:"last_enabled,omitempty"` +} + +func (k KeyRecord) Safe() map[string]any { + sourcePresent := k.SourcePresent + if k.Source == "" { + sourcePresent = true + } + return map[string]any{ + "id": k.ID, + "name": k.Name, + "enabled": k.Enabled, + "preview": k.Preview, + "rpm": k.RPM, + "concurrency": 0, + "max_active_sessions": 0, + "models": append([]string(nil), k.Models...), + "archived": k.Archived, + "archived_at": k.ArchivedAt, + "source": k.Source, + "source_present": sourcePresent, + "alias": k.Alias, + "inherited_from": k.InheritedFrom, + "inherit_conflict": k.InheritConflict, + "hidden": k.Hidden, + "last_enabled": k.LastEnabled, + "limits": map[string]any{ + "five_hour_usd": k.FiveHourUSD, + "daily_usd": k.DailyLimitUSD, + "weekly_usd": k.WeeklyLimitUSD, + "monthly_usd": k.MonthlyLimitUSD, + }, + "pricing": map[string]any{ + "models": k.Prices, + }, + } +} + +type KeyPolicyState struct { + Keys []KeyRecord +} + +func ValidateKeyRecord(key KeyRecord) error { + if strings.TrimSpace(key.ID) == "" { + return fmt.Errorf("missing key id") + } + if key.RPM < 0 || key.Concurrency < 0 || key.MaxActiveSessions < 0 { + return fmt.Errorf("limits must not be negative") + } + for _, limit := range []*float64{key.FiveHourUSD, key.DailyLimitUSD, key.WeeklyLimitUSD, key.MonthlyLimitUSD} { + if limit != nil && *limit < 0 { + return fmt.Errorf("usd limits must not be negative") + } + } + for name, price := range key.Prices { + if price.InputPerMillion < 0 || price.OutputPerMillion < 0 || + price.CacheReadPerMillion < 0 || price.CacheCreationPerMillion < 0 { + return fmt.Errorf("model price must not be negative: %s", name) + } + } + return nil +} + +func LoadKeyPolicyState(path string) (KeyPolicyState, error) { + raw, err := os.ReadFile(path) + if err != nil { + return KeyPolicyState{}, err + } + var data any + if err := json.Unmarshal(raw, &data); err != nil { + return KeyPolicyState{}, err + } + keys := extractKeys(data) + out := make([]KeyRecord, 0, len(keys)) + for _, rawKey := range keys { + if key, ok := parseKey(rawKey); ok { + out = append(out, key) + } + } + return KeyPolicyState{Keys: out}, nil +} + +func (s KeyPolicyState) FindByRawKey(rawKey string) (KeyRecord, bool) { + hash := SHA256Hex(rawKey) + return s.FindByRawHash(hash) +} + +func (s KeyPolicyState) FindByRawHash(hash string) (KeyRecord, bool) { + normalized, err := NormalizeHash(hash) + if err != nil { + return KeyRecord{}, false + } + for _, key := range s.Keys { + keyHash, err := NormalizeHash(key.KeyHash) + if err == nil && keyHash == normalized { + return key, true + } + } + return KeyRecord{}, false +} + +func extractKeys(data any) []map[string]any { + if arr, ok := data.([]any); ok { + return mapsFromArray(arr) + } + obj, ok := data.(map[string]any) + if !ok { + return nil + } + for _, path := range [][]string{{"keys"}, {"state", "keys"}, {"data", "keys"}, {"config", "keys"}} { + var cur any = obj + for _, part := range path { + m, ok := cur.(map[string]any) + if !ok { + cur = nil + break + } + cur = m[part] + } + if arr, ok := cur.([]any); ok { + return mapsFromArray(arr) + } + } + return nil +} + +func mapsFromArray(arr []any) []map[string]any { + out := make([]map[string]any, 0, len(arr)) + for _, item := range arr { + if m, ok := item.(map[string]any); ok { + out = append(out, m) + } + } + return out +} + +const ( + NativeCPASource = "native_cpa" + LegacyPlusSource = "legacy_plus" +) + +type NativeKeySyncInput struct { + RawKey string + Alias string +} + +func NativeKeyIDFromHash(hash string) string { + return "native_" + sanitizeIDPart(HashPreview(hash)) +} + +func NativeKeyPreviewFromHash(hash string) string { + return HashPreview(hash) +} + +func NativeKeyRecord(rawKey, alias string) (KeyRecord, bool) { + rawKey = NormalizeSubmittedKey(rawKey) + if rawKey == "" { + return KeyRecord{}, false + } + hash := SHA256Hex(rawKey) + normalized, err := NormalizeHash(hash) + if err != nil { + return KeyRecord{}, false + } + alias = strings.TrimSpace(alias) + preview := NativeKeyPreviewFromHash(normalized) + name := alias + if name == "" { + name = preview + } + return KeyRecord{ + ID: NativeKeyIDFromHash(normalized), + Name: name, + KeyHash: "sha256:" + normalized, + Enabled: true, + Preview: preview, + Source: NativeCPASource, + SourcePresent: true, + Alias: alias, + Hidden: false, + }, true +} + +func CopyPolicyFields(dst, src KeyRecord) KeyRecord { + dst.Enabled = src.Enabled + if (src.Source == NativeCPASource || src.Source == LegacyPlusSource) && !src.SourcePresent { + dst.Enabled = src.LastEnabled + } + dst.RPM = src.RPM + dst.Concurrency = 0 + dst.MaxActiveSessions = 0 + dst.Models = append([]string(nil), src.Models...) + dst.Prices = clonePrices(src.Prices) + dst.FiveHourUSD = cloneFloatPtr(src.FiveHourUSD) + dst.DailyLimitUSD = cloneFloatPtr(src.DailyLimitUSD) + dst.WeeklyLimitUSD = cloneFloatPtr(src.WeeklyLimitUSD) + dst.MonthlyLimitUSD = cloneFloatPtr(src.MonthlyLimitUSD) + return dst +} + +func clonePrices(in map[string]ModelPrice) map[string]ModelPrice { + if in == nil { + return nil + } + out := make(map[string]ModelPrice, len(in)) + for key, value := range in { + out[key] = value + } + return out +} + +func cloneFloatPtr(in *float64) *float64 { + if in == nil { + return nil + } + out := *in + return &out +} + +var idPartCleanup = regexp.MustCompile(`[^A-Za-z0-9_-]+`) + +func sanitizeIDPart(value string) string { + value = strings.ReplaceAll(value, "...", "_") + value = strings.ReplaceAll(value, "…", "_") + value = idPartCleanup.ReplaceAllString(value, "_") + value = strings.Trim(value, "_") + if value == "" { + return "unknown" + } + return value +} + +func parseKey(raw map[string]any) (KeyRecord, bool) { + rawHash := firstString(raw, "key_hash", "keyHash", "hash", "api_key_hash", "apiKeyHash") + if rawHash == "" { + return KeyRecord{}, false + } + normalized, err := NormalizeHash(rawHash) + if err != nil { + return KeyRecord{}, false + } + id := firstString(raw, "id", "key_id", "keyId") + if id == "" { + id = HashPreview(normalized) + } + name := firstString(raw, "name", "label", "alias", "description") + if name == "" { + name = id + } + enabled, hasEnabled := firstBool(raw, "enabled", "is_enabled", "isEnabled") + disabled, _ := firstBool(raw, "disabled", "is_disabled", "isDisabled") + if !hasEnabled { + enabled = !disabled + } + archived, _ := firstBool(raw, "archived", "is_archived", "isArchived") + modelItems := asList(firstAny(raw, "models", "allowed_models", "allowedModels", "model_allowlist", "modelAllowlist", "aliases")) + models := parseModels(modelItems) + prices := parsePrices(raw, modelItems) + return KeyRecord{ + ID: strings.TrimSpace(id), + Name: strings.TrimSpace(name), + KeyHash: "sha256:" + normalized, + Enabled: enabled && !disabled, + Preview: firstNonEmpty(firstString(raw, "preview", "key_preview", "keyPreview"), HashPreview(normalized)), + RPM: firstInt(raw, "rpm", "rpm_limit", "rpmLimit", "rpm_per_minute", "rpmPerMinute"), + Concurrency: firstInt(raw, "concurrency", "concurrency_limit", "concurrencyLimit", "max_concurrent", "maxConcurrent", "request_concurrency", "requestConcurrency"), + MaxActiveSessions: firstInt(raw, "max_active_sessions", "maxActiveSessions", "max_sessions", "maxSessions", "session_limit", "sessionLimit", "max_codex_windows", "maxCodexWindows"), + Models: models, + Prices: prices, + DailyLimitUSD: firstFloatPtr(raw, "daily_limit_usd", "dailyLimitUsd", "daily_limit", "dailyLimit", "daily_usd", "dailyUsd"), + WeeklyLimitUSD: firstFloatPtr(raw, "weekly_limit_usd", "weeklyLimitUsd", "weekly_limit", "weeklyLimit", "weekly_usd", "weeklyUsd"), + FiveHourUSD: firstFloatPtr(raw, "five_hour_limit_usd", "fiveHourLimitUsd", "five_hour_usd", "fiveHourUsd", "5h_limit_usd"), + MonthlyLimitUSD: firstFloatPtr(raw, "monthly_limit_usd", "monthlyLimitUsd", "monthly_usd", "monthlyUsd", "month_limit_usd"), + Archived: archived, + }, true +} + +func parseModels(items []any) []string { + seen := map[string]bool{} + var out []string + for _, item := range items { + name := "" + if m, ok := item.(map[string]any); ok { + name = modelName(m) + } else { + name = strings.TrimSpace(toString(item)) + } + if name != "" && !seen[name] { + seen[name] = true + out = append(out, name) + } + } + return out +} + +func ModelOptionsFromIDs(ids []string, source string, known bool) []ModelOption { + out := make([]ModelOption, 0, len(ids)) + seen := map[string]bool{} + for _, id := range ids { + id = strings.TrimSpace(id) + if id == "" { + continue + } + key := strings.ToLower(id) + if seen[key] { + continue + } + seen[key] = true + out = append(out, ModelOption{ID: id, DisplayName: id, Source: source, Known: known}) + } + return out +} + +func NormalizeModelOptions(data any, source string) []ModelOption { + items := modelOptionItems(data) + out := make([]ModelOption, 0, len(items)) + seen := map[string]bool{} + for _, item := range items { + option, ok := parseModelOption(item, source) + if !ok { + continue + } + key := strings.ToLower(option.ID) + if seen[key] { + continue + } + seen[key] = true + out = append(out, option) + } + return out +} + +func MergeModelOptions(groups ...[]ModelOption) []ModelOption { + out := []ModelOption{} + byID := map[string]int{} + for _, group := range groups { + for _, item := range group { + item.ID = strings.TrimSpace(item.ID) + if item.ID == "" { + continue + } + if strings.TrimSpace(item.DisplayName) == "" { + item.DisplayName = item.ID + } + key := strings.ToLower(item.ID) + if idx, ok := byID[key]; ok { + existing := out[idx] + if existing.DisplayName == existing.ID && item.DisplayName != "" { + existing.DisplayName = item.DisplayName + } + if existing.Type == "" { + existing.Type = item.Type + } + if existing.OwnedBy == "" { + existing.OwnedBy = item.OwnedBy + } + if existing.Source == "" { + existing.Source = item.Source + } + existing.Known = existing.Known || item.Known + out[idx] = existing + continue + } + byID[key] = len(out) + out = append(out, item) + } + } + return out +} + +func modelOptionItems(data any) []any { + switch v := data.(type) { + case nil: + return nil + case []any: + return v + case []string: + out := make([]any, len(v)) + for i := range v { + out[i] = v[i] + } + return out + case []ModelOption: + out := make([]any, len(v)) + for i := range v { + out[i] = v[i] + } + return out + case map[string]any: + for _, name := range []string{"models", "data", "items", "result"} { + if raw, ok := v[name]; ok { + if items := modelOptionItems(raw); len(items) > 0 { + return items + } + } + } + out := make([]any, 0, len(v)) + for id, raw := range v { + if m, ok := raw.(map[string]any); ok { + if _, hasID := m["id"]; !hasID { + m["id"] = id + } + out = append(out, m) + continue + } + out = append(out, id) + } + return out + default: + text := strings.TrimSpace(toString(v)) + if text == "" { + return nil + } + return []any{text} + } +} + +func parseModelOption(item any, source string) (ModelOption, bool) { + switch v := item.(type) { + case ModelOption: + v.ID = strings.TrimSpace(v.ID) + if v.ID == "" { + return ModelOption{}, false + } + if strings.TrimSpace(v.DisplayName) == "" { + v.DisplayName = v.ID + } + if v.Source == "" { + v.Source = source + } + return v, true + case string: + id := strings.TrimSpace(v) + if id == "" { + return ModelOption{}, false + } + return ModelOption{ID: id, DisplayName: id, Source: source, Known: true}, true + case map[string]any: + id := modelName(v) + if id == "" { + return ModelOption{}, false + } + display := firstString(v, "display_name", "displayName", "label", "name") + if display == "" { + display = id + } + known, hasKnown := firstBool(v, "known") + if !hasKnown { + known = true + } + src := firstString(v, "source") + if src == "" { + src = source + } + return ModelOption{ + ID: id, + DisplayName: display, + Type: firstString(v, "type", "object"), + OwnedBy: firstString(v, "owned_by", "ownedBy", "provider"), + Source: src, + Known: known, + }, true + default: + id := strings.TrimSpace(toString(v)) + if id == "" { + return ModelOption{}, false + } + return ModelOption{ID: id, DisplayName: id, Source: source, Known: true}, true + } +} + +func parsePrices(raw map[string]any, modelItems []any) map[string]ModelPrice { + out := map[string]ModelPrice{} + for _, item := range modelItems { + if m, ok := item.(map[string]any); ok { + if p, ok := parsePriceEntry(m, ""); ok { + out[p.Model] = p + } + } + } + priceRaw := firstAny(raw, "model_prices", "modelPrices", "prices") + switch v := priceRaw.(type) { + case map[string]any: + for name, item := range v { + if m, ok := item.(map[string]any); ok { + if p, ok := parsePriceEntry(m, name); ok { + out[p.Model] = p + } + } + } + case []any: + for _, item := range v { + if m, ok := item.(map[string]any); ok { + if p, ok := parsePriceEntry(m, ""); ok { + out[p.Model] = p + } + } + } + } + return out +} + +func parsePriceEntry(raw map[string]any, defaultModel string) (ModelPrice, bool) { + model := modelName(raw) + if model == "" { + model = strings.TrimSpace(defaultModel) + } + if model == "" { + return ModelPrice{}, false + } + price := ModelPrice{ + Model: model, + TargetModel: firstString(raw, "target_model", "targetModel", "upstream_model", "upstreamModel"), + Provider: firstString(raw, "provider", "type"), + InputPerMillion: firstFloat(raw, "input_price_per_million", "inputPricePerMillion", "input", "prompt", "prompt_price_per_million"), + OutputPerMillion: firstFloat(raw, "output_price_per_million", "outputPricePerMillion", "output", "completion", "completion_price_per_million"), + CacheReadPerMillion: firstFloat(raw, "cache_read_price_per_million", "cacheReadPricePerMillion", "cache_price_per_million", "cachePricePerMillion", "cache_read", "cacheRead", "cache"), + CacheCreationPerMillion: firstFloat(raw, "cache_creation_price_per_million", "cacheCreationPricePerMillion", "cache_write_price_per_million", "cacheWritePricePerMillion", "cache_creation", "cacheCreation", "cache_write", "cacheWrite"), + } + if price.InputPerMillion <= 0 && price.OutputPerMillion <= 0 && price.CacheReadPerMillion <= 0 && price.CacheCreationPerMillion <= 0 { + return ModelPrice{}, false + } + return price, true +} + +func modelName(raw map[string]any) string { + return firstString(raw, "alias", "model", "name", "id", "target_model", "targetModel", "upstream_model", "upstreamModel") +} + +func firstAny(raw map[string]any, names ...string) any { + for _, name := range names { + if value, ok := raw[name]; ok { + return value + } + } + return nil +} + +func firstString(raw map[string]any, names ...string) string { + for _, name := range names { + if value, ok := raw[name]; ok { + text := strings.TrimSpace(toString(value)) + if text != "" { + return text + } + } + } + return "" +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func firstBool(raw map[string]any, names ...string) (bool, bool) { + for _, name := range names { + if value, ok := raw[name]; ok { + switch v := value.(type) { + case bool: + return v, true + case string: + switch strings.ToLower(strings.TrimSpace(v)) { + case "true", "1", "yes", "enabled": + return true, true + case "false", "0", "no", "disabled": + return false, true + } + } + } + } + return false, false +} + +func firstInt(raw map[string]any, names ...string) int { + for _, name := range names { + if value, ok := raw[name]; ok { + switch v := value.(type) { + case float64: + return int(v) + case int: + return v + case string: + var n int + if err := json.NewDecoder(strings.NewReader(v)).Decode(&n); err == nil { + return n + } + } + } + } + return 0 +} + +func firstFloat(raw map[string]any, names ...string) float64 { + ptr := firstFloatPtr(raw, names...) + if ptr == nil { + return 0 + } + return *ptr +} + +func firstFloatPtr(raw map[string]any, names ...string) *float64 { + for _, name := range names { + if value, ok := raw[name]; ok { + switch v := value.(type) { + case float64: + return &v + case int: + f := float64(v) + return &f + case string: + var f float64 + if err := json.NewDecoder(strings.NewReader(v)).Decode(&f); err == nil { + return &f + } + } + } + } + return nil +} + +func asList(value any) []any { + switch v := value.(type) { + case []any: + return v + case []string: + out := make([]any, len(v)) + for i := range v { + out[i] = v[i] + } + return out + case string: + parts := strings.Split(v, ",") + out := make([]any, 0, len(parts)) + for _, part := range parts { + if text := strings.TrimSpace(part); text != "" { + out = append(out, text) + } + } + return out + default: + return nil + } +} + +func toString(value any) string { + switch v := value.(type) { + case string: + return v + case json.Number: + return v.String() + default: + if value == nil { + return "" + } + return strings.TrimSpace(fmt.Sprint(value)) + } +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/policyplus_test.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/policyplus_test.go new file mode 100644 index 0000000..17d9ce6 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/policyplus_test.go @@ -0,0 +1,1010 @@ +package policyplus + +import ( + "context" + "database/sql" + "encoding/json" + "errors" + "net/http" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +func ptr(v float64) *float64 { return &v } + +func TestSQLiteOpenStoreUsesSingleConnectionAndBusyTimeout(t *testing.T) { + ctx := context.Background() + path := filepath.Join(t.TempDir(), "policyplus.sqlite") + store, err := OpenStore(path) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if got := store.db.Stats().MaxOpenConnections; got != 1 { + t.Fatalf("main store should use one DB connection, got %d", got) + } + var timeout int + if err := store.db.QueryRowContext(ctx, `pragma busy_timeout`).Scan(&timeout); err != nil { + t.Fatal(err) + } + if timeout != sqliteBusyTimeoutMS { + t.Fatalf("busy_timeout=%d want %d", timeout, sqliteBusyTimeoutMS) + } +} + +func TestSQLiteBusyTimeoutWaitsForTransientWriteLock(t *testing.T) { + ctx := context.Background() + path := filepath.Join(t.TempDir(), "policyplus.sqlite") + store, err := OpenStore(path) + if err != nil { + t.Fatal(err) + } + defer store.Close() + locker, err := openSQLite(path, false) + if err != nil { + t.Fatal(err) + } + defer locker.Close() + tx, err := locker.BeginTx(ctx, nil) + if err != nil { + t.Fatal(err) + } + if _, err := tx.ExecContext(ctx, `insert into keys(id, name, key_hash, enabled, updated_at) values('lock-row', 'Lock', 'sha256:lock', 1, ?)`, time.Now().Unix()); err != nil { + t.Fatal(err) + } + done := make(chan error, 1) + go func() { + done <- store.UpsertKey(ctx, KeyRecord{ + ID: "native-wait", + Name: "Wait", + KeyHash: "sha256:" + SHA256Hex("sk-wait"), + Enabled: true, + Preview: HashPreview(SHA256Hex("sk-wait")), + Source: NativeCPASource, + SourcePresent: true, + }) + }() + time.Sleep(150 * time.Millisecond) + if err := tx.Commit(); err != nil { + t.Fatal(err) + } + select { + case err := <-done: + if err != nil { + if strings.Contains(err.Error(), "SQLITE_BUSY") || strings.Contains(err.Error(), "database is locked") { + t.Fatalf("operation should wait for transient lock, got %v", err) + } + t.Fatal(err) + } + case <-time.After(3 * time.Second): + t.Fatal("operation did not complete after lock release") + } +} + +func TestReadOnlySQLiteHelperRejectsWrites(t *testing.T) { + ctx := context.Background() + path := filepath.Join(t.TempDir(), "aliases.sqlite") + writer, err := sql.Open("sqlite", path) + if err != nil { + t.Fatal(err) + } + if _, err := writer.ExecContext(ctx, `create table api_key_aliases(api_key_hash text primary key, alias text)`); err != nil { + t.Fatal(err) + } + if err := writer.Close(); err != nil { + t.Fatal(err) + } + db, err := openSQLite(path, true) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if _, err := db.ExecContext(ctx, `insert into api_key_aliases(api_key_hash, alias) values('sha256:x', 'x')`); err == nil { + t.Fatal("read-only helper should reject writes") + } else if !errors.Is(err, sql.ErrNoRows) && !strings.Contains(strings.ToLower(err.Error()), "readonly") && !strings.Contains(strings.ToLower(err.Error()), "read-only") { + t.Fatalf("write rejected with unexpected error: %v", err) + } +} + +func writePolicyState(t *testing.T, dir string, rawKey string) string { + t.Helper() + path := filepath.Join(dir, "key-policy.json") + body := map[string]any{ + "keys": []map[string]any{{ + "id": "alice-key", + "name": "Alice", + "key_hash": "sha256:" + SHA256Hex(rawKey), + "enabled": true, + "rpm": 12, + "models": []map[string]any{{ + "alias": "gpt-5.5", + "target_model": "gpt-5.5", + "input_price_per_million": 5, + "output_price_per_million": 30, + "cache_read_price_per_million": 0.5, + }}, + "daily_limit_usd": 5, + "weekly_limit_usd": 30, + }}, + } + raw, err := json.Marshal(body) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, raw, 0o600); err != nil { + t.Fatal(err) + } + return path +} + +func TestKeyPolicyStateParsesSafeRecords(t *testing.T) { + dir := t.TempDir() + path := writePolicyState(t, dir, "cpa_live") + state, err := LoadKeyPolicyState(path) + if err != nil { + t.Fatal(err) + } + key, ok := state.FindByRawKey(" cpa_live ") + if !ok { + t.Fatal("raw key did not match policy state") + } + if key.ID != "alice-key" || key.Name != "Alice" || !key.Enabled { + t.Fatalf("unexpected key: %#v", key) + } + if len(key.Models) != 1 || key.Models[0] != "gpt-5.5" { + t.Fatalf("models = %#v", key.Models) + } + price, ok := PriceForModel(key.Prices, "GPT-5.5") + if !ok || price.InputPerMillion != 5 || price.OutputPerMillion != 30 || price.CacheReadPerMillion != 0.5 { + t.Fatalf("price = %#v ok=%v", price, ok) + } + safe := key.Safe() + encoded, _ := json.Marshal(safe) + if strings.Contains(string(encoded), SHA256Hex("cpa_live")) || strings.Contains(string(encoded), "cpa_live") { + t.Fatalf("safe projection leaked key material: %s", encoded) + } +} + +func TestNormalizeModelOptionsAcceptsCommonPayloadShapes(t *testing.T) { + payload := map[string]any{ + "models": []any{ + map[string]any{"id": "gpt-5.5", "display_name": "GPT 5.5", "owned_by": "openai"}, + map[string]any{"model": "gpt-5.4"}, + "gpt-5.5", + map[string]any{"alias": "codex-auto-review", "target_model": "gpt-5.5"}, + }, + } + options := NormalizeModelOptions(payload, "online") + if len(options) != 3 { + t.Fatalf("options = %#v", options) + } + if options[0].ID != "gpt-5.5" || options[0].DisplayName != "GPT 5.5" || options[0].OwnedBy != "openai" { + t.Fatalf("first option = %#v", options[0]) + } + if options[2].ID != "codex-auto-review" { + t.Fatalf("alias model was not parsed: %#v", options) + } +} + +func TestMergeModelOptionsPreservesUnknownConfiguredModels(t *testing.T) { + known := NormalizeModelOptions([]any{map[string]any{"id": "gpt-5.5", "display_name": "GPT 5.5"}}, "online") + configured := ModelOptionsFromIDs([]string{"gpt-5.5", "legacy-custom"}, "plus_configured", false) + merged := MergeModelOptions(known, configured) + if len(merged) != 2 { + t.Fatalf("merged = %#v", merged) + } + if !merged[0].Known || merged[0].DisplayName != "GPT 5.5" { + t.Fatalf("known metadata should win: %#v", merged[0]) + } + if merged[1].ID != "legacy-custom" || merged[1].Known { + t.Fatalf("unknown configured model should be preserved: %#v", merged[1]) + } +} + +func TestImportKeysDoesNotDeletePlusNativeKeys(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + native := KeyRecord{ + ID: "native-plus-key", + Name: "Native", + KeyHash: "sha256:" + SHA256Hex("cpa_native_plus"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa_native_plus")), + } + if err := store.UpsertKey(ctx, native); err != nil { + t.Fatal(err) + } + imported := KeyRecord{ + ID: "imported-key", + Name: "Imported", + KeyHash: "sha256:" + SHA256Hex("cpa_imported"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa_imported")), + } + if err := store.ImportKeys(ctx, KeyPolicyState{Keys: []KeyRecord{imported}}); err != nil { + t.Fatal(err) + } + keys, err := store.ListKeys(ctx) + if err != nil { + t.Fatal(err) + } + seen := map[string]bool{} + for _, key := range keys { + seen[key.ID] = true + } + if !seen[native.ID] || !seen[imported.ID] { + t.Fatalf("native/imported keys should both remain: %#v", seen) + } +} + +func TestPricingBreakdownUsesPerMillionAndCachedInput(t *testing.T) { + price := ModelPrice{ + Model: "gpt-5.5", + InputPerMillion: 5, + OutputPerMillion: 30, + CacheReadPerMillion: 0.5, + } + breakdown := CostForUsage(price, TokenUsage{ + InputTokens: 100, + CachedTokens: 20, + OutputTokens: 50, + ReasoningTokens: 30, + TotalTokens: 150, + }, "gpt-5.5") + if got := breakdown.Tokens["billable_uncached_input"]; got != 80 { + t.Fatalf("billable input = %d", got) + } + if got := breakdown.Tokens["visible_output_estimate"]; got != 20 { + t.Fatalf("visible output estimate = %d", got) + } + if breakdown.Costs["total"] <= 0 { + t.Fatalf("total cost should be positive: %#v", breakdown.Costs) + } + if breakdown.Costs["cached_input"] <= 0 { + t.Fatalf("cached input should be charged with cache read price: %#v", breakdown.Costs) + } +} + +func TestStoreUsageWindowsAndSoftReset(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + key := KeyRecord{ + ID: "alice-key", + Name: "Alice", + KeyHash: "sha256:" + SHA256Hex("cpa_live"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa_live")), + FiveHourUSD: ptr(1), + MonthlyLimitUSD: ptr(10), + } + if err := store.UpsertKey(ctx, key); err != nil { + t.Fatal(err) + } + now := time.Now().UTC() + if err := store.InsertUsage(ctx, UsageEvent{ + RequestID: "req-old", + KeyID: "alice-key", + RequestedAt: now.Add(-2 * time.Hour), + Cost: 0.5, + }); err != nil { + t.Fatal(err) + } + if err := store.InsertUsage(ctx, UsageEvent{ + RequestID: "req-new", + KeyID: "alice-key", + RequestedAt: now.Add(-30 * time.Minute), + Cost: 0.25, + }); err != nil { + t.Fatal(err) + } + used, err := store.UsageSum(ctx, "alice-key", WindowFor(Range5H, now)) + if err != nil { + t.Fatal(err) + } + if used != 0.75 { + t.Fatalf("used before reset = %v", used) + } + if err := store.Reset(ctx, "alice-key", Range5H, now.Add(-time.Hour)); err != nil { + t.Fatal(err) + } + used, err = store.UsageSum(ctx, "alice-key", WindowFor(Range5H, now)) + if err != nil { + t.Fatal(err) + } + if used != 0.25 { + t.Fatalf("used after reset = %v", used) + } + summary, err := store.UsageSummary(ctx, "alice-key", WindowFor(Range5H, now)) + if err != nil { + t.Fatal(err) + } + if summary.Calls != 1 || summary.TotalCost != 0.25 { + t.Fatalf("summary after reset = %#v", summary) + } +} + +func TestStoreDeleteKeyRemovesConfigAndKeepsHistory(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + key := KeyRecord{ + ID: "alice-key", + Name: "Alice", + KeyHash: "sha256:" + SHA256Hex("cpa_live"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa_live")), + } + if err := store.UpsertKey(ctx, key); err != nil { + t.Fatal(err) + } + now := time.Now() + if err := store.Reset(ctx, key.ID, Range5H, now.Add(-time.Hour)); err != nil { + t.Fatal(err) + } + if _, err := store.RegisterActiveSession(ctx, key.ID, newSessionIdentity("test", "window-a"), 3, DefaultSessionIdle, now); err != nil { + t.Fatal(err) + } + if err := store.InsertUsage(ctx, UsageEvent{ + RequestID: "req-delete-kept", + KeyID: key.ID, + RequestedAt: now, + Cost: 0.25, + }); err != nil { + t.Fatal(err) + } + if err := store.SaveCodexSummary(ctx, "codex-delete-kept", key.ID, "gpt-5.5", "protected_clean", map[string]any{ + "request_id": "codex-delete-kept", + "started_at": now.Format(time.RFC3339), + }); err != nil { + t.Fatal(err) + } + + if err := store.DeleteKey(ctx, key.ID); err != nil { + t.Fatal(err) + } + keys, err := store.ListKeys(ctx) + if err != nil || len(keys) != 0 { + t.Fatalf("keys err=%v keys=%#v", err, keys) + } + usage, err := store.UsageSummary(ctx, key.ID, WindowFor(Range24H, now)) + if err != nil { + t.Fatal(err) + } + if usage.Calls != 1 || usage.TotalCost != 0.25 { + t.Fatalf("usage history should be retained after delete: %#v", usage) + } + codex, err := store.RecentCodexSummaries(ctx, key.ID, 10) + if err != nil { + t.Fatal(err) + } + if len(codex) != 1 || codex[0].RequestID != "codex-delete-kept" { + t.Fatalf("codex history should be retained after delete: %#v", codex) + } + var resetCount, activeCount, auditCount int + if err := store.db.QueryRowContext(ctx, `select count(1) from reset_watermarks where key_id=?`, key.ID).Scan(&resetCount); err != nil { + t.Fatal(err) + } + if err := store.db.QueryRowContext(ctx, `select count(1) from active_sessions where key_id=?`, key.ID).Scan(&activeCount); err != nil { + t.Fatal(err) + } + if err := store.db.QueryRowContext(ctx, `select count(1) from audit_log where target=? and action='delete_key'`, key.ID).Scan(&auditCount); err != nil { + t.Fatal(err) + } + if resetCount != 0 || activeCount != 0 || auditCount != 1 { + t.Fatalf("delete cleanup reset=%d active=%d audit=%d", resetCount, activeCount, auditCount) + } +} + +func TestStoreImportsLegacyQuotaSQLite(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + plus, err := OpenStore(filepath.Join(dir, "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer plus.Close() + key := KeyRecord{ + ID: "alice-key", + Name: "Alice", + KeyHash: "sha256:" + SHA256Hex("cpa_live"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa_live")), + } + if err := plus.UpsertKey(ctx, key); err != nil { + t.Fatal(err) + } + legacyPath := filepath.Join(dir, "usage-admin.sqlite") + legacy, err := sql.Open("sqlite", legacyPath) + if err != nil { + t.Fatal(err) + } + if _, err := legacy.Exec(`create table key_limits(policy_id text primary key, five_hour_limit_usd real, monthly_limit_usd real, updated_at_ms integer)`); err != nil { + t.Fatal(err) + } + if _, err := legacy.Exec(`create table reset_watermarks(policy_id text, window text, reset_at_ms integer, updated_at_ms integer, primary key(policy_id, window))`); err != nil { + t.Fatal(err) + } + if _, err := legacy.Exec(`insert into key_limits(policy_id, five_hour_limit_usd, monthly_limit_usd, updated_at_ms) values('alice-key', 1.5, 20, 1000)`); err != nil { + t.Fatal(err) + } + if _, err := legacy.Exec(`insert into reset_watermarks(policy_id, window, reset_at_ms, updated_at_ms) values('alice-key', '5h', 2000000000000, 2000000000000)`); err != nil { + t.Fatal(err) + } + if err := legacy.Close(); err != nil { + t.Fatal(err) + } + result, err := plus.ImportLegacyQuotaSQLite(ctx, legacyPath, "usage-admin") + if err != nil { + t.Fatal(err) + } + if result.Limits != 1 || result.Resets != 1 { + t.Fatalf("import result = %#v", result) + } + keys, err := plus.ListKeys(ctx) + if err != nil || len(keys) != 1 { + t.Fatalf("keys err=%v keys=%#v", err, keys) + } + if keys[0].FiveHourUSD == nil || *keys[0].FiveHourUSD != 1.5 || keys[0].MonthlyLimitUSD == nil || *keys[0].MonthlyLimitUSD != 20 { + t.Fatalf("legacy limits not imported: %#v", keys[0]) + } + resetAt, ok := plus.ResetAt(ctx, "alice-key", Range5H) + if !ok || resetAt != 2000000000 { + t.Fatalf("legacy reset not imported: resetAt=%d ok=%v", resetAt, ok) + } +} + +func TestNativeKeyLoadersReadCPAConfigAndCPAMPAliases(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + configPath := filepath.Join(dir, "config.yaml") + if err := os.WriteFile(configPath, []byte("api-keys:\n - sk-native-one\n - ' sk-native-two '\n"), 0o600); err != nil { + t.Fatal(err) + } + keys, err := LoadNativeKeysFromCPAConfig(configPath) + if err != nil { + t.Fatal(err) + } + if len(keys) != 2 || keys[0] != "sk-native-one" || keys[1] != "sk-native-two" { + t.Fatalf("native keys = %#v", keys) + } + + dbPath := filepath.Join(dir, "cpamp.sqlite") + db, err := sql.Open("sqlite", dbPath) + if err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`create table api_key_aliases(api_key_hash text primary key, alias text, updated_at_ms integer)`); err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`insert into api_key_aliases(api_key_hash, alias, updated_at_ms) values(?, ?, ?)`, "sha256:"+SHA256Hex("sk-native-one"), "QQ专用", 1); err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`insert into api_key_aliases(api_key_hash, alias, updated_at_ms) values(?, ?, ?)`, SHA256Hex("sk-native-two"), "阿伟专用", 2); err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`insert into api_key_aliases(api_key_hash, alias, updated_at_ms) values(?, ?, ?)`, "SHA256:"+SHA256Hex("sk-native-three"), "Kuma专用", 3); err != nil { + t.Fatal(err) + } + if err := db.Close(); err != nil { + t.Fatal(err) + } + aliases, err := LoadAPIKeyAliasesFromSQLite(ctx, dbPath) + if err != nil { + t.Fatal(err) + } + if aliases[SHA256Hex("sk-native-one")] != "QQ专用" { + t.Fatalf("aliases = %#v", aliases) + } + if aliases[SHA256Hex("sk-native-two")] != "阿伟专用" { + t.Fatalf("bare-hash aliases = %#v", aliases) + } + if aliases[SHA256Hex("sk-native-three")] != "Kuma专用" { + t.Fatalf("case-insensitive prefix aliases = %#v", aliases) + } +} + +func TestNativeAliasLoaderFallsBackAcrossSQLitePaths(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + emptyPath := filepath.Join(dir, "empty.sqlite") + emptyDB, err := sql.Open("sqlite", emptyPath) + if err != nil { + t.Fatal(err) + } + if _, err := emptyDB.Exec(`create table unrelated(id text)`); err != nil { + t.Fatal(err) + } + if err := emptyDB.Close(); err != nil { + t.Fatal(err) + } + + aliasPath := filepath.Join(dir, "cpamp-real.sqlite") + aliasDB, err := sql.Open("sqlite", aliasPath) + if err != nil { + t.Fatal(err) + } + if _, err := aliasDB.Exec(`create table api_key_aliases(api_key_hash text primary key, alias text, updated_at_ms integer)`); err != nil { + t.Fatal(err) + } + if _, err := aliasDB.Exec(`insert into api_key_aliases(api_key_hash, alias, updated_at_ms) values(?, ?, ?)`, "sha256:"+SHA256Hex("sk-alice"), "alicea", 10); err != nil { + t.Fatal(err) + } + if err := aliasDB.Close(); err != nil { + t.Fatal(err) + } + + emptyAliases, err := LoadAPIKeyAliasesFromSQLite(ctx, emptyPath) + if err != nil { + t.Fatal(err) + } + if len(emptyAliases) != 0 { + t.Fatalf("missing alias table should be an empty source, got %#v", emptyAliases) + } + aliases, errs, err := LoadAPIKeyAliasesFromSQLitePaths(ctx, []string{emptyPath, aliasPath}) + if err != nil || len(errs) != 0 { + t.Fatalf("fallback alias load err=%v errs=%#v", err, errs) + } + if aliases[SHA256Hex("sk-alice")] != "alicea" { + t.Fatalf("fallback alias missing: %#v", aliases) + } +} + +func TestCheckLimitTreatsZeroAsExplicitLimit(t *testing.T) { + if decision := CheckLimit(0, nil); !decision.Allowed { + t.Fatalf("nil limit should mean unlimited: %#v", decision) + } + zero := 0.0 + if decision := CheckLimit(0, &zero); decision.Allowed || decision.LimitUSD == nil || *decision.LimitUSD != 0 { + t.Fatalf("zero limit should deny post-accounting requests: %#v", decision) + } + positive := 1.0 + if decision := CheckLimit(0.5, &positive); !decision.Allowed { + t.Fatalf("below positive limit should pass: %#v", decision) + } + if decision := CheckLimit(1, &positive); decision.Allowed { + t.Fatalf("used >= positive limit should deny: %#v", decision) + } +} + +func TestSyncNativeKeysLifecycleInheritanceAndHistory(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + + if err := store.SyncNativeKeys(ctx, []NativeKeySyncInput{{RawKey: "sk-native-old", Alias: "QQ专用"}}); err != nil { + t.Fatal(err) + } + oldID := NativeKeyIDFromHash(SHA256Hex("sk-native-old")) + old, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-native-old")) + if err != nil || !ok { + t.Fatalf("old native key err=%v ok=%v", err, ok) + } + if old.ID != oldID || !old.Enabled || old.Name != "QQ专用" || old.Alias != "QQ专用" || old.Source != NativeCPASource || !old.SourcePresent || old.Hidden { + t.Fatalf("new native key should be enabled current official policy row: %#v", old) + } + old.Enabled = true + old.RPM = 7 + old.Models = []string{"gpt-5.5"} + old.Prices = map[string]ModelPrice{"gpt-5.5": {Model: "gpt-5.5", InputPerMillion: 1}} + old.FiveHourUSD = ptr(2) + old.WeeklyLimitUSD = ptr(12) + if err := store.SaveKeySettings(ctx, old); err != nil { + t.Fatal(err) + } + if err := store.InsertUsage(ctx, UsageEvent{RequestID: "old-usage", KeyID: old.ID, RequestedAt: time.Now(), Cost: 1.25}); err != nil { + t.Fatal(err) + } + + if err := store.SyncNativeKeys(ctx, nil); err != nil { + t.Fatal(err) + } + removed, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-native-old")) + if err != nil || !ok { + t.Fatalf("removed native key err=%v ok=%v", err, ok) + } + if removed.Enabled || removed.SourcePresent || !removed.Hidden { + t.Fatalf("removed native key should be disabled and hidden: %#v", removed) + } + usage, err := store.UsageSummary(ctx, old.ID, WindowFor(Range24H, time.Now())) + if err != nil { + t.Fatal(err) + } + if usage.Calls != 1 || usage.TotalCost != 1.25 { + t.Fatalf("removed key history should remain: %#v", usage) + } + + if err := store.SyncNativeKeys(ctx, []NativeKeySyncInput{{RawKey: "sk-native-new", Alias: "QQ专用"}}); err != nil { + t.Fatal(err) + } + newKey, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-native-new")) + if err != nil || !ok { + t.Fatalf("new inherited native key err=%v ok=%v", err, ok) + } + if !newKey.Enabled || newKey.RPM != 7 || newKey.InheritedFrom != old.ID || len(newKey.Models) != 1 || newKey.Models[0] != "gpt-5.5" || newKey.FiveHourUSD == nil || *newKey.FiveHourUSD != 2 { + t.Fatalf("new same-alias key should inherit policy only: %#v", newKey) + } + newUsage, err := store.UsageSummary(ctx, newKey.ID, WindowFor(Range24H, time.Now())) + if err != nil { + t.Fatal(err) + } + if newUsage.Calls != 0 || newUsage.TotalCost != 0 { + t.Fatalf("new inherited key must start fresh ledger: %#v", newUsage) + } +} + +func TestSyncNativeKeysDisablesAmbiguousSameAliasInheritance(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + for _, raw := range []string{"sk-old-a", "sk-old-b"} { + rec, ok := NativeKeyRecord(raw, "阿伟专用") + if !ok { + t.Fatalf("failed to build native record for %s", raw) + } + rec.SourcePresent = false + rec.Hidden = true + rec.Enabled = false + if err := store.UpsertKey(ctx, rec); err != nil { + t.Fatal(err) + } + } + if err := store.SyncNativeKeys(ctx, []NativeKeySyncInput{{RawKey: "sk-new-c", Alias: "阿伟专用"}}); err != nil { + t.Fatal(err) + } + key, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-new-c")) + if err != nil || !ok { + t.Fatalf("new key err=%v ok=%v", err, ok) + } + if key.Enabled || !key.InheritConflict || key.InheritedFrom != "" { + t.Fatalf("ambiguous same-alias inheritance should require manual template: %#v", key) + } +} + +func TestSyncNativeKeysUpgradesExistingDefaultEmptyNativePolicy(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + rec, ok := NativeKeyRecord("sk-empty-existing", "") + if !ok { + t.Fatal("failed to build native record") + } + rec.Enabled = false + rec.LastEnabled = false + if err := store.UpsertKey(ctx, rec); err != nil { + t.Fatal(err) + } + if _, err := store.db.ExecContext(ctx, `update keys set enabled=0, last_enabled=0 where id=?`, rec.ID); err != nil { + t.Fatal(err) + } + if err := store.SyncNativeKeys(ctx, []NativeKeySyncInput{{RawKey: "sk-empty-existing"}}); err != nil { + t.Fatal(err) + } + upgraded, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-empty-existing")) + if err != nil || !ok { + t.Fatalf("upgraded key err=%v ok=%v", err, ok) + } + if !upgraded.Enabled { + t.Fatalf("default empty native policy should be upgraded to enabled: %#v", upgraded) + } + + upgraded.Enabled = false + upgraded.LastEnabled = false + upgraded.RPM = 30 + if err := store.SaveKeySettings(ctx, upgraded); err != nil { + t.Fatal(err) + } + if err := store.SyncNativeKeys(ctx, []NativeKeySyncInput{{RawKey: "sk-empty-existing"}}); err != nil { + t.Fatal(err) + } + manual, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-empty-existing")) + if err != nil || !ok { + t.Fatalf("manual key err=%v ok=%v", err, ok) + } + if manual.Enabled { + t.Fatalf("manual disabled policy with settings should not be auto-enabled: %#v", manual) + } +} + +func TestSyncNativeKeysInheritsFromLegacyPlusPolicyByAlias(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + legacy := KeyRecord{ + ID: "legacy-qq", + Name: "QQ专用", + KeyHash: "sha256:" + SHA256Hex("cpa-old-qq"), + Enabled: true, + Preview: HashPreview(SHA256Hex("cpa-old-qq")), + RPM: 9, + Models: []string{"gpt-5.5"}, + WeeklyLimitUSD: ptr(30), + MonthlyLimitUSD: ptr(100), + } + if err := store.UpsertKey(ctx, legacy); err != nil { + t.Fatal(err) + } + if err := store.InsertUsage(ctx, UsageEvent{RequestID: "legacy-usage", KeyID: legacy.ID, RequestedAt: time.Now(), Cost: 2}); err != nil { + t.Fatal(err) + } + if err := store.SyncNativeKeys(ctx, []NativeKeySyncInput{{RawKey: "sk-official-qq", Alias: "QQ专用"}}); err != nil { + t.Fatal(err) + } + native, ok, err := store.FindKeyByHash(ctx, SHA256Hex("sk-official-qq")) + if err != nil || !ok { + t.Fatalf("native key err=%v ok=%v", err, ok) + } + if native.InheritedFrom != legacy.ID || !native.Enabled || native.RPM != 9 || native.WeeklyLimitUSD == nil || *native.WeeklyLimitUSD != 30 { + t.Fatalf("native key should inherit legacy Plus policy: %#v", native) + } + keys, err := store.ListKeys(ctx) + if err != nil { + t.Fatal(err) + } + var retired KeyRecord + for _, key := range keys { + if key.ID == legacy.ID { + retired = key + break + } + } + if retired.ID == "" || retired.Source != LegacyPlusSource || retired.Enabled || retired.SourcePresent || !retired.Hidden || !retired.LastEnabled { + t.Fatalf("legacy Plus template should retire after inheritance: %#v", retired) + } + legacyUsage, err := store.UsageSummary(ctx, legacy.ID, WindowFor(Range24H, time.Now())) + if err != nil { + t.Fatal(err) + } + nativeUsage, err := store.UsageSummary(ctx, native.ID, WindowFor(Range24H, time.Now())) + if err != nil { + t.Fatal(err) + } + if legacyUsage.Calls != 1 || nativeUsage.Calls != 0 { + t.Fatalf("usage should stay on legacy ledger only: legacy=%#v native=%#v", legacyUsage, nativeUsage) + } +} + +func TestStoreMigratesOldUsageEventsSchema(t *testing.T) { + path := filepath.Join(t.TempDir(), "policyplus.sqlite") + db, err := sql.Open("sqlite", path) + if err != nil { + t.Fatal(err) + } + _, err = db.Exec(`create table usage_events ( + id integer primary key autoincrement, + request_id text, + key_id text, + key_preview text, + model text, + endpoint text, + requested_at integer not null, + latency_ms integer, + failed integer not null, + failure text, + input_tokens integer, + output_tokens integer, + cached_tokens integer, + cache_read_tokens integer, + cache_creation_tokens integer, + reasoning_tokens integer, + total_tokens integer, + cost real, + cost_breakdown_json text + )`) + if err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`insert into usage_events(request_id, key_id, model, requested_at, failed, cost, cost_breakdown_json) values('old-1', 'alice-key', 'gpt-5.5', ?, 0, 0.1, '{}')`, time.Now().Unix()); err != nil { + t.Fatal(err) + } + if err := db.Close(); err != nil { + t.Fatal(err) + } + store, err := OpenStore(path) + if err != nil { + t.Fatal(err) + } + defer store.Close() + events, err := store.RecentEvents(context.Background(), "alice-key", 10) + if err != nil { + t.Fatal(err) + } + if len(events) != 1 || events[0].RequestID != "old-1" { + t.Fatalf("events = %#v", events) + } + if events[0].RequestedModel != "" || events[0].TTFTMS != 0 || events[0].StatusCode != 0 { + t.Fatalf("old row should read safe zero values: %#v", events[0]) + } +} + +func TestSessionIdentityPriorityAndExpiry(t *testing.T) { + headers := http.Header{ + "X-Codex-Turn-Metadata": []string{`{"window_id":"turn-window","prompt_cache_key":"turn-cache"}`}, + "X-Session-ID": []string{"session-header"}, + } + body := []byte(`{"client_metadata":{"x-codex-window-id":"client-window"},"prompt_cache_key":"body-cache","conversation_id":"conv"}`) + identity := ExtractSessionIdentity(headers, body) + if identity.Source != "client_metadata.x-codex-window-id" { + t.Fatalf("identity priority = %#v", identity) + } + headers.Set("X-Codex-Window-Id", "header-window") + identity = ExtractSessionIdentity(headers, body) + if identity.Source != "x-codex-window-id" { + t.Fatalf("header window should win: %#v", identity) + } + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + now := time.Unix(1000, 0) + first, err := store.RegisterActiveSession(ctx, "alice", newSessionIdentity("test", "a"), 1, DefaultSessionIdle, now) + if err != nil || !first.Allowed || first.Active != 1 { + t.Fatalf("first session = %#v err=%v", first, err) + } + second, err := store.RegisterActiveSession(ctx, "alice", newSessionIdentity("test", "b"), 1, DefaultSessionIdle, now.Add(31*time.Minute)) + if err != nil || !second.Allowed || second.Active != 1 { + t.Fatalf("expired slot should be reusable: %#v err=%v", second, err) + } + missing, err := store.RegisterActiveSession(ctx, "alice", SessionIdentity{}, 1, DefaultSessionIdle, now) + if err != nil || !missing.Allowed || !missing.Missing { + t.Fatalf("missing identity should be allowed: %#v err=%v", missing, err) + } + count, err := store.AuditCount(ctx, "missing_session_identity") + if err != nil || count != 1 { + t.Fatalf("missing identity audit count=%d err=%v", count, err) + } +} + +func TestStoreRecentCodexSummariesFiltersByKey(t *testing.T) { + ctx := context.Background() + store, err := OpenStore(filepath.Join(t.TempDir(), "policyplus.sqlite")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if err := store.SaveCodexSummary(ctx, "req-a", "alice-key", "gpt-5.5", "auto_continued", map[string]any{ + "request_id": "req-a", + "protection": "auto_continued", + }); err != nil { + t.Fatal(err) + } + if err := store.SaveCodexSummary(ctx, "req-b", "bob-key", "gpt-5.5", "protected_clean", map[string]any{ + "request_id": "req-b", + "protection": "protected_clean", + }); err != nil { + t.Fatal(err) + } + alice, err := store.RecentCodexSummaries(ctx, "alice-key", 10) + if err != nil { + t.Fatal(err) + } + if len(alice) != 1 || alice[0].RequestID != "req-a" || alice[0].Protection != "auto_continued" { + t.Fatalf("alice summaries = %#v", alice) + } + all, err := store.RecentCodexSummaries(ctx, "all", 10) + if err != nil { + t.Fatal(err) + } + if len(all) != 2 { + t.Fatalf("all summaries = %#v", all) + } +} + +func TestRecentCodexSummariesFromSQLiteReadsExternalExecutorStore(t *testing.T) { + ctx := context.Background() + path := filepath.Join(t.TempDir(), "executor.sqlite") + store, err := OpenStore(path) + if err != nil { + t.Fatal(err) + } + if err := store.SaveCodexSummary(ctx, "exec-a", "alice-key", "gpt-5.5", "auto_continued", map[string]any{ + "request_id": "exec-a", + "key_identity": map[string]any{"known": true, "id": "alice-key"}, + "protection": "auto_continued", + }); err != nil { + t.Fatal(err) + } + if err := store.SaveCodexSummary(ctx, "exec-b", "bob-key", "gpt-5.5", "protected_clean", map[string]any{ + "request_id": "exec-b", + "key_identity": map[string]any{"known": true, "id": "bob-key"}, + "protection": "protected_clean", + }); err != nil { + t.Fatal(err) + } + if err := store.Close(); err != nil { + t.Fatal(err) + } + alice, err := RecentCodexSummariesFromSQLite(ctx, path, "alice-key", 10) + if err != nil { + t.Fatal(err) + } + if len(alice) != 1 || alice[0].RequestID != "exec-a" || alice[0].Protection != "auto_continued" { + t.Fatalf("alice executor summaries = %#v", alice) + } + missing, err := RecentCodexSummariesFromSQLite(ctx, filepath.Join(t.TempDir(), "missing.sqlite"), "alice-key", 10) + if err == nil || missing != nil { + t.Fatalf("missing executor db should fail soft for caller: items=%#v err=%v", missing, err) + } +} + +func TestSecurityAndRedaction(t *testing.T) { + raw := " cpa_live " + hash := SHA256Hex(raw) + if hash != SHA256Hex(strings.TrimSpace(raw)) { + t.Fatal("SHA256Hex should trim raw keys") + } + if hash != SHA256Hex("Bearer cpa_live") || hash != SHA256Hex("Authorization: Bearer cpa_live") { + t.Fatal("SHA256Hex should normalize pasted bearer prefixes") + } + token, err := SignSession(SessionPayload{KeyID: "alice", KeyHash: "sha256:" + hash, ExpiresAt: time.Now().Add(time.Hour).Unix()}, "secret") + if err != nil { + t.Fatal(err) + } + payload, ok := VerifySession(token, "secret", time.Now()) + if !ok || payload.KeyID != "alice" { + t.Fatalf("session verify failed: %#v ok=%v", payload, ok) + } + if _, ok := VerifySession(token, "wrong", time.Now()); ok { + t.Fatal("session verified with wrong secret") + } + brief := Brief("Authorization: Bearer secret and api_key=abc", 200) + if strings.Contains(brief, "secret") || strings.Contains(brief, "abc") { + t.Fatalf("secret leaked in brief: %s", brief) + } +} + +func TestSubmittedKeyHints(t *testing.T) { + cases := []struct { + name string + in string + code string + }{ + {name: "missing", in: " ", code: "missing_api_key"}, + {name: "native", in: "sk-abc", code: "invalid_api_key"}, + {name: "preview", in: "cpa_abcd...efgh", code: "key_preview_not_usable"}, + {name: "unsupported", in: "abc", code: "unsupported_key_format"}, + {name: "short cpa", in: "Bearer cpa_live", code: "legacy_cpa_key_retired"}, + {name: "full cpa", in: "Bearer cpa_abcdefghijklmnopqrstuvwxyz0123456789", code: "legacy_cpa_key_retired"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + hint := ExplainUnmatchedSubmittedKey(tc.in) + if hint.Error != tc.code || hint.Message == "" { + t.Fatalf("hint = %#v", hint) + } + }) + } + if got := NormalizeSubmittedKey("Authorization: Bearer Bearer cpa_live "); got != "cpa_live" { + t.Fatalf("normalized key = %q", got) + } + if got := NormalizeSubmittedKey("\ufeff“Bearer cpa_live\u200b”"); got != "cpa_live" { + t.Fatalf("normalized decorated key = %q", got) + } +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/pricing.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/pricing.go new file mode 100644 index 0000000..d054dd6 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/pricing.go @@ -0,0 +1,102 @@ +package policyplus + +import "strings" + +const perMillion = 1_000_000.0 + +type TokenUsage struct { + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` + CachedTokens int64 `json:"cached_tokens"` + CacheReadTokens int64 `json:"cache_read_tokens"` + CacheCreationTokens int64 `json:"cache_creation_tokens"` + ReasoningTokens int64 `json:"reasoning_tokens"` + TotalTokens int64 `json:"total_tokens"` +} + +type CostBreakdown struct { + Source string `json:"source"` + Model string `json:"model"` + Prices ModelPrice `json:"prices"` + Tokens map[string]int64 `json:"tokens"` + Costs map[string]float64 `json:"costs"` +} + +func PriceForModel(prices map[string]ModelPrice, model string) (ModelPrice, bool) { + model = strings.TrimSpace(model) + if model == "" { + return ModelPrice{}, false + } + if price, ok := prices[model]; ok { + return price, true + } + lower := strings.ToLower(model) + for name, price := range prices { + if strings.ToLower(name) == lower { + return price, true + } + } + return ModelPrice{}, false +} + +func CostForUsage(price ModelPrice, usage TokenUsage, model string) CostBreakdown { + input := max64(usage.InputTokens, 0) + output := max64(usage.OutputTokens, 0) + cached := max64(usage.CachedTokens, 0) + cacheRead := max64(usage.CacheReadTokens, 0) + cacheCreation := max64(usage.CacheCreationTokens, 0) + reasoning := max64(usage.ReasoningTokens, 0) + billableInput := max64(input-cached, 0) + cacheReadPrice := price.CacheReadPerMillion + if cacheReadPrice <= 0 { + cacheReadPrice = price.InputPerMillion + } + cacheCreationPrice := price.CacheCreationPerMillion + if cacheCreationPrice <= 0 { + cacheCreationPrice = price.InputPerMillion + } + inputCost := float64(billableInput) * price.InputPerMillion / perMillion + cachedCost := float64(cached) * cacheReadPrice / perMillion + cacheReadCost := float64(cacheRead) * cacheReadPrice / perMillion + cacheCreationCost := float64(cacheCreation) * cacheCreationPrice / perMillion + outputCost := float64(output) * price.OutputPerMillion / perMillion + total := inputCost + cachedCost + cacheReadCost + cacheCreationCost + outputCost + totalTokens := usage.TotalTokens + if totalTokens <= 0 { + totalTokens = input + output + } + if strings.TrimSpace(model) == "" { + model = price.Model + } + return CostBreakdown{ + Source: "key_policy_plus_price_book", + Model: model, + Prices: price, + Tokens: map[string]int64{ + "input": input, + "billable_uncached_input": billableInput, + "cached_input": cached, + "cache_read": cacheRead, + "cache_creation": cacheCreation, + "output": output, + "reasoning": reasoning, + "visible_output_estimate": max64(output-reasoning, 0), + "total": totalTokens, + }, + Costs: map[string]float64{ + "input": inputCost, + "cached_input": cachedCost, + "cache_read": cacheReadCost, + "cache_creation": cacheCreationCost, + "output": outputCost, + "total": total, + }, + } +} + +func max64(a, b int64) int64 { + if a > b { + return a + } + return b +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/quota.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/quota.go new file mode 100644 index 0000000..65a80ba --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/quota.go @@ -0,0 +1,66 @@ +package policyplus + +import ( + "strings" + "time" +) + +const ( + Range5H = "5h" + Range24H = "24h" + Range7D = "7d" + RangeMonth = "month" +) + +type Window struct { + Name string `json:"name"` + From time.Time `json:"from"` + To time.Time `json:"to"` +} + +func WindowFor(rangeName string, now time.Time) Window { + now = now.UTC() + switch strings.ToLower(strings.TrimSpace(rangeName)) { + case Range5H: + return Window{Name: Range5H, From: now.Add(-5 * time.Hour), To: now} + case Range7D: + return Window{Name: Range7D, From: now.Add(-7 * 24 * time.Hour), To: now} + case RangeMonth: + loc := time.FixedZone("Asia/Shanghai", 8*60*60) + local := now.In(loc) + start := time.Date(local.Year(), local.Month(), 1, 0, 0, 0, 0, loc) + return Window{Name: RangeMonth, From: start.UTC(), To: now} + default: + return Window{Name: Range24H, From: now.Add(-24 * time.Hour), To: now} + } +} + +type QuotaDecision struct { + Allowed bool `json:"allowed"` + Reason string `json:"reason,omitempty"` + UsedUSD float64 `json:"used_usd"` + LimitUSD *float64 `json:"limit_usd,omitempty"` +} + +func CheckLimit(used float64, limit *float64) QuotaDecision { + if limit == nil { + return QuotaDecision{Allowed: true, UsedUSD: used} + } + if used >= *limit { + return QuotaDecision{Allowed: false, Reason: "quota_exceeded", UsedUSD: used, LimitUSD: limit} + } + return QuotaDecision{Allowed: true, UsedUSD: used, LimitUSD: limit} +} + +func ModelAllowed(allowed []string, model string) bool { + if len(allowed) == 0 { + return true + } + model = strings.TrimSpace(model) + for _, item := range allowed { + if strings.EqualFold(strings.TrimSpace(item), model) { + return true + } + } + return false +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/redaction.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/redaction.go new file mode 100644 index 0000000..009650a --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/redaction.go @@ -0,0 +1,35 @@ +package policyplus + +import ( + "regexp" + "strings" +) + +var ( + bearerRe = regexp.MustCompile(`(?i)\bBearer\s+[A-Za-z0-9._~+/=-]+`) + secretRe = regexp.MustCompile(`(?i)\b(api[_-]?key|access[_-]?token|refresh[_-]?token|authorization|cookie|secret)\s*[:=]\s*['"]?[^'"\s,;]+`) + spaceRe = regexp.MustCompile(`\s+`) +) + +func RedactString(value string) string { + value = bearerRe.ReplaceAllString(value, "Bearer [REDACTED]") + value = secretRe.ReplaceAllStringFunc(value, func(match string) string { + parts := strings.FieldsFunc(match, func(r rune) bool { return r == ':' || r == '=' }) + if len(parts) == 0 { + return "[REDACTED]" + } + return strings.TrimSpace(parts[0]) + "=[REDACTED]" + }) + return value +} + +func Brief(value string, limit int) string { + value = spaceRe.ReplaceAllString(strings.TrimSpace(RedactString(value)), " ") + if limit <= 0 || len(value) <= limit { + return value + } + if limit <= 3 { + return value[:limit] + } + return strings.TrimSpace(value[:limit-3]) + "..." +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/security.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/security.go new file mode 100644 index 0000000..90c4524 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/security.go @@ -0,0 +1,133 @@ +package policyplus + +import ( + "crypto/hmac" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/json" + "errors" + "regexp" + "strings" + "time" + "unicode" +) + +func SHA256Hex(value string) string { + sum := sha256.Sum256([]byte(NormalizeSubmittedKey(value))) + return hex.EncodeToString(sum[:]) +} + +var bearerPrefixPattern = regexp.MustCompile(`(?i)^\s*(authorization\s*:\s*)?(bearer\s+)+`) + +// NormalizeSubmittedKey accepts the common clipboard shapes users paste into +// the self-service portal, while keeping hashing deterministic. +func NormalizeSubmittedKey(value string) string { + text := strings.TrimSpace(value) + text = strings.Map(func(r rune) rune { + if unicode.Is(unicode.Cf, r) { + return -1 + } + return r + }, text) + text = strings.TrimSpace(strings.Trim(text, `"'`+"`"+`“”‘’「」『』<>`)) + text = bearerPrefixPattern.ReplaceAllString(text, "") + return strings.TrimSpace(strings.Trim(text, `"'`+"`"+`“”‘’「」『』<>`)) +} + +type SubmittedKeyHint struct { + Error string + Message string +} + +func ExplainUnmatchedSubmittedKey(value string) SubmittedKeyHint { + key := NormalizeSubmittedKey(value) + lower := strings.ToLower(key) + switch { + case key == "": + return SubmittedKeyHint{Error: "missing_api_key", Message: "请粘贴完整的 CPA 原生 sk- Key。"} + case strings.Contains(key, "...") || strings.Contains(key, "…"): + return SubmittedKeyHint{Error: "key_preview_not_usable", Message: "你粘贴的是缩略预览，不是完整 Key。请使用 CPAMP 中完整的 CPA 原生 sk- Key。"} + case strings.HasPrefix(lower, "sk-") || strings.HasPrefix(lower, "sk_"): + return SubmittedKeyHint{Error: "invalid_api_key", Message: "这个 sk- Key 没有匹配到已同步并启用的 CPA Key Policy+ 策略。请先在 CPAMP 确认原生 Key 存在，并在 Plus 管理页启用对应策略。"} + case strings.HasPrefix(lower, "cpa_"): + return SubmittedKeyHint{Error: "legacy_cpa_key_retired", Message: "旧的 cpa_ Key 已迁移下线。现在请使用 CPA/CPAMP 管理的原生 sk- Key 登录用量页。"} + case !strings.HasPrefix(lower, "sk-") && !strings.HasPrefix(lower, "sk_"): + return SubmittedKeyHint{Error: "unsupported_key_format", Message: "用量自助页现在只接受 CPA 原生 sk- Key。"} + default: + return SubmittedKeyHint{Error: "invalid_api_key", Message: "这个 Key 没有匹配到当前 CPA Key Policy+ 策略。请确认原生 Key 仍存在并已同步。"} + } +} + +func NormalizeHash(value string) (string, error) { + text := strings.TrimSpace(value) + if strings.HasPrefix(strings.ToLower(text), "sha256:") { + text = text[len("sha256:"):] + } + if len(text) != 64 { + return "", errors.New("invalid hash length") + } + _, err := hex.DecodeString(text) + if err != nil { + return "", err + } + return strings.ToLower(text), nil +} + +func HashPreview(value string) string { + normalized, err := NormalizeHash(value) + if err != nil { + normalized = SHA256Hex(value) + } + if len(normalized) <= 16 { + return normalized + } + return normalized[:8] + "..." + normalized[len(normalized)-6:] +} + +type SessionPayload struct { + KeyID string `json:"key_id"` + KeyHash string `json:"key_hash"` + ExpiresAt int64 `json:"expires_at"` +} + +func SignSession(payload SessionPayload, secret string) (string, error) { + if strings.TrimSpace(secret) == "" { + return "", errors.New("session secret is required") + } + raw, err := json.Marshal(payload) + if err != nil { + return "", err + } + body := base64.RawURLEncoding.EncodeToString(raw) + mac := hmac.New(sha256.New, []byte(secret)) + _, _ = mac.Write([]byte(body)) + sig := base64.RawURLEncoding.EncodeToString(mac.Sum(nil)) + return body + "." + sig, nil +} + +func VerifySession(token, secret string, now time.Time) (SessionPayload, bool) { + parts := strings.Split(token, ".") + if len(parts) != 2 || strings.TrimSpace(secret) == "" { + return SessionPayload{}, false + } + mac := hmac.New(sha256.New, []byte(secret)) + _, _ = mac.Write([]byte(parts[0])) + want := mac.Sum(nil) + got, err := base64.RawURLEncoding.DecodeString(parts[1]) + if err != nil || !hmac.Equal(got, want) { + return SessionPayload{}, false + } + raw, err := base64.RawURLEncoding.DecodeString(parts[0]) + if err != nil { + return SessionPayload{}, false + } + var payload SessionPayload + if err := json.Unmarshal(raw, &payload); err != nil { + return SessionPayload{}, false + } + if payload.ExpiresAt > 0 && now.Unix() > payload.ExpiresAt { + return SessionPayload{}, false + } + return payload, true +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/session.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/session.go new file mode 100644 index 0000000..d3f523b --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/session.go @@ -0,0 +1,162 @@ +package policyplus + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "net/http" + "strings" + "time" +) + +const DefaultSessionIdle = 30 * time.Minute + +type SessionIdentity struct { + Source string `json:"source"` + Value string `json:"-"` + Hash string `json:"hash"` +} + +type SessionDecision struct { + Allowed bool `json:"allowed"` + Active int `json:"active"` + Limit int `json:"limit"` + Missing bool `json:"missing"` +} + +func ExtractSessionIdentity(headers http.Header, body []byte) SessionIdentity { + if headers != nil { + if value := strings.TrimSpace(headers.Get("X-Codex-Window-Id")); value != "" { + return newSessionIdentity("x-codex-window-id", value) + } + } + if value := nestedString(body, "client_metadata", "x-codex-window-id"); value != "" { + return newSessionIdentity("client_metadata.x-codex-window-id", value) + } + if headers != nil { + if raw := strings.TrimSpace(headers.Get("X-Codex-Turn-Metadata")); raw != "" { + if value := stringFromJSON([]byte(raw), "window_id"); value != "" { + return newSessionIdentity("x-codex-turn-metadata.window_id", value) + } + if value := stringFromJSON([]byte(raw), "prompt_cache_key"); value != "" { + return newSessionIdentity("x-codex-turn-metadata.prompt_cache_key", value) + } + } + } + if raw := nestedString(body, "client_metadata", "x-codex-turn-metadata"); raw != "" { + if value := stringFromJSON([]byte(raw), "window_id"); value != "" { + return newSessionIdentity("client_metadata.x-codex-turn-metadata.window_id", value) + } + if value := stringFromJSON([]byte(raw), "prompt_cache_key"); value != "" { + return newSessionIdentity("client_metadata.x-codex-turn-metadata.prompt_cache_key", value) + } + } + if value := nestedString(body, "prompt_cache_key"); value != "" { + return newSessionIdentity("prompt_cache_key", value) + } + if headers != nil { + for _, name := range []string{"Session_id", "session_id", "Session-Id", "X-Session-ID"} { + if value := strings.TrimSpace(headers.Get(name)); value != "" { + return newSessionIdentity(strings.ToLower(name), value) + } + } + } + if value := nestedString(body, "conversation_id"); value != "" { + return newSessionIdentity("conversation_id", value) + } + return SessionIdentity{} +} + +func newSessionIdentity(source, value string) SessionIdentity { + value = strings.TrimSpace(value) + if value == "" { + return SessionIdentity{} + } + sum := sha256.Sum256([]byte(source + "\x00" + value)) + return SessionIdentity{ + Source: source, + Value: value, + Hash: "sha256:" + hex.EncodeToString(sum[:]), + } +} + +func nestedString(body []byte, path ...string) string { + if len(body) == 0 { + return "" + } + var cur any + if err := json.Unmarshal(body, &cur); err != nil { + return "" + } + for _, part := range path { + m, ok := cur.(map[string]any) + if !ok { + return "" + } + cur = m[part] + } + if text, ok := cur.(string); ok { + return strings.TrimSpace(text) + } + return "" +} + +func stringFromJSON(body []byte, key string) string { + var raw map[string]any + if err := json.Unmarshal(body, &raw); err != nil { + return "" + } + if text, ok := raw[key].(string); ok { + return strings.TrimSpace(text) + } + return "" +} + +func (s *Store) RegisterActiveSession(ctx context.Context, keyID string, identity SessionIdentity, limit int, idle time.Duration, now time.Time) (SessionDecision, error) { + if limit <= 0 { + return SessionDecision{Allowed: true, Limit: limit, Missing: identity.Hash == ""}, nil + } + if identity.Hash == "" { + _ = s.Audit(ctx, "frontend_auth", "missing_session_identity", keyID, map[string]any{"limit": limit}) + active, _ := s.ActiveSessionCount(ctx, keyID, idle, now) + return SessionDecision{Allowed: true, Active: active, Limit: limit, Missing: true}, nil + } + if idle <= 0 { + idle = DefaultSessionIdle + } + cutoff := now.Add(-idle).Unix() + _, _ = s.db.ExecContext(ctx, `delete from active_sessions where key_id=? and last_seen < ?`, keyID, cutoff) + var existing int + if err := s.db.QueryRowContext(ctx, `select count(1) from active_sessions where key_id=? and session_id=?`, keyID, identity.Hash).Scan(&existing); err != nil { + return SessionDecision{}, err + } + if existing > 0 { + _, err := s.db.ExecContext(ctx, `update active_sessions set last_seen=?, source=? where key_id=? and session_id=?`, now.Unix(), identity.Source, keyID, identity.Hash) + active, _ := s.ActiveSessionCount(ctx, keyID, idle, now) + return SessionDecision{Allowed: err == nil, Active: active, Limit: limit}, err + } + active, err := s.ActiveSessionCount(ctx, keyID, idle, now) + if err != nil { + return SessionDecision{}, err + } + if active >= limit { + _ = s.Audit(ctx, "frontend_auth", "session_limit_exceeded", keyID, map[string]any{"active": active, "limit": limit, "source": identity.Source}) + return SessionDecision{Allowed: false, Active: active, Limit: limit}, nil + } + _, err = s.db.ExecContext(ctx, `insert into active_sessions(key_id, session_id, source, first_seen, last_seen) values(?, ?, ?, ?, ?)`, keyID, identity.Hash, identity.Source, now.Unix(), now.Unix()) + if err != nil { + return SessionDecision{}, err + } + return SessionDecision{Allowed: true, Active: active + 1, Limit: limit}, nil +} + +func (s *Store) ActiveSessionCount(ctx context.Context, keyID string, idle time.Duration, now time.Time) (int, error) { + if idle <= 0 { + idle = DefaultSessionIdle + } + cutoff := now.Add(-idle).Unix() + var count int + err := s.db.QueryRowContext(ctx, `select count(1) from active_sessions where key_id=? and last_seen >= ?`, keyID, cutoff).Scan(&count) + return count, err +} diff --git a/cpa_key_policy_plus_plugin/go/internal/policyplus/store.go b/cpa_key_policy_plus_plugin/go/internal/policyplus/store.go new file mode 100644 index 0000000..034ff96 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/internal/policyplus/store.go @@ -0,0 +1,1370 @@ +package policyplus + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "net/url" + "os" + "path/filepath" + "strings" + "time" + + "gopkg.in/yaml.v3" + _ "modernc.org/sqlite" +) + +type Store struct { + db *sql.DB +} + +type UsageEvent struct { + RequestID string `json:"request_id"` + KeyID string `json:"key_id"` + KeyPreview string `json:"key_preview"` + Model string `json:"model"` + RequestedModel string `json:"requested_model,omitempty"` + ActualModel string `json:"actual_model,omitempty"` + Provider string `json:"provider,omitempty"` + ExecutorType string `json:"executor_type,omitempty"` + Endpoint string `json:"endpoint,omitempty"` + RequestedAt time.Time `json:"requested_at"` + LatencyMS int64 `json:"latency_ms"` + TTFTMS int64 `json:"ttft_ms,omitempty"` + ReasoningEffort string `json:"reasoning_effort,omitempty"` + ServiceTier string `json:"service_tier,omitempty"` + StatusCode int `json:"status_code,omitempty"` + Failed bool `json:"failed"` + Failure string `json:"failure,omitempty"` + Usage TokenUsage `json:"usage"` + Cost float64 `json:"cost"` + CostBreakdown CostBreakdown `json:"cost_breakdown"` +} + +type UsageSummary struct { + Calls int64 `json:"calls"` + Failed int64 `json:"failed"` + TotalCost float64 `json:"total_cost"` + Usage TokenUsage `json:"usage"` +} + +type CodexSummary struct { + RequestID string `json:"request_id"` + KeyID string `json:"key_id,omitempty"` + Model string `json:"model,omitempty"` + Protection string `json:"protection,omitempty"` + Summary map[string]any `json:"summary"` + UpdatedAt time.Time `json:"updated_at"` +} + +type LegacyImportResult struct { + Limits int + Resets int +} + +const sqliteBusyTimeoutMS = 5000 + +func OpenStore(path string) (*Store, error) { + if path == "" { + path = "cpa-policyplus.sqlite" + } + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil && filepath.Dir(path) != "." { + return nil, err + } + db, err := openSQLite(path, false) + if err != nil { + return nil, err + } + db.SetMaxOpenConns(1) + db.SetMaxIdleConns(1) + store := &Store{db: db} + if err := store.EnsureSchema(context.Background()); err != nil { + _ = db.Close() + return nil, err + } + return store, nil +} + +func openSQLite(path string, readOnly bool) (*sql.DB, error) { + dsn := sqliteDSN(path, readOnly) + db, err := sql.Open("sqlite", dsn) + if err != nil { + return nil, err + } + if readOnly { + db.SetMaxOpenConns(2) + db.SetMaxIdleConns(1) + } + return db, nil +} + +func sqliteDSN(path string, readOnly bool) string { + path = strings.TrimSpace(path) + query := url.Values{} + query.Add("_pragma", fmt.Sprintf("busy_timeout(%d)", sqliteBusyTimeoutMS)) + if readOnly { + query.Set("mode", "ro") + } else { + query.Add("_pragma", "journal_mode(WAL)") + } + if strings.HasPrefix(path, "file:") { + sep := "?" + if strings.Contains(path, "?") { + sep = "&" + } + return path + sep + query.Encode() + } + if readOnly { + return "file:" + filepath.ToSlash(path) + "?" + query.Encode() + } + return path + "?" + query.Encode() +} + +func (s *Store) Close() error { + if s == nil || s.db == nil { + return nil + } + return s.db.Close() +} + +func (s *Store) EnsureSchema(ctx context.Context) error { + stmts := []string{ + `pragma journal_mode=wal`, + `create table if not exists keys ( + id text primary key, + name text not null, + key_hash text not null unique, + enabled integer not null, + preview text, + rpm integer, + concurrency integer, + max_active_sessions integer, + models_json text, + prices_json text, + five_hour_limit_usd real, + daily_limit_usd real, + weekly_limit_usd real, + monthly_limit_usd real, + archived integer not null default 0, + archived_at integer not null default 0, + source text default '', + source_present integer not null default 1, + alias text default '', + inherited_from text default '', + inherit_conflict integer not null default 0, + hidden integer not null default 0, + last_enabled integer not null default 0, + updated_at integer not null + )`, + `create table if not exists reset_watermarks ( + key_id text not null, + window text not null, + reset_at integer not null, + primary key(key_id, window) + )`, + `create table if not exists active_sessions ( + key_id text not null, + session_id text not null, + source text, + first_seen integer not null, + last_seen integer not null, + primary key(key_id, session_id) + )`, + `create table if not exists usage_events ( + id integer primary key autoincrement, + request_id text, + key_id text, + key_preview text, + model text, + requested_model text, + actual_model text, + provider text, + executor_type text, + endpoint text, + requested_at integer not null, + latency_ms integer, + ttft_ms integer, + reasoning_effort text, + service_tier text, + status_code integer, + failed integer not null, + failure text, + input_tokens integer, + output_tokens integer, + cached_tokens integer, + cache_read_tokens integer, + cache_creation_tokens integer, + reasoning_tokens integer, + total_tokens integer, + cost real, + cost_breakdown_json text + )`, + `create table if not exists codexcont_summaries ( + request_id text primary key, + key_id text, + model text, + protection text, + summary_json text, + updated_at integer not null + )`, + `create table if not exists audit_log ( + id integer primary key autoincrement, + timestamp integer not null, + actor text, + action text not null, + target text, + detail_json text + )`, + `create table if not exists settings ( + key text primary key, + value text not null, + updated_at integer not null + )`, + } + for _, stmt := range stmts { + if _, err := s.db.ExecContext(ctx, stmt); err != nil { + return err + } + } + if err := s.ensureColumns(ctx, "usage_events", map[string]string{ + "requested_model": "text default ''", + "actual_model": "text default ''", + "provider": "text default ''", + "executor_type": "text default ''", + "ttft_ms": "integer default 0", + "reasoning_effort": "text default ''", + "service_tier": "text default ''", + "status_code": "integer default 0", + }); err != nil { + return err + } + if err := s.ensureColumns(ctx, "keys", map[string]string{ + "max_active_sessions": "integer default 0", + "archived": "integer not null default 0", + "archived_at": "integer not null default 0", + "source": "text default ''", + "source_present": "integer not null default 1", + "alias": "text default ''", + "inherited_from": "text default ''", + "inherit_conflict": "integer not null default 0", + "hidden": "integer not null default 0", + "last_enabled": "integer not null default 0", + }); err != nil { + return err + } + return nil +} + +func (s *Store) ensureColumns(ctx context.Context, table string, columns map[string]string) error { + rows, err := s.db.QueryContext(ctx, fmt.Sprintf(`pragma table_info(%s)`, table)) + if err != nil { + return err + } + defer rows.Close() + existing := map[string]bool{} + for rows.Next() { + var cid int + var name, typ string + var notNull int + var defaultValue sql.NullString + var pk int + if err := rows.Scan(&cid, &name, &typ, ¬Null, &defaultValue, &pk); err != nil { + return err + } + existing[name] = true + } + if err := rows.Err(); err != nil { + return err + } + for name, typ := range columns { + if existing[name] { + continue + } + if _, err := s.db.ExecContext(ctx, fmt.Sprintf(`alter table %s add column %s %s`, table, name, typ)); err != nil { + return err + } + } + return nil +} + +func (s *Store) UpsertKey(ctx context.Context, key KeyRecord) error { + if err := ValidateKeyRecord(key); err != nil { + return err + } + if key.Source == "" { + key.SourcePresent = true + } + models, _ := json.Marshal(key.Models) + prices, _ := json.Marshal(key.Prices) + _, err := s.db.ExecContext( + ctx, + `insert into keys( + id, name, key_hash, enabled, preview, rpm, concurrency, max_active_sessions, models_json, prices_json, + five_hour_limit_usd, daily_limit_usd, weekly_limit_usd, monthly_limit_usd, + archived, archived_at, source, source_present, alias, inherited_from, inherit_conflict, hidden, last_enabled, updated_at + ) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + on conflict(id) do update set + name=excluded.name, + key_hash=excluded.key_hash, + enabled=excluded.enabled, + preview=excluded.preview, + rpm=excluded.rpm, + concurrency=excluded.concurrency, + max_active_sessions=excluded.max_active_sessions, + models_json=excluded.models_json, + prices_json=excluded.prices_json, + five_hour_limit_usd=coalesce(keys.five_hour_limit_usd, excluded.five_hour_limit_usd), + daily_limit_usd=excluded.daily_limit_usd, + weekly_limit_usd=excluded.weekly_limit_usd, + monthly_limit_usd=coalesce(keys.monthly_limit_usd, excluded.monthly_limit_usd), + archived=case when excluded.archived != 0 then excluded.archived else keys.archived end, + archived_at=case when excluded.archived != 0 then excluded.archived_at else keys.archived_at end, + source=case when excluded.source != '' then excluded.source else keys.source end, + source_present=excluded.source_present, + alias=excluded.alias, + inherited_from=case when excluded.inherited_from != '' then excluded.inherited_from else keys.inherited_from end, + inherit_conflict=excluded.inherit_conflict, + hidden=excluded.hidden, + last_enabled=case when excluded.last_enabled != 0 then excluded.last_enabled else keys.last_enabled end, + updated_at=excluded.updated_at`, + key.ID, + key.Name, + key.KeyHash, + boolInt(key.Enabled), + key.Preview, + key.RPM, + key.Concurrency, + key.MaxActiveSessions, + string(models), + string(prices), + key.FiveHourUSD, + key.DailyLimitUSD, + key.WeeklyLimitUSD, + key.MonthlyLimitUSD, + boolInt(key.Archived), + key.ArchivedAt, + key.Source, + boolInt(key.SourcePresent), + key.Alias, + key.InheritedFrom, + boolInt(key.InheritConflict), + boolInt(key.Hidden), + boolInt(key.LastEnabled), + time.Now().Unix(), + ) + return err +} + +func (s *Store) ImportKeys(ctx context.Context, state KeyPolicyState) error { + for _, key := range state.Keys { + if err := s.UpsertKey(ctx, key); err != nil { + return err + } + } + return nil +} + +func (s *Store) ImportLegacyQuotaSQLite(ctx context.Context, path, source string) (LegacyImportResult, error) { + path = strings.TrimSpace(path) + if path == "" { + return LegacyImportResult{}, nil + } + db, err := openSQLite(path, true) + if err != nil { + return LegacyImportResult{}, err + } + defer db.Close() + result := LegacyImportResult{} + if ok, _ := legacyTableExists(ctx, db, "key_limits"); ok { + n, err := s.importLegacyPortalLimits(ctx, db) + if err != nil { + return result, err + } + result.Limits += n + } + if ok, _ := legacyTableExists(ctx, db, "keys"); ok { + n, err := s.importLegacyGovernorLimits(ctx, db) + if err != nil { + return result, err + } + result.Limits += n + } + if ok, _ := legacyTableExists(ctx, db, "reset_watermarks"); ok { + n, err := s.importLegacyResets(ctx, db) + if err != nil { + return result, err + } + result.Resets += n + } + if result.Limits > 0 || result.Resets > 0 { + _ = s.Audit(ctx, "system", "import_legacy_quota_sqlite", source, map[string]any{ + "path": path, + "limits": result.Limits, + "resets": result.Resets, + }) + } + return result, nil +} + +func legacyTableExists(ctx context.Context, db *sql.DB, name string) (bool, error) { + var count int + err := db.QueryRowContext(ctx, `select count(1) from sqlite_master where type='table' and name=?`, name).Scan(&count) + return count > 0, err +} + +func legacyColumns(ctx context.Context, db *sql.DB, table string) (map[string]bool, error) { + rows, err := db.QueryContext(ctx, fmt.Sprintf(`pragma table_info(%s)`, table)) + if err != nil { + return nil, err + } + defer rows.Close() + out := map[string]bool{} + for rows.Next() { + var cid int + var name, typ string + var notNull int + var defaultValue sql.NullString + var pk int + if err := rows.Scan(&cid, &name, &typ, ¬Null, &defaultValue, &pk); err != nil { + return nil, err + } + out[name] = true + } + return out, rows.Err() +} + +func (s *Store) importLegacyPortalLimits(ctx context.Context, db *sql.DB) (int, error) { + rows, err := db.QueryContext(ctx, `select policy_id, five_hour_limit_usd, monthly_limit_usd from key_limits`) + if err != nil { + return 0, err + } + defer rows.Close() + count := 0 + for rows.Next() { + var id string + var fiveHour, monthly sql.NullFloat64 + if err := rows.Scan(&id, &fiveHour, &monthly); err != nil { + return count, err + } + if strings.TrimSpace(id) == "" { + continue + } + res, err := s.db.ExecContext(ctx, `update keys set + five_hour_limit_usd=coalesce(five_hour_limit_usd, ?), + monthly_limit_usd=coalesce(monthly_limit_usd, ?), + updated_at=? + where id=?`, + nullFloatValue(fiveHour), nullFloatValue(monthly), time.Now().Unix(), id) + if err != nil { + return count, err + } + if affected, _ := res.RowsAffected(); affected > 0 { + count++ + } + } + return count, rows.Err() +} + +func (s *Store) importLegacyGovernorLimits(ctx context.Context, db *sql.DB) (int, error) { + columns, err := legacyColumns(ctx, db, "keys") + if err != nil { + return 0, err + } + for _, required := range []string{"id", "five_hour_limit_usd", "daily_limit_usd", "weekly_limit_usd", "monthly_limit_usd"} { + if !columns[required] { + return 0, nil + } + } + rows, err := db.QueryContext(ctx, `select id, five_hour_limit_usd, daily_limit_usd, weekly_limit_usd, monthly_limit_usd from keys`) + if err != nil { + return 0, err + } + defer rows.Close() + count := 0 + for rows.Next() { + var id string + var fiveHour, daily, weekly, monthly sql.NullFloat64 + if err := rows.Scan(&id, &fiveHour, &daily, &weekly, &monthly); err != nil { + return count, err + } + if strings.TrimSpace(id) == "" { + continue + } + res, err := s.db.ExecContext(ctx, `update keys set + five_hour_limit_usd=coalesce(five_hour_limit_usd, ?), + daily_limit_usd=coalesce(daily_limit_usd, ?), + weekly_limit_usd=coalesce(weekly_limit_usd, ?), + monthly_limit_usd=coalesce(monthly_limit_usd, ?), + updated_at=? + where id=?`, + nullFloatValue(fiveHour), nullFloatValue(daily), nullFloatValue(weekly), + nullFloatValue(monthly), time.Now().Unix(), id) + if err != nil { + return count, err + } + if affected, _ := res.RowsAffected(); affected > 0 { + count++ + } + } + return count, rows.Err() +} + +func (s *Store) importLegacyResets(ctx context.Context, db *sql.DB) (int, error) { + columns, err := legacyColumns(ctx, db, "reset_watermarks") + if err != nil { + return 0, err + } + idColumn := "" + for _, candidate := range []string{"key_id", "policy_id"} { + if columns[candidate] { + idColumn = candidate + break + } + } + timeColumn := "" + for _, candidate := range []string{"reset_at", "reset_at_ms"} { + if columns[candidate] { + timeColumn = candidate + break + } + } + if idColumn == "" || timeColumn == "" || !columns["window"] { + return 0, nil + } + rows, err := db.QueryContext(ctx, fmt.Sprintf(`select %s, window, %s from reset_watermarks`, idColumn, timeColumn)) + if err != nil { + return 0, err + } + defer rows.Close() + count := 0 + for rows.Next() { + var id, window string + var resetAt sql.NullInt64 + if err := rows.Scan(&id, &window, &resetAt); err != nil { + return count, err + } + if strings.TrimSpace(id) == "" || strings.TrimSpace(window) == "" || !resetAt.Valid { + continue + } + ts := resetAt.Int64 + if ts > 1_000_000_000_000 { + ts = ts / 1000 + } + res, err := s.db.ExecContext(ctx, `insert into reset_watermarks(key_id, window, reset_at) values(?, ?, ?) + on conflict(key_id, window) do update set reset_at=excluded.reset_at + where excluded.reset_at > reset_watermarks.reset_at`, id, window, ts) + if err != nil { + return count, err + } + if affected, _ := res.RowsAffected(); affected > 0 { + count++ + } + } + return count, rows.Err() +} + +func (s *Store) syncDeletedKeys(ctx context.Context, seen map[string]bool) error { + rows, err := s.db.QueryContext(ctx, `select id from keys`) + if err != nil { + return err + } + var stale []string + for rows.Next() { + var id string + if err := rows.Scan(&id); err != nil { + _ = rows.Close() + return err + } + if !seen[id] { + stale = append(stale, id) + } + } + if err := rows.Close(); err != nil { + return err + } + for _, id := range stale { + if _, err := s.db.ExecContext(ctx, `delete from keys where id=?`, id); err != nil { + return err + } + } + return nil +} + +func (s *Store) ListKeys(ctx context.Context) ([]KeyRecord, error) { + rows, err := s.db.QueryContext(ctx, `select id, name, key_hash, enabled, preview, rpm, concurrency, max_active_sessions, models_json, prices_json, + five_hour_limit_usd, daily_limit_usd, weekly_limit_usd, monthly_limit_usd, archived, archived_at, + coalesce(source, ''), coalesce(source_present, 1), coalesce(alias, ''), coalesce(inherited_from, ''), + coalesce(inherit_conflict, 0), coalesce(hidden, 0), coalesce(last_enabled, enabled) + from keys order by hidden asc, name collate nocase`) + if err != nil { + return nil, err + } + defer rows.Close() + var out []KeyRecord + for rows.Next() { + var key KeyRecord + var enabled, archived, sourcePresent, inheritConflict, hidden, lastEnabled int + var modelsJSON, pricesJSON string + var fiveHour, daily, weekly, monthly sql.NullFloat64 + if err := rows.Scan( + &key.ID, &key.Name, &key.KeyHash, &enabled, &key.Preview, &key.RPM, &key.Concurrency, &key.MaxActiveSessions, + &modelsJSON, &pricesJSON, &fiveHour, &daily, &weekly, &monthly, + &archived, &key.ArchivedAt, &key.Source, &sourcePresent, &key.Alias, &key.InheritedFrom, + &inheritConflict, &hidden, &lastEnabled, + ); err != nil { + return nil, err + } + key.Enabled = enabled != 0 + key.Archived = archived != 0 + key.SourcePresent = sourcePresent != 0 + key.InheritConflict = inheritConflict != 0 + key.Hidden = hidden != 0 + key.LastEnabled = lastEnabled != 0 + key.FiveHourUSD = nullFloatPtr(fiveHour) + key.DailyLimitUSD = nullFloatPtr(daily) + key.WeeklyLimitUSD = nullFloatPtr(weekly) + key.MonthlyLimitUSD = nullFloatPtr(monthly) + _ = json.Unmarshal([]byte(modelsJSON), &key.Models) + _ = json.Unmarshal([]byte(pricesJSON), &key.Prices) + out = append(out, key) + } + return out, rows.Err() +} + +func (s *Store) SetArchived(ctx context.Context, id string, archived bool, at time.Time) error { + if strings.TrimSpace(id) == "" { + return fmt.Errorf("missing key id") + } + archivedAt := int64(0) + if archived { + archivedAt = at.Unix() + } + res, err := s.db.ExecContext(ctx, `update keys set archived=?, archived_at=?, updated_at=? where id=?`, + boolInt(archived), archivedAt, time.Now().Unix(), id) + if err != nil { + return err + } + if affected, _ := res.RowsAffected(); affected == 0 { + return fmt.Errorf("unknown key: %s", id) + } + action := "archive_key" + if !archived { + action = "restore_key" + } + return s.Audit(ctx, "admin", action, id, map[string]any{"archived": archived, "archived_at": archivedAt}) +} + +func (s *Store) DeleteKey(ctx context.Context, id string) error { + id = strings.TrimSpace(id) + if id == "" { + return fmt.Errorf("missing key id") + } + tx, err := s.db.BeginTx(ctx, nil) + if err != nil { + return err + } + defer tx.Rollback() + var name, preview string + if err := tx.QueryRowContext(ctx, `select name, preview from keys where id=?`, id).Scan(&name, &preview); err != nil { + if err == sql.ErrNoRows { + return fmt.Errorf("unknown key: %s", id) + } + return err + } + if _, err := tx.ExecContext(ctx, `delete from reset_watermarks where key_id=?`, id); err != nil { + return err + } + if _, err := tx.ExecContext(ctx, `delete from active_sessions where key_id=?`, id); err != nil { + return err + } + if _, err := tx.ExecContext(ctx, `delete from keys where id=?`, id); err != nil { + return err + } + raw, _ := json.Marshal(map[string]any{"name": name, "preview": preview, "kept_usage_history": true}) + if _, err := tx.ExecContext(ctx, `insert into audit_log(timestamp, actor, action, target, detail_json) values(?, ?, ?, ?, ?)`, + time.Now().Unix(), "admin", "delete_key", id, string(raw)); err != nil { + return err + } + return tx.Commit() +} + +func (s *Store) FindKeyByHash(ctx context.Context, hash string) (KeyRecord, bool, error) { + keys, err := s.ListKeys(ctx) + if err != nil { + return KeyRecord{}, false, err + } + normalized, err := NormalizeHash(hash) + if err != nil { + return KeyRecord{}, false, nil + } + for _, key := range keys { + keyHash, err := NormalizeHash(key.KeyHash) + if err == nil && keyHash == normalized { + return key, true, nil + } + } + return KeyRecord{}, false, nil +} + +func (s *Store) SetLimits(ctx context.Context, id string, fiveHour, monthly *float64) error { + for _, limit := range []*float64{fiveHour, monthly} { + if limit != nil && *limit < 0 { + return fmt.Errorf("usd limits must not be negative") + } + } + res, err := s.db.ExecContext(ctx, `update keys set five_hour_limit_usd=?, monthly_limit_usd=?, updated_at=? where id=?`, fiveHour, monthly, time.Now().Unix(), id) + if err != nil { + return err + } + if affected, _ := res.RowsAffected(); affected == 0 { + return fmt.Errorf("unknown key: %s", id) + } + return s.Audit(ctx, "admin", "set_limits", id, map[string]any{"five_hour_usd": fiveHour, "monthly_usd": monthly}) +} + +func (s *Store) SaveKeySettings(ctx context.Context, key KeyRecord) error { + if err := ValidateKeyRecord(key); err != nil { + return err + } + models, _ := json.Marshal(key.Models) + prices, _ := json.Marshal(key.Prices) + res, err := s.db.ExecContext(ctx, `update keys set + enabled=?, + rpm=?, + concurrency=?, + max_active_sessions=?, + models_json=?, + prices_json=?, + five_hour_limit_usd=?, + daily_limit_usd=?, + weekly_limit_usd=?, + monthly_limit_usd=?, + last_enabled=?, + updated_at=? + where id=?`, + boolInt(key.Enabled), + key.RPM, + key.Concurrency, + key.MaxActiveSessions, + string(models), + string(prices), + key.FiveHourUSD, + key.DailyLimitUSD, + key.WeeklyLimitUSD, + key.MonthlyLimitUSD, + boolInt(key.Enabled), + time.Now().Unix(), + key.ID, + ) + if err != nil { + return err + } + if affected, _ := res.RowsAffected(); affected == 0 { + return fmt.Errorf("unknown key: %s", key.ID) + } + return s.Audit(ctx, "admin", "save_key_settings", key.ID, map[string]any{ + "enabled": key.Enabled, + "rpm": key.RPM, + "concurrency": key.Concurrency, + "max_active_sessions": key.MaxActiveSessions, + }) +} + +func (s *Store) SyncNativeKeys(ctx context.Context, inputs []NativeKeySyncInput) error { + existing, err := s.ListKeys(ctx) + if err != nil { + return err + } + byID := make(map[string]KeyRecord, len(existing)) + for _, key := range existing { + byID[key.ID] = key + } + seen := map[string]bool{} + now := time.Now().Unix() + for _, input := range inputs { + next, ok := NativeKeyRecord(input.RawKey, input.Alias) + if !ok { + continue + } + seen[next.ID] = true + if current, exists := byID[next.ID]; exists { + enabled := current.Enabled + if isDefaultEmptyNativePolicy(current) { + enabled = true + } + current.KeyHash = next.KeyHash + current.Preview = next.Preview + current.Source = NativeCPASource + current.SourcePresent = true + current.Hidden = false + current.Alias = next.Alias + current.Name = next.Name + if _, err := s.db.ExecContext(ctx, `update keys set + name=?, key_hash=?, preview=?, enabled=?, source=?, source_present=1, hidden=0, alias=?, updated_at=? + where id=?`, + current.Name, current.KeyHash, current.Preview, boolInt(enabled), current.Source, current.Alias, now, current.ID); err != nil { + return err + } + continue + } + templates := policyTemplatesByAlias(existing, next.Alias) + if len(templates) == 1 { + next = CopyPolicyFields(next, templates[0]) + next.InheritedFrom = templates[0].ID + } else if len(templates) > 1 { + next.Enabled = false + next.InheritConflict = true + } + if err := s.UpsertKey(ctx, next); err != nil { + return err + } + if next.InheritedFrom != "" { + _ = s.Audit(ctx, "system", "native_key_policy_inherited", next.ID, map[string]any{"from": next.InheritedFrom, "alias": next.Alias}) + } + } + for _, key := range existing { + if key.Source != NativeCPASource || seen[key.ID] || !key.SourcePresent { + continue + } + if _, err := s.db.ExecContext(ctx, `update keys set enabled=0, source_present=0, hidden=1, last_enabled=?, updated_at=? where id=?`, boolInt(key.Enabled), now, key.ID); err != nil { + return err + } + _ = s.Audit(ctx, "system", "native_key_source_removed", key.ID, map[string]any{"alias": key.Alias, "preview": key.Preview}) + } + if err := s.retireInheritedLegacyTemplates(ctx, now); err != nil { + return err + } + return nil +} + +func isDefaultEmptyNativePolicy(key KeyRecord) bool { + return key.Source == NativeCPASource && + key.SourcePresent && + !key.Enabled && + !key.LastEnabled && + key.RPM == 0 && + key.Concurrency == 0 && + key.MaxActiveSessions == 0 && + len(key.Models) == 0 && + len(key.Prices) == 0 && + key.FiveHourUSD == nil && + key.DailyLimitUSD == nil && + key.WeeklyLimitUSD == nil && + key.MonthlyLimitUSD == nil && + !key.InheritConflict +} + +func policyTemplatesByAlias(keys []KeyRecord, alias string) []KeyRecord { + alias = strings.ToLower(strings.TrimSpace(alias)) + if alias == "" { + return nil + } + var out []KeyRecord + for _, key := range keys { + if templateAlias(key) != alias { + continue + } + if key.Source == NativeCPASource && !key.SourcePresent { + out = append(out, key) + continue + } + if key.Source == "" && key.SourcePresent && !key.Hidden { + out = append(out, key) + } + } + return out +} + +func templateAlias(key KeyRecord) string { + return strings.ToLower(strings.TrimSpace(firstNonEmpty(key.Alias, key.Name))) +} + +func (s *Store) retireInheritedLegacyTemplates(ctx context.Context, now int64) error { + rows, err := s.db.QueryContext(ctx, `select distinct inherited_from from keys where source=? and inherited_from != ''`, NativeCPASource) + if err != nil { + return err + } + var ids []string + for rows.Next() { + var id string + if err := rows.Scan(&id); err != nil { + _ = rows.Close() + return err + } + if strings.TrimSpace(id) == "" { + continue + } + ids = append(ids, id) + } + if err := rows.Err(); err != nil { + _ = rows.Close() + return err + } + if err := rows.Close(); err != nil { + return err + } + for _, id := range ids { + if _, err := s.db.ExecContext(ctx, `update keys set enabled=0, source=?, source_present=0, hidden=1, last_enabled=case when last_enabled != 0 then last_enabled else enabled end, updated_at=? where id=? and coalesce(source, '')=''`, + LegacyPlusSource, now, id); err != nil { + return err + } + } + return nil +} + +func LoadNativeKeysFromCPAConfig(path string) ([]string, error) { + path = strings.TrimSpace(path) + if path == "" { + return nil, nil + } + raw, err := os.ReadFile(path) + if err != nil { + return nil, err + } + var body struct { + APIKeys []string `yaml:"api-keys"` + } + if err := yaml.Unmarshal(raw, &body); err != nil { + return nil, err + } + out := make([]string, 0, len(body.APIKeys)) + for _, key := range body.APIKeys { + key = NormalizeSubmittedKey(key) + if key == "" { + continue + } + out = append(out, key) + } + return out, nil +} + +func LoadAPIKeyAliasesFromSQLite(ctx context.Context, path string) (map[string]string, error) { + path = strings.TrimSpace(path) + if path == "" { + return nil, nil + } + if _, err := os.Stat(path); err != nil { + return nil, err + } + db, err := openSQLite(path, true) + if err != nil { + return nil, err + } + defer db.Close() + exists, err := legacyTableExists(ctx, db, "api_key_aliases") + if err != nil { + return nil, err + } + if !exists { + return map[string]string{}, nil + } + rows, err := db.QueryContext(ctx, `select api_key_hash, alias from api_key_aliases`) + if err != nil { + return nil, err + } + defer rows.Close() + out := map[string]string{} + for rows.Next() { + var hash, alias string + if err := rows.Scan(&hash, &alias); err != nil { + return nil, err + } + normalized, err := NormalizeHash(hash) + if err != nil { + continue + } + if alias = strings.TrimSpace(alias); alias != "" { + out[normalized] = alias + } + } + return out, rows.Err() +} + +func LoadAPIKeyAliasesFromSQLitePaths(ctx context.Context, paths []string) (map[string]string, []string, error) { + out := map[string]string{} + var errs []string + for _, path := range paths { + path = strings.TrimSpace(path) + if path == "" { + continue + } + aliases, err := LoadAPIKeyAliasesFromSQLite(ctx, path) + if err != nil { + errs = append(errs, fmt.Sprintf("%s: %s", path, Brief(err.Error(), 160))) + continue + } + for hash, alias := range aliases { + if strings.TrimSpace(alias) == "" { + continue + } + out[hash] = alias + } + } + if len(errs) > 0 && len(out) == 0 { + return out, errs, fmt.Errorf(strings.Join(errs, "; ")) + } + return out, errs, nil +} + +func (s *Store) Reset(ctx context.Context, id, window string, at time.Time) error { + _, err := s.db.ExecContext(ctx, `insert into reset_watermarks(key_id, window, reset_at) values(?, ?, ?) + on conflict(key_id, window) do update set reset_at=excluded.reset_at`, id, window, at.Unix()) + if err != nil { + return err + } + return s.Audit(ctx, "admin", "reset_usage", id, map[string]any{"window": window, "reset_at": at.Unix()}) +} + +func (s *Store) InsertUsage(ctx context.Context, event UsageEvent) error { + if event.RequestedAt.IsZero() { + event.RequestedAt = time.Now() + } + breakdown, _ := json.Marshal(event.CostBreakdown) + _, err := s.db.ExecContext( + ctx, + `insert into usage_events( + request_id, key_id, key_preview, model, requested_model, actual_model, provider, executor_type, + endpoint, requested_at, latency_ms, ttft_ms, reasoning_effort, service_tier, status_code, failed, failure, + input_tokens, output_tokens, cached_tokens, cache_read_tokens, cache_creation_tokens, + reasoning_tokens, total_tokens, cost, cost_breakdown_json + ) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + event.RequestID, event.KeyID, event.KeyPreview, event.Model, event.RequestedModel, event.ActualModel, + event.Provider, event.ExecutorType, event.Endpoint, event.RequestedAt.Unix(), + event.LatencyMS, event.TTFTMS, event.ReasoningEffort, event.ServiceTier, event.StatusCode, + boolInt(event.Failed), event.Failure, + event.Usage.InputTokens, event.Usage.OutputTokens, event.Usage.CachedTokens, event.Usage.CacheReadTokens, + event.Usage.CacheCreationTokens, event.Usage.ReasoningTokens, event.Usage.TotalTokens, + event.Cost, string(breakdown), + ) + return err +} + +func (s *Store) UsageSum(ctx context.Context, keyID string, window Window) (float64, error) { + var total sql.NullFloat64 + from := window.From.Unix() + if resetAt, ok := s.ResetAt(ctx, keyID, window.Name); ok && resetAt > from { + from = resetAt + } + args := []any{from, window.To.Unix()} + query := `select coalesce(sum(cost), 0) from usage_events where requested_at >= ? and requested_at <= ?` + if keyID != "" && keyID != "all" { + query += ` and key_id = ?` + args = append(args, keyID) + } + if err := s.db.QueryRowContext(ctx, query, args...).Scan(&total); err != nil { + return 0, err + } + if !total.Valid { + return 0, nil + } + return total.Float64, nil +} + +func (s *Store) ResetAt(ctx context.Context, keyID, window string) (int64, bool) { + if s == nil || s.db == nil || keyID == "" || window == "" || keyID == "all" { + return 0, false + } + var resetAt sql.NullInt64 + if err := s.db.QueryRowContext(ctx, `select reset_at from reset_watermarks where key_id=? and window=?`, keyID, window).Scan(&resetAt); err != nil { + return 0, false + } + return resetAt.Int64, resetAt.Valid +} + +func (s *Store) UsageSummary(ctx context.Context, keyID string, window Window) (UsageSummary, error) { + from := window.From.Unix() + if resetAt, ok := s.ResetAt(ctx, keyID, window.Name); ok && resetAt > from { + from = resetAt + } + args := []any{from, window.To.Unix()} + query := `select + count(*), + coalesce(sum(case when failed != 0 then 1 else 0 end), 0), + coalesce(sum(cost), 0), + coalesce(sum(input_tokens), 0), + coalesce(sum(output_tokens), 0), + coalesce(sum(cached_tokens), 0), + coalesce(sum(cache_read_tokens), 0), + coalesce(sum(cache_creation_tokens), 0), + coalesce(sum(reasoning_tokens), 0), + coalesce(sum(total_tokens), 0) + from usage_events where requested_at >= ? and requested_at <= ?` + if keyID != "" && keyID != "all" { + query += ` and key_id = ?` + args = append(args, keyID) + } + var summary UsageSummary + if err := s.db.QueryRowContext(ctx, query, args...).Scan( + &summary.Calls, + &summary.Failed, + &summary.TotalCost, + &summary.Usage.InputTokens, + &summary.Usage.OutputTokens, + &summary.Usage.CachedTokens, + &summary.Usage.CacheReadTokens, + &summary.Usage.CacheCreationTokens, + &summary.Usage.ReasoningTokens, + &summary.Usage.TotalTokens, + ); err != nil { + return UsageSummary{}, err + } + return summary, nil +} + +func (s *Store) RecentEvents(ctx context.Context, keyID string, limit int) ([]UsageEvent, error) { + if limit <= 0 || limit > 200 { + limit = 100 + } + query := `select coalesce(request_id, ''), coalesce(key_id, ''), coalesce(key_preview, ''), coalesce(model, ''), + coalesce(requested_model, ''), coalesce(actual_model, ''), coalesce(provider, ''), coalesce(executor_type, ''), + coalesce(endpoint, ''), requested_at, coalesce(latency_ms, 0), coalesce(ttft_ms, 0), + coalesce(reasoning_effort, ''), coalesce(service_tier, ''), coalesce(status_code, 0), coalesce(failed, 0), coalesce(failure, ''), + coalesce(input_tokens, 0), coalesce(output_tokens, 0), coalesce(cached_tokens, 0), coalesce(cache_read_tokens, 0), + coalesce(cache_creation_tokens, 0), coalesce(reasoning_tokens, 0), coalesce(total_tokens, 0), coalesce(cost, 0), coalesce(cost_breakdown_json, '{}') + from usage_events` + args := []any{} + if keyID != "" && keyID != "all" { + query += ` where key_id = ?` + args = append(args, keyID) + } + query += ` order by requested_at desc, id desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []UsageEvent + for rows.Next() { + var event UsageEvent + var ts int64 + var failed int + var breakdown string + if err := rows.Scan( + &event.RequestID, &event.KeyID, &event.KeyPreview, &event.Model, &event.RequestedModel, &event.ActualModel, + &event.Provider, &event.ExecutorType, &event.Endpoint, &ts, &event.LatencyMS, &event.TTFTMS, + &event.ReasoningEffort, &event.ServiceTier, &event.StatusCode, &failed, &event.Failure, + &event.Usage.InputTokens, &event.Usage.OutputTokens, &event.Usage.CachedTokens, &event.Usage.CacheReadTokens, + &event.Usage.CacheCreationTokens, &event.Usage.ReasoningTokens, &event.Usage.TotalTokens, &event.Cost, &breakdown, + ); err != nil { + return nil, err + } + event.RequestedAt = time.Unix(ts, 0) + event.Failed = failed != 0 + _ = json.Unmarshal([]byte(breakdown), &event.CostBreakdown) + out = append(out, event) + } + return out, rows.Err() +} + +func (s *Store) RecentEventsWindow(ctx context.Context, keyID string, window Window, limit int) ([]UsageEvent, error) { + if limit <= 0 || limit > 200 { + limit = 100 + } + from := window.From.Unix() + if resetAt, ok := s.ResetAt(ctx, keyID, window.Name); ok && resetAt > from { + from = resetAt + } + query := `select coalesce(request_id, ''), coalesce(key_id, ''), coalesce(key_preview, ''), coalesce(model, ''), + coalesce(requested_model, ''), coalesce(actual_model, ''), coalesce(provider, ''), coalesce(executor_type, ''), + coalesce(endpoint, ''), requested_at, coalesce(latency_ms, 0), coalesce(ttft_ms, 0), + coalesce(reasoning_effort, ''), coalesce(service_tier, ''), coalesce(status_code, 0), coalesce(failed, 0), coalesce(failure, ''), + coalesce(input_tokens, 0), coalesce(output_tokens, 0), coalesce(cached_tokens, 0), coalesce(cache_read_tokens, 0), + coalesce(cache_creation_tokens, 0), coalesce(reasoning_tokens, 0), coalesce(total_tokens, 0), coalesce(cost, 0), coalesce(cost_breakdown_json, '{}') + from usage_events where requested_at >= ? and requested_at <= ?` + args := []any{from, window.To.Unix()} + if keyID != "" && keyID != "all" { + query += ` and key_id = ?` + args = append(args, keyID) + } + query += ` order by requested_at desc, id desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []UsageEvent + for rows.Next() { + var event UsageEvent + var ts int64 + var failed int + var breakdown string + if err := rows.Scan( + &event.RequestID, &event.KeyID, &event.KeyPreview, &event.Model, &event.RequestedModel, &event.ActualModel, + &event.Provider, &event.ExecutorType, &event.Endpoint, &ts, &event.LatencyMS, &event.TTFTMS, + &event.ReasoningEffort, &event.ServiceTier, &event.StatusCode, &failed, &event.Failure, + &event.Usage.InputTokens, &event.Usage.OutputTokens, &event.Usage.CachedTokens, &event.Usage.CacheReadTokens, + &event.Usage.CacheCreationTokens, &event.Usage.ReasoningTokens, &event.Usage.TotalTokens, &event.Cost, &breakdown, + ); err != nil { + return nil, err + } + event.RequestedAt = time.Unix(ts, 0) + event.Failed = failed != 0 + _ = json.Unmarshal([]byte(breakdown), &event.CostBreakdown) + out = append(out, event) + } + return out, rows.Err() +} + +func (s *Store) SaveCodexSummary(ctx context.Context, requestID, keyID, model, protection string, summary any) error { + raw, _ := json.Marshal(summary) + _, err := s.db.ExecContext(ctx, `insert into codexcont_summaries(request_id, key_id, model, protection, summary_json, updated_at) + values(?, ?, ?, ?, ?, ?) + on conflict(request_id) do update set key_id=excluded.key_id, model=excluded.model, + protection=excluded.protection, summary_json=excluded.summary_json, updated_at=excluded.updated_at`, + requestID, keyID, model, protection, string(raw), time.Now().Unix()) + return err +} + +func (s *Store) RecentCodexSummaries(ctx context.Context, keyID string, limit int) ([]CodexSummary, error) { + if limit <= 0 || limit > 200 { + limit = 100 + } + query := `select request_id, key_id, model, protection, summary_json, updated_at from codexcont_summaries` + args := []any{} + if keyID != "" && keyID != "all" { + query += ` where key_id = ?` + args = append(args, keyID) + } + query += ` order by updated_at desc limit ?` + args = append(args, limit) + rows, err := s.db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []CodexSummary + for rows.Next() { + var item CodexSummary + var raw string + var ts int64 + if err := rows.Scan(&item.RequestID, &item.KeyID, &item.Model, &item.Protection, &raw, &ts); err != nil { + return nil, err + } + item.UpdatedAt = time.Unix(ts, 0) + _ = json.Unmarshal([]byte(raw), &item.Summary) + if item.Summary == nil { + item.Summary = map[string]any{} + } + out = append(out, item) + } + return out, rows.Err() +} + +func RecentCodexSummariesFromSQLite(ctx context.Context, path, keyID string, limit int) ([]CodexSummary, error) { + path = strings.TrimSpace(path) + if path == "" { + return nil, nil + } + if _, err := os.Stat(path); err != nil { + return nil, err + } + db, err := openSQLite(path, true) + if err != nil { + return nil, err + } + defer db.Close() + if limit <= 0 || limit > 200 { + limit = 100 + } + query := `select request_id, key_id, model, protection, summary_json, updated_at from codexcont_summaries` + args := []any{} + if keyID != "" && keyID != "all" { + query += ` where key_id = ?` + args = append(args, keyID) + } + query += ` order by updated_at desc limit ?` + args = append(args, limit) + rows, err := db.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var out []CodexSummary + for rows.Next() { + var item CodexSummary + var raw string + var ts int64 + if err := rows.Scan(&item.RequestID, &item.KeyID, &item.Model, &item.Protection, &raw, &ts); err != nil { + return nil, err + } + item.UpdatedAt = time.Unix(ts, 0) + _ = json.Unmarshal([]byte(raw), &item.Summary) + if item.Summary == nil { + item.Summary = map[string]any{} + } + out = append(out, item) + } + return out, rows.Err() +} + +func (s *Store) Audit(ctx context.Context, actor, action, target string, detail any) error { + raw, _ := json.Marshal(detail) + _, err := s.db.ExecContext(ctx, `insert into audit_log(timestamp, actor, action, target, detail_json) values(?, ?, ?, ?, ?)`, + time.Now().Unix(), actor, action, target, string(raw)) + return err +} + +func (s *Store) AuditCount(ctx context.Context, action string) (int, error) { + var count int + args := []any{} + query := `select count(1) from audit_log` + if strings.TrimSpace(action) != "" { + query += ` where action = ?` + args = append(args, action) + } + err := s.db.QueryRowContext(ctx, query, args...).Scan(&count) + return count, err +} + +func (s *Store) LoadSettings(ctx context.Context) (map[string]string, error) { + rows, err := s.db.QueryContext(ctx, `select key, value from settings`) + if err != nil { + return nil, err + } + defer rows.Close() + out := map[string]string{} + for rows.Next() { + var key, value string + if err := rows.Scan(&key, &value); err != nil { + return nil, err + } + out[key] = value + } + return out, rows.Err() +} + +func (s *Store) SaveSettings(ctx context.Context, values map[string]string) error { + for key, value := range values { + if _, err := s.db.ExecContext(ctx, `insert into settings(key, value, updated_at) values(?, ?, ?) + on conflict(key) do update set value=excluded.value, updated_at=excluded.updated_at`, + key, value, time.Now().Unix()); err != nil { + return err + } + } + return s.Audit(ctx, "admin", "set_settings", "policyplus", values) +} + +func boolInt(value bool) int { + if value { + return 1 + } + return 0 +} + +func nullFloatPtr(value sql.NullFloat64) *float64 { + if !value.Valid { + return nil + } + v := value.Float64 + return &v +} + +func nullFloatValue(value sql.NullFloat64) any { + if !value.Valid { + return nil + } + return value.Float64 +} diff --git a/cpa_key_policy_plus_plugin/go/main.go b/cpa_key_policy_plus_plugin/go/main.go new file mode 100644 index 0000000..a5625e3 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/main.go @@ -0,0 +1,2471 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "net/url" + "os" + "sort" + "strconv" + "strings" + "sync" + "time" + + "codexcont/cpa-key-policy-plus-plugin/internal/policyplus" + _ "embed" + "gopkg.in/yaml.v3" +) + +const ( + pluginID = "cpa-key-policy-plus" + plusSessionCookieName = "cpa_key_policy_plus_session" + executorModelScopeBoth = "both" + executorFormatOpenAIResponse = "openai-response" + policyDenyMetadataPrefix = "policy_deny_" +) + +var executorUsageModelAliases = map[string]string{ + "gpt-5.3-codex-spark": "gpt-5.4", +} + +//go:embed assets/admin.html +var adminHTMLTemplate string + +//go:embed assets/user.html +var userHTMLTemplate string + +//go:embed assets/shared.css +var sharedCSSTemplate string + +type envelope struct { + OK bool `json:"ok"` + Result json.RawMessage `json:"result,omitempty"` + Error *envelopeError `json:"error,omitempty"` +} + +type envelopeError struct { + Code string `json:"code"` + Message string `json:"message"` +} + +type lifecycleRequest struct { + ConfigYAML []byte `json:"config_yaml"` +} + +type registration struct { + SchemaVersion uint32 `json:"schema_version"` + Metadata metadata `json:"metadata"` + Capabilities capabilities `json:"capabilities"` +} + +type metadata struct { + Name string `json:"Name"` + Version string `json:"Version"` + Author string `json:"Author"` + GitHubRepository string `json:"GitHubRepository"` + ConfigFields []configField `json:"ConfigFields"` +} + +type capabilities struct { + FrontendAuthProvider bool `json:"frontend_auth_provider"` + FrontendAuthProviderExclusive bool `json:"frontend_auth_provider_exclusive"` + ModelRouter bool `json:"model_router"` + Executor bool `json:"executor"` + ExecutorModelScope string `json:"executor_model_scope"` + ExecutorInputFormats []string `json:"executor_input_formats"` + ExecutorOutputFormats []string `json:"executor_output_formats"` + UsagePlugin bool `json:"usage_plugin"` + ManagementAPI bool `json:"management_api"` +} + +type runtimeState struct { + mu sync.RWMutex + cfg policyplus.Config + store *policyplus.Store + keyState policyplus.KeyPolicyState + keyStatePath string + keyStateModTime time.Time + keyStateLastCheck time.Time + rpmBuckets map[string][]time.Time +} + +type policyDecision struct { + Allowed bool + StatusCode int + Type string + Code string + Message string + Param string + Window string + UsedUSD float64 + LimitUSD float64 + UsedCount int + LimitCount int + KeyID string + KeyName string +} + +var state = runtimeState{ + rpmBuckets: map[string][]time.Time{}, +} + +func main() { runPreviewIfRequested() } + +func runPreviewIfRequested() { + addr := strings.TrimSpace(os.Getenv("CPA_KEY_POLICY_PLUS_PREVIEW_ADDR")) + if addr == "" { + return + } + cfg := policyplus.DefaultConfig() + previewDir, err := os.MkdirTemp("", "cpa-key-policy-plus-preview-*") + if err != nil { + panic(err) + } + cfg.StateDBPath = previewDir + "/policyplus.sqlite" + cfg.SessionSecret = "preview-secret" + cfg.CodexContEnabled = true + cfg.CodexContURL = "http://" + addr + store, err := policyplus.OpenStore(cfg.StateDBPath) + if err != nil { + panic(err) + } + previewRawKey := "cpa_preview_abcdefghijklmnopqrstuvwxyz0123456789AB" + previewKey := policyplus.KeyRecord{ + ID: "preview-key", + Name: "演示用户", + KeyHash: "sha256:" + policyplus.SHA256Hex(previewRawKey), + Enabled: true, + Preview: policyplus.HashPreview(policyplus.SHA256Hex(previewRawKey)), + RPM: 60, + Concurrency: 0, + Models: []string{"gpt-5.5", "gpt-5.4"}, + FiveHourUSD: floatPtr(2), + DailyLimitUSD: floatPtr(8), + WeeklyLimitUSD: floatPtr(40), + MonthlyLimitUSD: floatPtr(120), + Prices: map[string]policyplus.ModelPrice{ + "gpt-5.5": {Model: "gpt-5.5", InputPerMillion: 5, OutputPerMillion: 30, CacheReadPerMillion: 0.5}, + }, + } + disabledKey := policyplus.KeyRecord{ + ID: "preview-disabled", + Name: "禁用演示", + KeyHash: "sha256:" + policyplus.SHA256Hex("cpa_preview_disabled_abcdefghijklmnopqrstuvwxyz0123"), + Enabled: false, + Preview: policyplus.HashPreview(policyplus.SHA256Hex("cpa_preview_disabled_abcdefghijklmnopqrstuvwxyz0123")), + RPM: 30, + Concurrency: 0, + MaxActiveSessions: 0, + Models: []string{"gpt-5.4-mini"}, + DailyLimitUSD: floatPtr(3), + } + noLimitKey := policyplus.KeyRecord{ + ID: "preview-no-limit", + Name: "无限额演示", + KeyHash: "sha256:" + policyplus.SHA256Hex("cpa_preview_nolimit_abcdefghijklmnopqrstuvwxyz0123"), + Enabled: true, + Preview: policyplus.HashPreview(policyplus.SHA256Hex("cpa_preview_nolimit_abcdefghijklmnopqrstuvwxyz0123")), + RPM: 0, + Concurrency: 0, + MaxActiveSessions: 0, + } + _ = store.UpsertKey(context.Background(), previewKey) + _ = store.UpsertKey(context.Background(), disabledKey) + _ = store.UpsertKey(context.Background(), noLimitKey) + _ = store.InsertUsage(context.Background(), policyplus.UsageEvent{ + RequestID: "req-preview-a", + KeyID: previewKey.ID, + KeyPreview: previewKey.Preview, + Model: "gpt-5.5", + RequestedModel: "gpt-5.5", + ActualModel: "gpt-5.5", + Provider: "openai", + ExecutorType: "codex", + Endpoint: "/v1/responses", + RequestedAt: time.Now().Add(-3 * time.Minute), + LatencyMS: 14320, + TTFTMS: 1180, + ReasoningEffort: "high", + ServiceTier: "default", + StatusCode: 200, + Usage: policyplus.TokenUsage{ + InputTokens: 120000, + CachedTokens: 103000, + OutputTokens: 2100, + ReasoningTokens: 516, + TotalTokens: 122100, + }, + Cost: 0.151, + CostBreakdown: policyplus.CostForUsage(previewKey.Prices["gpt-5.5"], policyplus.TokenUsage{ + InputTokens: 120000, + CachedTokens: 103000, + OutputTokens: 2100, + ReasoningTokens: 516, + TotalTokens: 122100, + }, "gpt-5.5"), + }) + _ = store.SaveCodexSummary(context.Background(), "req-preview-a", previewKey.ID, "gpt-5.5", "auto_continued", map[string]any{ + "request_id": "req-preview-a", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": time.Now().Add(-2 * time.Minute).Format(time.RFC3339), + "updated_at": time.Now().Add(-90 * time.Second).Format(time.RFC3339), + "duration_ms": 5570, + "status": "completed", + "protection": "auto_continued", + "key_identity": previewKey.Safe(), + "latest_round": 2, + "latest_reasoning_tokens": 181, + "first_truncation_round": 1, + "first_truncation_reasoning_tokens": 516, + "first_truncation_decision": "continue", + "continuation_count": 1, + "stopped_reason": "completed", + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 516, "decision": "continue", "truncation_match": true}, {"round": 2, "reasoning_tokens": 181, "decision": "clean", "truncation_match": false}}, + }) + _ = store.InsertUsage(context.Background(), policyplus.UsageEvent{ + RequestID: "req-preview-disabled", + KeyID: disabledKey.ID, + KeyPreview: disabledKey.Preview, + Model: "gpt-5.4-mini", + RequestedAt: time.Now().Add(-25 * time.Minute), + Cost: 0.18, + }) + state.mu.Lock() + state.cfg = cfg + state.store = store + state.keyState = policyplus.KeyPolicyState{Keys: []policyplus.KeyRecord{previewKey, disabledKey, noLimitKey}} + state.rpmBuckets = map[string][]time.Time{} + state.mu.Unlock() + mux := http.NewServeMux() + mux.HandleFunc("/v0/resource/plugins/cpa-key-policy-plus/admin", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "text/html; charset=utf-8") + _, _ = w.Write([]byte(adminHTML())) + }) + mux.HandleFunc("/v0/resource/plugins/cpa-key-policy-plus/user", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "text/html; charset=utf-8") + _, _ = w.Write([]byte(userHTML())) + }) + writePreviewStatus := func(w http.ResponseWriter) { + w.Header().Set("content-type", "application/json; charset=utf-8") + _ = json.NewEncoder(w).Encode(map[string]any{ + "ok": true, + "counters": map[string]any{ + "total_requests": 4, + "active_requests": 17, + "continuations": 1, + "truncation_hits": 1, + "failures": 0, + }, + "last_error": nil, + }) + } + writePreviewRequests := func(w http.ResponseWriter) { + w.Header().Set("content-type", "application/json; charset=utf-8") + now := time.Now().UTC() + identity := previewKey.Safe() + _ = json.NewEncoder(w).Encode(map[string]any{"requests": []map[string]any{ + { + "request_id": "req-preview-a", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": now.Add(-4 * time.Minute).Format(time.RFC3339), + "updated_at": now.Add(-3 * time.Minute).Format(time.RFC3339), + "duration_ms": 5570, + "status": "completed", + "protection": "auto_continued", + "key_identity": identity, + "latest_round": 2, + "latest_reasoning_tokens": 181, + "first_truncation_round": 1, + "first_truncation_reasoning_tokens": 516, + "continuation_count": 1, + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 516, "decision": "continue", "truncation_match": true}, {"round": 2, "reasoning_tokens": 181, "decision": "clean", "truncation_match": false}}, + }, + { + "request_id": "req-preview-stale", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": now.Add(-45 * time.Minute).Format(time.RFC3339), + "updated_at": now.Add(-44 * time.Minute).Format(time.RFC3339), + "status": "processing", + "protection": "processing", + "key_identity": identity, + "latest_round": 1, + "latest_reasoning_tokens": 140, + "continuation_count": 0, + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 140, "decision": "clean", "truncation_match": false}}, + }, + { + "request_id": "req-preview-live", + "model": "gpt-5.5", + "path": "/v1/responses", + "started_at": now.Add(-30 * time.Second).Format(time.RFC3339), + "updated_at": now.Add(-5 * time.Second).Format(time.RFC3339), + "status": "processing", + "protection": "processing", + "key_identity": identity, + "latest_round": 1, + "latest_reasoning_tokens": 140, + "continuation_count": 0, + "rounds": []map[string]any{{"round": 1, "reasoning_tokens": 140, "decision": "clean", "truncation_match": false}}, + }, + }}) + } + mux.HandleFunc("/governor/codexcont/admin/status", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewStatus(w) + }) + mux.HandleFunc("/admin/status", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewStatus(w) + }) + mux.HandleFunc("/governor/codexcont/admin/requests", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewRequests(w) + }) + mux.HandleFunc("/admin/requests", func(w http.ResponseWriter, r *http.Request) { + _ = r + writePreviewRequests(w) + }) + mux.HandleFunc("/governor/codexcont/admin/logs/stream", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "text/event-stream") + w.Header().Set("cache-control", "no-cache") + _, _ = w.Write([]byte("event: ready\ndata: {\"ok\":true}\n\n")) + _, _ = w.Write([]byte("event: log\ndata: {\"ts\":\"2026-07-02T02:09:59Z\",\"level\":\"info\",\"event\":\"round_decision\",\"message\":\"preview round decision\",\"fields\":{\"request_id\":\"req-preview-a\"}}\n\n")) + }) + mux.HandleFunc("/engine/healthz", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.Header().Set("content-type", "application/json; charset=utf-8") + _, _ = w.Write([]byte(`{"ok":true,"mode":"preview"}`)) + }) + mux.HandleFunc("/favicon.ico", func(w http.ResponseWriter, r *http.Request) { + _ = r + w.WriteHeader(http.StatusNoContent) + }) + mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + rawReq, _ := json.Marshal(managementRequest{ + Method: r.Method, + Path: r.URL.Path, + Headers: r.Header, + Query: r.URL.Query(), + Body: body, + }) + rawResp, _ := managementHandle(rawReq) + var env envelope + _ = json.Unmarshal(rawResp, &env) + var resp managementResponse + _ = json.Unmarshal(env.Result, &resp) + for key, values := range resp.Headers { + for _, value := range values { + w.Header().Add(key, value) + } + } + if resp.StatusCode != 0 { + w.WriteHeader(resp.StatusCode) + } + _, _ = w.Write(resp.Body) + }) + if err := http.ListenAndServe(addr, mux); err != nil { + panic(err) + } +} + +func floatPtr(value float64) *float64 { return &value } + +func shutdownPlugin() { + state.mu.Lock() + defer state.mu.Unlock() + if state.store != nil { + _ = state.store.Close() + state.store = nil + } +} + +func handleMethod(method string, request []byte) ([]byte, error) { + switch method { + case methodPluginRegister, methodPluginReconfigure: + if err := configure(request); err != nil { + return nil, err + } + return okEnvelope(pluginRegistration()) + case methodFrontendAuthIdentifier: + return okEnvelope(map[string]string{"identifier": pluginID}) + case methodFrontendAuthAuthenticate: + return frontendAuth(request) + case methodModelRoute: + return routeModel(request) + case methodExecutorIdentifier: + return okEnvelope(map[string]string{"identifier": pluginID}) + case methodExecutorExecute: + return executorExecute(request) + case methodExecutorExecuteStream: + return executorExecuteStream(request) + case methodExecutorCountTokens: + return okEnvelope(executorResponse{Payload: []byte(`{"input_tokens":0}`)}) + case methodUsageHandle: + return usageHandle(request) + case methodManagementRegister: + return managementRegister() + case methodManagementHandle: + return managementHandle(request) + default: + return errorEnvelope("unknown_method", "unknown method: "+method), nil + } +} + +func configure(raw []byte) error { + cfg := policyplus.DefaultConfig() + if len(raw) > 0 { + var req lifecycleRequest + if err := json.Unmarshal(raw, &req); err != nil { + return err + } + if len(req.ConfigYAML) > 0 { + if err := yaml.Unmarshal(req.ConfigYAML, &cfg); err != nil { + return err + } + } + } + cfg = cfg.Normalize() + store, err := policyplus.OpenStore(cfg.StateDBPath) + if err != nil { + return err + } + cfg = applyStoredSettings(cfg, store) + var keyState policyplus.KeyPolicyState + if strings.TrimSpace(cfg.KeyPolicyStatePath) != "" { + loaded, err := policyplus.LoadKeyPolicyState(cfg.KeyPolicyStatePath) + if err == nil { + keyState = loaded + _ = store.ImportKeys(context.Background(), loaded) + } + } + importLegacySQLite(store, cfg.LegacyQuotaDBPath, "usage-admin") + importLegacySQLite(store, cfg.GovernorStateDBPath, "governor") + syncNativeKeysFromConfig(store, cfg) + state.mu.Lock() + old := state.store + state.cfg = cfg + state.store = store + state.keyState = keyState + state.keyStatePath = strings.TrimSpace(cfg.KeyPolicyStatePath) + state.keyStateModTime = time.Time{} + state.keyStateLastCheck = time.Time{} + state.mu.Unlock() + if old != nil { + _ = old.Close() + } + _ = refreshKeyPolicyState(true) + _ = syncNativeKeysFromLoadedConfig() + return nil +} + +func syncNativeKeysFromLoadedConfig() error { + state.mu.RLock() + cfg := state.cfg + store := state.store + state.mu.RUnlock() + return syncNativeKeysFromConfig(store, cfg) +} + +func syncNativeKeysFromConfig(store *policyplus.Store, cfg policyplus.Config) error { + if store == nil || strings.TrimSpace(cfg.NativeKeysConfigPath) == "" { + return nil + } + rawKeys, err := policyplus.LoadNativeKeysFromCPAConfig(cfg.NativeKeysConfigPath) + if err != nil { + _ = store.Audit(context.Background(), "system", "native_key_sync_failed", "cpa_config", map[string]any{"error": policyplus.Brief(err.Error(), 240)}) + return err + } + aliasPaths := cfg.AliasDBPaths() + aliases, aliasErrors, err := policyplus.LoadAPIKeyAliasesFromSQLitePaths(context.Background(), aliasPaths) + if len(aliasErrors) > 0 { + _ = store.Audit(context.Background(), "system", "native_alias_read_partial", "cpamp", map[string]any{"errors": aliasErrors}) + } + if err != nil && len(aliasPaths) > 0 { + _ = store.Audit(context.Background(), "system", "native_alias_read_failed", "cpamp", map[string]any{"error": policyplus.Brief(err.Error(), 240)}) + } + inputs := make([]policyplus.NativeKeySyncInput, 0, len(rawKeys)) + for _, rawKey := range rawKeys { + hash := policyplus.SHA256Hex(rawKey) + alias := aliases[hash] + inputs = append(inputs, policyplus.NativeKeySyncInput{RawKey: rawKey, Alias: alias}) + } + if err := store.SyncNativeKeys(context.Background(), inputs); err != nil { + _ = store.Audit(context.Background(), "system", "native_key_sync_failed", "store", map[string]any{"error": policyplus.Brief(err.Error(), 240)}) + return err + } + return nil +} + +func applyStoredSettings(cfg policyplus.Config, store *policyplus.Store) policyplus.Config { + if store == nil { + return cfg + } + settings, err := store.LoadSettings(context.Background()) + if err != nil { + return cfg + } + if value, ok := settings["codexcont_enabled"]; ok { + if parsed, err := strconv.ParseBool(value); err == nil { + cfg.CodexContEnabled = parsed + } + } + if value := strings.TrimSpace(settings["codexcont_url"]); value != "" { + cfg.CodexContURL = strings.TrimRight(value, "/") + } + if value := strings.TrimSpace(settings["fail_mode"]); value != "" { + cfg.FailMode = strings.ToLower(value) + } + return cfg.Normalize() +} + +func importLegacySQLite(store *policyplus.Store, path, source string) { + path = strings.TrimSpace(path) + if store == nil || path == "" { + return + } + if _, err := os.Stat(path); err != nil { + _ = store.Audit(context.Background(), "system", "legacy_import_skipped", source, map[string]any{ + "path": path, + "error": policyplus.Brief(err.Error(), 240), + }) + return + } + result, err := store.ImportLegacyQuotaSQLite(context.Background(), path, source) + if err != nil { + _ = store.Audit(context.Background(), "system", "legacy_import_failed", source, map[string]any{ + "path": path, + "error": policyplus.Brief(err.Error(), 240), + }) + return + } + if result.Limits > 0 || result.Resets > 0 { + _ = store.Audit(context.Background(), "system", "legacy_import_completed", source, map[string]any{ + "path": path, + "limits": result.Limits, + "resets": result.Resets, + }) + } +} + +func pluginRegistration() registration { + cfg := loadedConfig() + return registration{ + SchemaVersion: schemaVersion, + Metadata: metadata{ + Name: "cpa-key-policy-plus", + Version: "0.1.0", + Author: "konbakuyomu/CodexCont", + GitHubRepository: "https://local/CodexCont", + ConfigFields: []configField{ + {Name: "enabled", Type: configBoolean, Description: "Enable Key Policy Plus user-key authentication and management surfaces."}, + {Name: "exclusive_auth", Type: configBoolean, Description: "When true, Key Policy Plus is the exclusive frontend auth provider for user keys."}, + {Name: "state_db_path", Type: configString, Description: "SQLite path for Key Policy Plus state."}, + {Name: "key_policy_state_path", Type: configString, Description: "Optional old CPA Key Policy state JSON path to import once or mirror during cutover."}, + {Name: "legacy_quota_db_path", Type: configString, Description: "Optional old usage-admin SQLite path for one-way 5H/month limit and reset import."}, + {Name: "governor_state_db_path", Type: configString, Description: "Optional old Governor SQLite path for one-way limit and reset import."}, + {Name: "codex_summary_db_path", Type: configString, Description: "Optional read-only CodexCont executor SQLite path for safe protection summaries."}, + {Name: "native_keys_config_path", Type: configString, Description: "Optional CPA config YAML path whose top-level api-keys are synced as native policy keys."}, + {Name: "cpamp_alias_db_path", Type: configString, Description: "Optional CPAMP manager SQLite path for read-only api_key_aliases lookup."}, + {Name: "cpamp_alias_db_paths", Type: configString, Description: "Optional comma/semicolon-separated fallback CPAMP SQLite paths for read-only api_key_aliases lookup."}, + {Name: "session_secret", Type: configString, Description: "Secret used to sign user portal sessions."}, + {Name: "codexcont_enabled", Type: configBoolean, Description: "Enable CodexCont status lookup for user summaries."}, + {Name: "codexcont_route", Type: configBoolean, Description: "Deprecated in Key Policy Plus; keep false and let Governor own CodexCont routing."}, + {Name: "codexcont_url", Type: configString, Description: "Internal CodexCont engine base URL."}, + {Name: "fail_mode", Type: configEnum, EnumValues: []string{"fallback", "fail_closed"}, Description: "Behavior when engine is unavailable."}, + }, + }, + Capabilities: capabilities{ + FrontendAuthProvider: true, + FrontendAuthProviderExclusive: cfg.ExclusiveAuth, + ModelRouter: true, + Executor: true, + ExecutorModelScope: executorModelScopeBoth, + ExecutorInputFormats: []string{executorFormatOpenAIResponse}, + ExecutorOutputFormats: []string{executorFormatOpenAIResponse}, + UsagePlugin: true, + ManagementAPI: true, + }, + } +} + +func loadedConfig() policyplus.Config { + state.mu.RLock() + defer state.mu.RUnlock() + if state.cfg.StateDBPath == "" { + return policyplus.DefaultConfig() + } + return state.cfg +} + +func loadedStore() *policyplus.Store { + state.mu.RLock() + defer state.mu.RUnlock() + return state.store +} + +func frontendAuth(raw []byte) ([]byte, error) { + var req frontendAuthRequest + if err := json.Unmarshal(raw, &req); err != nil { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + cfg := loadedConfig() + if !cfg.Enabled { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + key := bearer(req.Headers.Get("Authorization")) + if key == "" { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + model := requestedModelFromBody(req.Body) + record, decision, ok := policyDecisionForRawKey(key, model, true) + if !ok { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + metadata := authMetadata(record) + if !decision.Allowed { + if !shouldSurfacePolicyDeny(req) { + return okEnvelope(frontendAuthResponse{Authenticated: false}) + } + metadata = decisionMetadata(metadata, decision) + } + return okEnvelope(frontendAuthResponse{ + Authenticated: true, + Principal: record.ID, + Metadata: metadata, + }) +} + +func shouldSurfacePolicyDeny(req frontendAuthRequest) bool { + path := strings.ToLower(strings.TrimSpace(req.Path)) + if strings.Contains(path, "/v1/responses") || strings.Contains(path, "/responses") { + return true + } + return requestedModelFromBody(req.Body) != "" +} + +func policyDecisionForRawKey(rawKey, model string, consumeRPM bool) (policyplus.KeyRecord, policyDecision, bool) { + submitted := policyplus.NormalizeSubmittedKey(rawKey) + if strings.HasPrefix(strings.ToLower(submitted), "cpa_") { + return policyplus.KeyRecord{}, policyDecision{}, false + } + record, ok := findKeyByRaw(rawKey) + if ok { + return record, evaluatePolicy(record, model, consumeRPM), true + } + if !isNativeSubmittedKey(submitted) { + return policyplus.KeyRecord{}, policyDecision{}, false + } + hash := policyplus.SHA256Hex(submitted) + preview := policyplus.HashPreview(hash) + record = policyplus.KeyRecord{ + ID: policyplus.NativeKeyIDFromHash(hash), + Name: preview, + KeyHash: "sha256:" + hash, + Preview: preview, + Source: policyplus.NativeCPASource, + SourcePresent: false, + } + base := policyDecision{ + Allowed: true, + StatusCode: http.StatusOK, + KeyID: record.ID, + KeyName: preview, + } + decision := denyDecision(base, "invalid_request_error", "policy_missing", "policy_missing", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 没有对应的 Plus 策略，请先在管理页同步并启用策略。", preview)) + return record, decision, true +} + +func isNativeSubmittedKey(key string) bool { + lower := strings.ToLower(strings.TrimSpace(key)) + return strings.HasPrefix(lower, "sk-") || strings.HasPrefix(lower, "sk_") +} + +func authMetadata(record policyplus.KeyRecord) map[string]string { + return map[string]string{ + "provider": "cpa-key-policy-plus", + "key_id": record.ID, + "key_name": record.Name, + "preview": record.Preview, + "source": record.Source, + "source_present": strconv.FormatBool(record.SourcePresent), + } +} + +func decisionMetadata(metadata map[string]string, decision policyDecision) map[string]string { + out := map[string]string{} + for k, v := range metadata { + out[k] = v + } + out[policyDenyMetadataPrefix+"code"] = decision.Code + out[policyDenyMetadataPrefix+"message"] = decision.Message + out[policyDenyMetadataPrefix+"window"] = decision.Window + out[policyDenyMetadataPrefix+"param"] = decision.Param + return out +} + +func findKeyByRaw(rawKey string) (policyplus.KeyRecord, bool) { + _ = syncNativeKeysFromLoadedConfig() + _ = refreshKeyPolicyState(false) + if key, ok := lookupKeyByRaw(rawKey); ok { + return key, true + } + if strings.HasPrefix(strings.ToLower(policyplus.NormalizeSubmittedKey(rawKey)), "cpa_") { + _ = refreshKeyPolicyState(true) + return lookupKeyByRaw(rawKey) + } + return policyplus.KeyRecord{}, false +} + +func lookupKeyByRaw(rawKey string) (policyplus.KeyRecord, bool) { + state.mu.RLock() + keyState := state.keyState + store := state.store + state.mu.RUnlock() + if store != nil { + key, ok, err := store.FindKeyByHash(context.Background(), policyplus.SHA256Hex(rawKey)) + if err == nil && ok { + return key, true + } + } + if key, ok := keyState.FindByRawKey(rawKey); ok { + return key, true + } + return policyplus.KeyRecord{}, false +} + +func refreshKeyPolicyState(force bool) error { + state.mu.RLock() + path := strings.TrimSpace(state.keyStatePath) + lastMod := state.keyStateModTime + store := state.store + state.mu.RUnlock() + if path == "" || store == nil { + return nil + } + now := time.Now() + info, err := os.Stat(path) + if err != nil { + state.mu.Lock() + state.keyStateLastCheck = now + state.mu.Unlock() + return err + } + if !force && !info.ModTime().After(lastMod) { + state.mu.Lock() + state.keyStateLastCheck = now + state.mu.Unlock() + return nil + } + loaded, err := policyplus.LoadKeyPolicyState(path) + if err != nil { + state.mu.Lock() + state.keyStateLastCheck = now + state.mu.Unlock() + return err + } + if err := store.ImportKeys(context.Background(), loaded); err != nil { + return err + } + state.mu.Lock() + state.keyState = loaded + state.keyStateModTime = info.ModTime() + state.keyStateLastCheck = now + state.mu.Unlock() + return nil +} + +func routeModel(raw []byte) ([]byte, error) { + var req modelRouteRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + cfg := loadedConfig() + if !cfg.Enabled { + return okEnvelope(modelRouteResponse{Handled: false}) + } + decision, ok := policyDecisionForHeaders(req.Headers, firstNonEmpty(req.RequestedModel, requestedModelFromBody(req.Body)), false) + if !ok || decision.Allowed { + return okEnvelope(modelRouteResponse{Handled: false}) + } + return okEnvelope(modelRouteResponse{ + Handled: true, + TargetKind: routeTargetSelf, + Reason: "cpa_key_policy_plus_policy_denied", + }) +} + +func isResponsesRequest(source string, body []byte) bool { + source = strings.ToLower(strings.TrimSpace(source)) + if strings.Contains(source, "response") || source == "openai" { + return true + } + return strings.Contains(string(body), `"stream"`) && strings.Contains(string(body), `"model"`) +} + +func requestedModelFromBody(body []byte) string { + if len(body) == 0 { + return "" + } + var raw map[string]any + if err := json.Unmarshal(body, &raw); err != nil { + return "" + } + if text, ok := raw["model"].(string); ok { + return strings.TrimSpace(text) + } + return "" +} + +func policyDecisionForHeaders(headers http.Header, model string, consumeRPM bool) (policyDecision, bool) { + rawKey := bearer(headers.Get("Authorization")) + if rawKey == "" { + return policyDecision{}, false + } + _, decision, ok := policyDecisionForRawKey(rawKey, model, consumeRPM) + if !ok { + return policyDecision{}, false + } + return decision, true +} + +func evaluatePolicy(key policyplus.KeyRecord, model string, consumeRPM bool) policyDecision { + base := policyDecision{ + Allowed: true, + StatusCode: http.StatusOK, + KeyID: key.ID, + KeyName: safeKeyDisplayName(key), + } + if key.ID == "" { + return denyDecision(base, "invalid_request_error", "missing_policy", "policy_missing", "", "CPA Key Policy+ 已拦截：当前 Key 没有对应的 Plus 策略。") + } + if key.Source == policyplus.NativeCPASource && !key.SourcePresent { + return denyDecision(base, "invalid_request_error", "api_key_source_removed", "source_removed", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 已从 CPA 官方 Key 列表移除。", safeKeyDisplayName(key))) + } + if !key.Enabled || key.Archived { + return denyDecision(base, "invalid_request_error", "api_key_disabled", "disabled", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 当前已禁用。", safeKeyDisplayName(key))) + } + if model != "" && !policyplus.ModelAllowed(key.Models, model) { + return denyDecision(base, "invalid_request_error", "model_not_allowed", "model", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 不允许使用模型 %s。", safeKeyDisplayName(key), model)) + } + if rpm := checkRPM(key, consumeRPM); !rpm.Allowed { + base.UsedCount = rpm.Used + base.LimitCount = rpm.Limit + base.Window = "rpm" + return denyDecision(base, "rate_limit_exceeded", "rpm_rate_limit_exceeded", "rpm", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 触发 RPM 限制，最近 1 分钟请求 %d / 上限 %d。", safeKeyDisplayName(key), rpm.Used, rpm.Limit)) + } + if quota := checkQuota(key); !quota.Allowed { + base.Window = quota.Window + base.Param = quota.Window + base.UsedUSD = quota.Used + base.LimitUSD = quota.Limit + return denyDecision(base, "rate_limit_exceeded", quota.Code, quota.Window, quota.Window, fmt.Sprintf("CPA Key Policy+ 已拦截：%s 触发 %s费用限额，已用 $%.2f / 上限 $%.2f。", safeKeyDisplayName(key), windowDisplayName(quota.Window), quota.Used, quota.Limit)) + } + return base +} + +func evaluateIdentityPolicy(key policyplus.KeyRecord) policyDecision { + base := policyDecision{ + Allowed: true, + StatusCode: http.StatusOK, + KeyID: key.ID, + KeyName: safeKeyDisplayName(key), + } + if key.ID == "" { + return denyDecision(base, "invalid_request_error", "missing_policy", "policy_missing", "", "CPA Key Policy+ 已拦截：当前 Key 没有对应的 Plus 策略。") + } + if key.Source == policyplus.NativeCPASource && !key.SourcePresent { + return denyDecision(base, "invalid_request_error", "api_key_source_removed", "source_removed", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 已从 CPA 官方 Key 列表移除。", safeKeyDisplayName(key))) + } + if !key.Enabled || key.Archived { + return denyDecision(base, "invalid_request_error", "api_key_disabled", "disabled", "", fmt.Sprintf("CPA Key Policy+ 已拦截：%s 当前已禁用。", safeKeyDisplayName(key))) + } + return base +} + +func denyDecision(base policyDecision, typ, code, reason, param, message string) policyDecision { + base.Allowed = false + base.StatusCode = http.StatusTooManyRequests + base.Type = typ + base.Code = code + base.Param = param + if base.Param == "" { + base.Param = reason + } + base.Message = message + return base +} + +func safeKeyDisplayName(key policyplus.KeyRecord) string { + for _, value := range []string{key.Name, key.Alias, key.Preview, key.ID} { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "当前 Key" +} + +type rpmDecision struct { + Allowed bool + Used int + Limit int +} + +func checkRPM(key policyplus.KeyRecord, consume bool) rpmDecision { + if key.RPM <= 0 { + return rpmDecision{Allowed: true} + } + state.mu.Lock() + defer state.mu.Unlock() + now := time.Now() + cutoff := now.Add(-time.Minute) + bucket := state.rpmBuckets[key.ID] + kept := bucket[:0] + for _, ts := range bucket { + if ts.After(cutoff) { + kept = append(kept, ts) + } + } + if len(kept) >= key.RPM { + state.rpmBuckets[key.ID] = kept + return rpmDecision{Allowed: false, Used: len(kept), Limit: key.RPM} + } + if consume { + state.rpmBuckets[key.ID] = append(kept, now) + } else { + state.rpmBuckets[key.ID] = kept + } + return rpmDecision{Allowed: true, Used: len(kept), Limit: key.RPM} +} + +type quotaDecision struct { + Allowed bool + Window string + Used float64 + Limit float64 + Code string +} + +func checkQuota(key policyplus.KeyRecord) quotaDecision { + store := loadedStore() + if store == nil { + return quotaDecision{Allowed: true} + } + ctx := context.Background() + now := time.Now() + limits := []struct { + name string + limit *float64 + code string + }{ + {policyplus.Range5H, key.FiveHourUSD, "five_hour_quota_exceeded"}, + {policyplus.Range24H, key.DailyLimitUSD, "daily_quota_exceeded"}, + {policyplus.Range7D, key.WeeklyLimitUSD, "weekly_quota_exceeded"}, + {policyplus.RangeMonth, key.MonthlyLimitUSD, "monthly_quota_exceeded"}, + } + for _, item := range limits { + if item.limit == nil { + continue + } + used, err := store.UsageSum(ctx, key.ID, policyplus.WindowFor(item.name, now)) + if err != nil { + continue + } + if !policyplus.CheckLimit(used, item.limit).Allowed { + return quotaDecision{Allowed: false, Window: item.name, Used: used, Limit: *item.limit, Code: item.code} + } + } + return quotaDecision{Allowed: true} +} + +func windowDisplayName(window string) string { + switch window { + case policyplus.Range5H: + return "5小时" + case policyplus.Range24H: + return "24小时" + case policyplus.Range7D: + return "7天" + case policyplus.RangeMonth: + return "本月" + default: + return window + } +} + +func executorUnavailable() ([]byte, error) { + cfg := loadedConfig() + if cfg.CodexContRoute && cfg.FailMode == "fail_closed" { + return errorEnvelope("codexcont_executor_pending", "CPA Key Policy+ protected executor is not enabled in this build"), nil + } + return errorEnvelope("codexcont_executor_pending", "CPA Key Policy+ protected executor is in passive mode; disable codexcont_enabled to route through CPA provider path"), nil +} + +func executorExecute(raw []byte) ([]byte, error) { + var req executorCallRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + execReq := normalizedExecutorRequest(req) + if decision, ok := policyDenyForExecutor(execReq); ok { + return okEnvelope(policyDenyExecutorResponse(decision)) + } + cfg := loadedConfig() + if !cfg.CodexContRoute { + return executorUnavailable() + } + payload := execReq.Payload + if len(payload) == 0 { + payload = execReq.OriginalRequest + } + result, err := callHost(methodHostModelExecute, hostModelExecutionRequest{ + EntryProtocol: firstNonEmpty(execReq.SourceFormat, "openai"), + ExitProtocol: firstNonEmpty(execReq.Format, "openai"), + Model: execReq.Model, + Stream: false, + Body: payload, + Headers: cloneHeader(execReq.Headers), + Query: cloneValues(execReq.Query), + Alt: execReq.Alt, + HostCallbackID: req.HostCallbackID, + }) + if err != nil { + return errorEnvelope("host_model_execute_error", err.Error()), nil + } + var resp hostModelExecutionResponse + if err := json.Unmarshal(result, &resp); err != nil { + return errorEnvelope("host_model_execute_decode_error", err.Error()), nil + } + if resp.StatusCode >= 400 { + return errorEnvelope("host_model_execute_http_error", fmt.Sprintf("upstream returned %d", resp.StatusCode)), nil + } + return okEnvelope(executorResponse{Payload: resp.Body, Headers: resp.Headers}) +} + +func executorExecuteStream(raw []byte) ([]byte, error) { + var req executorCallRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + execReq := normalizedExecutorRequest(req) + if decision, ok := policyDenyForExecutor(execReq); ok { + if strings.TrimSpace(req.StreamID) == "" { + return okEnvelope(executorStreamResponse{Headers: policyDenyHeaders(decision)}) + } + go emitPolicyDenyStream(req.StreamID, decision) + return okEnvelope(executorStreamResponse{Headers: policyDenyHeaders(decision)}) + } + cfg := loadedConfig() + if !cfg.CodexContRoute { + return executorUnavailable() + } + if strings.TrimSpace(req.StreamID) == "" { + return errorEnvelope("stream_id_required", "stream_id is required for executor.execute_stream"), nil + } + payload := execReq.Payload + if len(payload) == 0 { + payload = execReq.OriginalRequest + } + result, err := callHost(methodHostModelExecuteStream, hostModelExecutionRequest{ + EntryProtocol: firstNonEmpty(execReq.SourceFormat, "openai"), + ExitProtocol: firstNonEmpty(execReq.Format, "openai"), + Model: execReq.Model, + Stream: true, + Body: payload, + Headers: cloneHeader(execReq.Headers), + Query: cloneValues(execReq.Query), + Alt: execReq.Alt, + HostCallbackID: req.HostCallbackID, + }) + if err != nil { + return errorEnvelope("host_model_stream_error", err.Error()), nil + } + var resp hostModelStreamResponse + if err := json.Unmarshal(result, &resp); err != nil { + return errorEnvelope("host_model_stream_decode_error", err.Error()), nil + } + if resp.StatusCode >= 400 { + return errorEnvelope("host_model_stream_http_error", fmt.Sprintf("upstream returned %d", resp.StatusCode)), nil + } + if resp.StreamID == "" { + return errorEnvelope("host_model_stream_empty", "host returned empty stream id"), nil + } + go forwardHostStream(req.StreamID, resp.StreamID) + return okEnvelope(executorStreamResponse{Headers: resp.Headers}) +} + +func policyDenyForExecutor(req executorRequest) (policyDecision, bool) { + model := firstNonEmpty(req.Model, requestedModelFromBody(req.OriginalRequest), requestedModelFromBody(req.Payload)) + decision, ok := policyDecisionForHeaders(req.Headers, model, false) + if !ok || decision.Allowed { + return policyDecision{}, false + } + return decision, true +} + +func normalizedExecutorRequest(req executorCallRequest) executorRequest { + if !isZeroExecutorRequest(req.executorRequest) { + return req.executorRequest + } + return req.NestedExecutorRequest +} + +func isZeroExecutorRequest(req executorRequest) bool { + return strings.TrimSpace(req.AuthID) == "" && + strings.TrimSpace(req.AuthProvider) == "" && + strings.TrimSpace(req.Model) == "" && + strings.TrimSpace(req.Format) == "" && + !req.Stream && + strings.TrimSpace(req.Alt) == "" && + len(req.Headers) == 0 && + len(req.Query) == 0 && + len(req.OriginalRequest) == 0 && + strings.TrimSpace(req.SourceFormat) == "" && + len(req.Payload) == 0 && + len(req.Metadata) == 0 && + len(req.StorageJSON) == 0 && + len(req.AuthMetadata) == 0 && + len(req.AuthAttributes) == 0 +} + +func policyDenyExecutorResponse(decision policyDecision) executorResponse { + return executorResponse{ + Payload: policyDenyBody(decision), + Headers: policyDenyHeaders(decision), + Metadata: map[string]any{ + "policy_denied": true, + "code": decision.Code, + "window": decision.Window, + }, + } +} + +func policyDenyHeaders(decision policyDecision) http.Header { + headers := http.Header{} + headers.Set("Content-Type", "application/json; charset=utf-8") + headers.Set("X-CPA-Policy-Reason", decision.Code) + if window := firstNonEmpty(decision.Window, decision.Param); window != "" { + headers.Set("X-CPA-Policy-Window", window) + } + headers.Set("Retry-After", "60") + return headers +} + +func policyDenyBody(decision policyDecision) []byte { + body, _ := json.Marshal(map[string]any{ + "error": map[string]any{ + "message": decision.Message, + "type": firstNonEmpty(decision.Type, "rate_limit_exceeded"), + "code": decision.Code, + "param": decision.Param, + }, + }) + return body +} + +func emitPolicyDenyStream(streamID string, decision policyDecision) { + defer func() { + _, _ = callHost(methodHostStreamClose, hostStreamCloseRequest{StreamID: streamID}) + }() + payload := append([]byte("event: error\ndata: "), policyDenyBody(decision)...) + payload = append(payload, []byte("\n\n")...) + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: streamID, + Payload: payload, + }) +} + +func usageHandle(raw []byte) ([]byte, error) { + var rec usageRecord + if err := json.Unmarshal(raw, &rec); err != nil { + return nil, err + } + _ = refreshKeyPolicyState(false) + store := loadedStore() + if store == nil { + return okEnvelope(map[string]any{}) + } + keys, _ := store.ListKeys(context.Background()) + keyByID := map[string]policyplus.KeyRecord{} + for _, key := range keys { + keyByID[key.ID] = key + } + key := keyByID[rec.AuthID] + if key.ID == "" { + key = keyByID[rec.APIKey] + } + if key.ID == "" { + key = keyByID[rec.Source] + } + visibleModel := visibleUsageModel(key, rec) + usage := policyplus.TokenUsage{ + InputTokens: rec.Detail.InputTokens, + OutputTokens: rec.Detail.OutputTokens, + CachedTokens: rec.Detail.CachedTokens, + CacheReadTokens: rec.Detail.CacheReadTokens, + CacheCreationTokens: rec.Detail.CacheCreationTokens, + ReasoningTokens: rec.Detail.ReasoningTokens, + TotalTokens: rec.Detail.TotalTokens, + } + var cost float64 + var breakdown policyplus.CostBreakdown + if key.ID != "" { + if price, ok := policyplus.PriceForModel(key.Prices, visibleModel); ok { + breakdown = policyplus.CostForUsage(price, usage, visibleModel) + cost = breakdown.Costs["total"] + } + } + event := policyplus.UsageEvent{ + RequestID: firstNonEmpty(rec.ResponseHeaders.Get("x-request-id"), rec.ResponseHeaders.Get("x-openai-request-id")), + KeyID: key.ID, + KeyPreview: key.Preview, + Model: visibleModel, + RequestedModel: visibleModel, + ActualModel: rec.Model, + Provider: rec.Provider, + ExecutorType: rec.ExecutorType, + Endpoint: rec.Source, + RequestedAt: rec.RequestedAt, + LatencyMS: rec.Latency.Milliseconds(), + TTFTMS: rec.TTFT.Milliseconds(), + ReasoningEffort: rec.ReasoningEffort, + ServiceTier: rec.ServiceTier, + StatusCode: rec.Failure.StatusCode, + Failed: rec.Failed, + Failure: policyplus.Brief(rec.Failure.Body, 600), + Usage: usage, + Cost: cost, + CostBreakdown: breakdown, + } + _ = store.InsertUsage(context.Background(), event) + return okEnvelope(map[string]any{}) +} + +func visibleUsageModel(key policyplus.KeyRecord, rec usageRecord) string { + if model := strings.TrimSpace(rec.Alias); model != "" { + if alias := visibleExecutorAliasForReportedModel(model); alias != "" { + return alias + } + return model + } + reported := strings.TrimSpace(rec.Model) + if reported == "" || key.ID == "" { + return reported + } + allowed := cleanStrings(key.Models) + for _, model := range allowed { + if strings.EqualFold(model, reported) { + return reported + } + } + if alias := visibleExecutorAliasForReportedModel(reported); alias != "" { + return alias + } + if len(allowed) == 1 { + return allowed[0] + } + priceModels := make([]string, 0, len(key.Prices)) + for name, price := range key.Prices { + model := strings.TrimSpace(price.Model) + if model == "" { + model = strings.TrimSpace(name) + } + if model != "" { + priceModels = append(priceModels, model) + } + } + priceModels = cleanStrings(priceModels) + for _, model := range priceModels { + if strings.EqualFold(model, reported) { + return reported + } + } + if len(priceModels) == 1 { + return priceModels[0] + } + return reported +} + +func visibleExecutorAliasForReportedModel(reported string) string { + reported = strings.TrimSpace(reported) + if reported == "" { + return "" + } + for actual, visible := range executorUsageModelAliases { + if strings.EqualFold(strings.TrimSpace(actual), reported) { + return strings.TrimSpace(visible) + } + } + return "" +} + +func managementRegister() ([]byte, error) { + resp := managementRegistrationResponse{ + Routes: []managementRoute{ + {Method: http.MethodGet, Path: "/plugins/cpa-key-policy-plus/keys"}, + {Method: http.MethodGet, Path: "/plugins/cpa-key-policy-plus/models"}, + {Method: http.MethodPut, Path: "/plugins/cpa-key-policy-plus/keys/save"}, + {Method: http.MethodPut, Path: "/plugins/cpa-key-policy-plus/keys/limits"}, + {Method: http.MethodPost, Path: "/plugins/cpa-key-policy-plus/keys/reset"}, + {Method: http.MethodGet, Path: "/plugins/cpa-key-policy-plus/events"}, + {Method: http.MethodGet, Path: "/plugins/cpa-key-policy-plus/codexcont"}, + {Method: http.MethodPut, Path: "/plugins/cpa-key-policy-plus/codexcont"}, + }, + Resources: []resourceRoute{ + {Path: "/admin", Menu: "CPA Key Policy+", Description: "Unified user key policy dashboard"}, + {Path: "/admin/api/keys"}, + {Path: "/admin/api/models"}, + {Path: "/admin/api/keys/save"}, + {Path: "/admin/api/keys/limits"}, + {Path: "/admin/api/keys/reset"}, + {Path: "/admin/api/events"}, + {Path: "/admin/api/codexcont"}, + {Path: "/user", Description: "Self-service usage dashboard"}, + {Path: "/user/api/session"}, + {Path: "/user/api/me"}, + {Path: "/user/api/usage"}, + {Path: "/user/api/events"}, + {Path: "/user/api/codexcont"}, + }, + } + return okEnvelope(resp) +} + +func managementHandle(raw []byte) ([]byte, error) { + var req managementRequest + if err := json.Unmarshal(raw, &req); err != nil { + return nil, err + } + path := strings.TrimSpace(req.Path) + switch { + case path == "/v0/resource/plugins/cpa-key-policy-plus/admin" || path == "/admin": + return managementHTML(adminHTML()) + case path == "/v0/resource/plugins/cpa-key-policy-plus/user" || path == "/user": + return managementHTML(userHTML()) + case strings.HasSuffix(path, "/admin/api/keys"): + return adminKeys(req) + case strings.HasSuffix(path, "/admin/api/models"): + return adminModels(req) + case strings.HasSuffix(path, "/admin/api/keys/create"): + return adminCreateKey(req) + case strings.HasSuffix(path, "/admin/api/keys/save"): + return adminSaveKeys(req) + case strings.HasSuffix(path, "/admin/api/keys/limits"): + return adminSetLimits(req) + case strings.HasSuffix(path, "/admin/api/keys/reset"): + return adminReset(req) + case strings.HasSuffix(path, "/admin/api/keys/archive"): + return adminArchiveKey(req) + case strings.HasSuffix(path, "/admin/api/keys/delete"): + return adminDeleteKey(req) + case strings.HasSuffix(path, "/admin/api/events"): + return adminEvents(req) + case strings.HasSuffix(path, "/admin/api/codexcont"): + return adminCodexCont(req) + case strings.Contains(path, "/user/api/session"): + return userSession(req) + case strings.Contains(path, "/user/api/me"): + return userMe(req) + case strings.Contains(path, "/user/api/usage"): + return userUsage(req) + case strings.Contains(path, "/user/api/events"): + return userEvents(req) + case strings.Contains(path, "/user/api/codexcont"): + return userCodexCont(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys"): + return adminKeys(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/models"): + return adminModels(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys/create"): + return adminCreateKey(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys/save"): + return adminSaveKeys(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys/limits"): + return adminSetLimits(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys/reset"): + return adminReset(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys/archive"): + return adminArchiveKey(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/keys/delete"): + return adminDeleteKey(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/events"): + return adminEvents(req) + case strings.HasSuffix(path, "/plugins/cpa-key-policy-plus/codexcont"): + return adminCodexCont(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys"): + return adminKeys(req) + case strings.HasSuffix(path, "/key-policy-plus/api/models"): + return adminModels(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys/create"): + return adminCreateKey(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys/save"): + return adminSaveKeys(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys/limits"): + return adminSetLimits(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys/reset"): + return adminReset(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys/archive"): + return adminArchiveKey(req) + case strings.HasSuffix(path, "/key-policy-plus/api/keys/delete"): + return adminDeleteKey(req) + case strings.HasSuffix(path, "/key-policy-plus/api/events"): + return adminEvents(req) + case strings.HasSuffix(path, "/key-policy-plus/api/codexcont"): + return adminCodexCont(req) + default: + return jsonResponse(http.StatusNotFound, map[string]any{"ok": false, "error": "not_found"}) + } +} + +func adminKeys(req managementRequest) ([]byte, error) { + if err := syncNativeKeysFromLoadedConfig(); err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "native_key_sync_failed", "message": policyplus.Brief(err.Error(), 240)}) + } + _ = refreshKeyPolicyState(false) + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + keys, err := store.ListKeys(context.Background()) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + includeRemoved := truthyQuery(req.Query.Get("include_removed")) || truthyQuery(req.Query.Get("show_removed")) + keys = currentAdminKeyRows(keys, includeRemoved) + safe := make([]map[string]any, 0, len(keys)) + now := time.Now() + for _, key := range keys { + row := key.Safe() + usage := usageWindows(context.Background(), store, key.ID, now) + row["usage"] = usage + row["quota"] = quotaWindows(key, usage) + safe = append(safe, row) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "keys": safe, "codexcont": codexcontStatus()}) +} + +func truthyQuery(value string) bool { + switch strings.ToLower(strings.TrimSpace(value)) { + case "1", "true", "yes", "on": + return true + default: + return false + } +} + +func currentAdminKeyRows(keys []policyplus.KeyRecord, includeRemoved bool) []policyplus.KeyRecord { + out := make([]policyplus.KeyRecord, 0, len(keys)) + for _, key := range keys { + if key.Source != policyplus.NativeCPASource { + continue + } + if !includeRemoved && (!key.SourcePresent || key.Hidden) { + continue + } + out = append(out, key) + } + return out +} + +func adminModels(_ managementRequest) ([]byte, error) { + models, warnings := adminModelCatalog() + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "models": models, "warnings": warnings}) +} + +func adminModelCatalog() ([]policyplus.ModelOption, []string) { + warnings := []string{} + hostModels, hostWarnings := hostAuthModelHints() + warnings = append(warnings, hostWarnings...) + configured := configuredModelOptions() + models := policyplus.MergeModelOptions(hostModels, configured) + if len(models) == 0 { + warnings = append(warnings, "当前没有从 CPA 或 Plus 配置中发现模型；可以先创建允许全部模型的 Key，或在编辑模型时手动输入模型名。") + } + sort.SliceStable(models, func(i, j int) bool { + if models[i].Known != models[j].Known { + return models[i].Known + } + return strings.ToLower(models[i].ID) < strings.ToLower(models[j].ID) + }) + return models, warnings +} + +func configuredModelOptions() []policyplus.ModelOption { + _ = syncNativeKeysFromLoadedConfig() + store := loadedStore() + if store == nil { + return nil + } + keys, err := store.ListKeys(context.Background()) + if err != nil { + return nil + } + ids := []string{} + for _, key := range keys { + ids = append(ids, key.Models...) + for model := range key.Prices { + ids = append(ids, model) + } + } + return policyplus.ModelOptionsFromIDs(cleanStrings(ids), "plus_configured", false) +} + +func hostAuthModelHints() ([]policyplus.ModelOption, []string) { + result, err := callHost(methodHostAuthList, map[string]any{}) + if err != nil { + return nil, []string{"宿主 auth 列表不可用，已使用 Plus 当前配置模型兜底。"} + } + var body struct { + Files []map[string]any `json:"files"` + } + if err := json.Unmarshal(result, &body); err != nil { + return nil, []string{"宿主 auth 列表格式无法解析，已使用 Plus 当前配置模型兜底。"} + } + ids := []string{} + for _, file := range body.Files { + for _, field := range []string{"models", "available_models", "model_aliases"} { + ids = append(ids, modelIDsFromAny(file[field])...) + } + } + return policyplus.ModelOptionsFromIDs(cleanStrings(ids), "host_auth", true), nil +} + +func modelIDsFromAny(raw any) []string { + options := policyplus.NormalizeModelOptions(raw, "host_auth") + out := make([]string, 0, len(options)) + for _, option := range options { + out = append(out, option.ID) + } + return out +} + +func adminCreateKey(req managementRequest) ([]byte, error) { + _ = req + return jsonResponse(http.StatusGone, map[string]any{ + "ok": false, + "error": "native_key_lifecycle_owned_by_cpa", + "message": "Key 新增、删除、复制和别名已交给 CPA/CPAMP 管理；Plus 只编辑已同步 Key 的策略。", + }) +} + +func adminSaveKeys(req managementRequest) ([]byte, error) { + var body struct { + Keys []struct { + ID string `json:"id"` + Name string `json:"name"` + Enabled *bool `json:"enabled"` + RPM int `json:"rpm"` + Concurrency int `json:"concurrency"` + MaxActiveSessions int `json:"max_active_sessions"` + Models []string `json:"models"` + Prices map[string]policyplus.ModelPrice `json:"prices"` + FiveHourUSD *float64 `json:"five_hour_usd"` + DailyUSD *float64 `json:"daily_usd"` + WeeklyUSD *float64 `json:"weekly_usd"` + MonthlyUSD *float64 `json:"monthly_usd"` + } `json:"keys"` + } + if err := json.Unmarshal(req.Body, &body); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "invalid_json"}) + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + if err := syncNativeKeysFromLoadedConfig(); err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "native_key_sync_failed", "message": policyplus.Brief(err.Error(), 240)}) + } + existing, err := store.ListKeys(context.Background()) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + byID := map[string]policyplus.KeyRecord{} + for _, key := range existing { + byID[key.ID] = key + } + for _, item := range body.Keys { + key, ok := byID[strings.TrimSpace(item.ID)] + if !ok { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "unknown_key"}) + } + if key.Source == policyplus.NativeCPASource && !key.SourcePresent { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "source_removed_key_read_only"}) + } + if item.Enabled != nil { + key.Enabled = *item.Enabled + } + key.RPM = item.RPM + key.Concurrency = 0 + key.MaxActiveSessions = 0 + key.Models = cleanStrings(item.Models) + if item.Prices != nil { + key.Prices = cleanPrices(item.Prices) + } + key.FiveHourUSD = item.FiveHourUSD + key.DailyLimitUSD = item.DailyUSD + key.WeeklyLimitUSD = item.WeeklyUSD + key.MonthlyLimitUSD = item.MonthlyUSD + if err := store.SaveKeySettings(context.Background(), key); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": err.Error()}) + } + } + return adminKeys(req) +} + +func cleanPrices(items map[string]policyplus.ModelPrice) map[string]policyplus.ModelPrice { + if items == nil { + return nil + } + out := map[string]policyplus.ModelPrice{} + for name, price := range items { + model := strings.TrimSpace(price.Model) + if model == "" { + model = strings.TrimSpace(name) + } + if model == "" { + continue + } + price.Model = model + out[model] = price + } + return out +} + +func cleanStrings(items []string) []string { + seen := map[string]bool{} + out := make([]string, 0, len(items)) + for _, item := range items { + item = strings.TrimSpace(item) + if item == "" || seen[item] { + continue + } + seen[item] = true + out = append(out, item) + } + return out +} + +func adminSetLimits(req managementRequest) ([]byte, error) { + var body struct { + Limits []struct { + ID string `json:"id"` + FiveHourUSD *float64 `json:"five_hour_usd"` + MonthlyUSD *float64 `json:"monthly_usd"` + } `json:"limits"` + } + if err := json.Unmarshal(req.Body, &body); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "invalid_json"}) + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + if err := syncNativeKeysFromLoadedConfig(); err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "native_key_sync_failed", "message": policyplus.Brief(err.Error(), 240)}) + } + for _, item := range body.Limits { + if strings.TrimSpace(item.ID) == "" { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "missing_key_id"}) + } + if err := store.SetLimits(context.Background(), item.ID, item.FiveHourUSD, item.MonthlyUSD); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": err.Error()}) + } + } + return adminKeys(req) +} + +func adminReset(req managementRequest) ([]byte, error) { + var body struct { + ID string `json:"id"` + Window string `json:"window"` + } + if err := json.Unmarshal(req.Body, &body); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": "invalid_json"}) + } + if body.Window == "" { + body.Window = "all" + } + windows := []string{body.Window} + if body.Window == "all" { + windows = []string{policyplus.Range5H, policyplus.Range24H, policyplus.Range7D, policyplus.RangeMonth} + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + if err := syncNativeKeysFromLoadedConfig(); err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "native_key_sync_failed", "message": policyplus.Brief(err.Error(), 240)}) + } + for _, window := range windows { + if err := store.Reset(context.Background(), body.ID, window, time.Now()); err != nil { + return jsonResponse(http.StatusBadRequest, map[string]any{"ok": false, "error": err.Error()}) + } + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true}) +} + +func adminArchiveKey(req managementRequest) ([]byte, error) { + _ = req + return jsonResponse(http.StatusGone, map[string]any{ + "ok": false, + "error": "native_key_lifecycle_owned_by_cpa", + "message": "Key 生命周期已交给 CPA/CPAMP 管理；Plus 只保留策略和历史。", + }) +} + +func adminDeleteKey(req managementRequest) ([]byte, error) { + _ = req + return jsonResponse(http.StatusGone, map[string]any{ + "ok": false, + "error": "native_key_lifecycle_owned_by_cpa", + "message": "Key 删除请在 CPA/CPAMP 中完成；Plus 会在同步后自动标记官方已移除并保留历史。", + }) +} + +func adminEvents(req managementRequest) ([]byte, error) { + keyID := req.Query.Get("key_id") + if keyID == "" { + keyID = "all" + } + return eventsResponseFromRequest(req, keyID) +} + +func adminCodexCont(req managementRequest) ([]byte, error) { + if req.Method == http.MethodPut || (req.Method == http.MethodGet && req.Query.Get("action") == "save") { + var body struct { + Enabled *bool `json:"enabled"` + URL string `json:"url"` + FailMode string `json:"fail_mode"` + } + if req.Method == http.MethodGet { + if rawEnabled := strings.TrimSpace(req.Query.Get("enabled")); rawEnabled != "" { + if parsed, err := strconv.ParseBool(rawEnabled); err == nil { + body.Enabled = &parsed + } + } + body.URL = req.Query.Get("url") + body.FailMode = req.Query.Get("fail_mode") + } else { + _ = json.Unmarshal(req.Body, &body) + } + store := loadedStore() + state.mu.Lock() + if body.Enabled != nil { + state.cfg.CodexContEnabled = *body.Enabled + } + if strings.TrimSpace(body.URL) != "" { + state.cfg.CodexContURL = strings.TrimRight(strings.TrimSpace(body.URL), "/") + } + if strings.TrimSpace(body.FailMode) != "" { + state.cfg.FailMode = strings.ToLower(strings.TrimSpace(body.FailMode)) + } + state.cfg = state.cfg.Normalize() + cfg := state.cfg + state.mu.Unlock() + if store != nil { + settings := map[string]string{ + "codexcont_enabled": strconv.FormatBool(cfg.CodexContEnabled), + "codexcont_url": cfg.CodexContURL, + "fail_mode": cfg.FailMode, + } + if err := store.SaveSettings(context.Background(), settings); err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "save_settings_failed"}) + } + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "codexcont": codexcontStatus()}) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "codexcont": codexcontStatus()}) +} + +func userSession(req managementRequest) ([]byte, error) { + key := userSubmittedKey(req) + if key == "" { + hint := policyplus.ExplainUnmatchedSubmittedKey(key) + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": hint.Error, "message": hint.Message}) + } + record, decision, ok := policyDecisionForRawKey(key, "", false) + if !ok { + hint := policyplus.ExplainUnmatchedSubmittedKey(key) + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": hint.Error, "message": hint.Message}) + } + if record.ID != "" && decision.Code != "policy_missing" { + decision = evaluateIdentityPolicy(record) + } + if !decision.Allowed { + return jsonResponse(http.StatusForbidden, map[string]any{"ok": false, "error": decision.Code, "category": "auth", "message": decision.Message}) + } + cfg := loadedConfig() + token, err := policyplus.SignSession(policyplus.SessionPayload{ + KeyID: record.ID, + KeyHash: record.KeyHash, + ExpiresAt: time.Now().Add(policyplus.SessionTTL()).Unix(), + }, cfg.SessionSecret) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": "session_error"}) + } + body, err := json.Marshal(map[string]any{"ok": true, "me": record.Safe()}) + if err != nil { + return nil, err + } + resp := managementResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{ + "Content-Type": []string{"application/json; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + "Set-Cookie": userSessionSetCookies(token), + }, + Body: body, + } + return okEnvelope(resp) +} + +func userSessionSetCookies(token string) []string { + paths := []string{ + "/", + "/v0/resource/plugins/cpa-key-policy-plus/user", + "/key-policy-plus-user", + } + out := make([]string, 0, len(paths)) + for _, path := range paths { + out = append(out, (&http.Cookie{ + Name: plusSessionCookieName, + Value: token, + Path: path, + HttpOnly: true, + SameSite: http.SameSiteLaxMode, + MaxAge: int(policyplus.SessionTTL().Seconds()), + }).String()) + } + return out +} + +func userSubmittedKey(req managementRequest) string { + raw := firstNonEmpty( + headerFirst(req.Headers, "X-CPA-Key-Policy-Plus-Key"), + headerFirst(req.Headers, "X-CPA-Governor-Key"), + headerFirst(req.Headers, "X-CPA-User-Key"), + ) + if raw == "" { + raw = bearer(headerFirst(req.Headers, "Authorization")) + } + return policyplus.NormalizeSubmittedKey(raw) +} + +func userMe(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated", "category": "auth", "message": "会话已过期，请重新登录。"}) + } + row := key.Safe() + if store := loadedStore(); store != nil { + usage := usageWindows(context.Background(), store, key.ID, time.Now()) + row["usage"] = usage + row["quota"] = quotaWindows(key, usage) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "me": row}) +} + +func userUsage(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated", "category": "auth", "message": "会话已过期，请重新登录。"}) + } + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + rangeName := req.Query.Get("range") + if rangeName == "" { + rangeName = policyplus.Range24H + } + summary, err := store.UsageSummary(context.Background(), key.ID, policyplus.WindowFor(rangeName, time.Now())) + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + success := summary.Calls - summary.Failed + successRate := 0.0 + if summary.Calls > 0 { + successRate = float64(success) / float64(summary.Calls) + } + return jsonResponse(http.StatusOK, map[string]any{ + "ok": true, + "range": rangeName, + "limits": key.Safe()["limits"], + "summary": map[string]any{ + "calls": summary.Calls, + "success": success, + "failed": summary.Failed, + "success_rate": successRate, + "total_cost": summary.TotalCost, + "usage": summary.Usage, + }, + }) +} + +func userEvents(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated", "category": "auth", "message": "会话已过期，请重新登录。"}) + } + return eventsResponseFromRequest(req, key.ID) +} + +func userCodexCont(req managementRequest) ([]byte, error) { + key, ok := keyFromSession(req) + if !ok { + return jsonResponse(http.StatusUnauthorized, map[string]any{"ok": false, "error": "not_authenticated", "category": "auth", "message": "会话已过期，请重新登录。"}) + } + limit := 80 + if rawLimit := strings.TrimSpace(req.Query.Get("limit")); rawLimit != "" { + if parsed, err := strconv.Atoi(rawLimit); err == nil { + limit = parsed + } + } + if limit <= 0 || limit > 200 { + limit = 80 + } + requests, source := codexRequestsForKey(key, limit) + sortCodexSummariesNewestFirst(requests) + return jsonResponse(http.StatusOK, map[string]any{ + "ok": true, + "codexcont": codexcontStatus(), + "requests": requests, + "source": source, + }) +} + +func eventsResponse(keyID string, limit int) ([]byte, error) { + return eventsResponseWithRange(keyID, "", limit) +} + +func eventsResponseFromRequest(req managementRequest, keyID string) ([]byte, error) { + limit := 100 + if rawLimit := strings.TrimSpace(req.Query.Get("limit")); rawLimit != "" { + if parsed, err := strconv.Atoi(rawLimit); err == nil { + limit = parsed + } + } + return eventsResponseWithRange(keyID, req.Query.Get("range"), limit) +} + +func eventsResponseWithRange(keyID string, rangeName string, limit int) ([]byte, error) { + store := loadedStore() + if store == nil { + return jsonResponse(http.StatusServiceUnavailable, map[string]any{"ok": false, "error": "store_unavailable"}) + } + var events []policyplus.UsageEvent + var err error + if strings.TrimSpace(rangeName) == "" { + events, err = store.RecentEvents(context.Background(), keyID, limit) + } else { + events, err = store.RecentEventsWindow(context.Background(), keyID, policyplus.WindowFor(rangeName, time.Now()), limit) + } + if err != nil { + return jsonResponse(http.StatusInternalServerError, map[string]any{"ok": false, "error": err.Error()}) + } + return jsonResponse(http.StatusOK, map[string]any{"ok": true, "events": events}) +} + +func usageWindows(ctx context.Context, store *policyplus.Store, keyID string, now time.Time) map[string]float64 { + out := map[string]float64{} + for _, name := range []string{policyplus.Range5H, policyplus.Range24H, policyplus.Range7D, policyplus.RangeMonth} { + value, _ := store.UsageSum(ctx, keyID, policyplus.WindowFor(name, now)) + out[name] = value + } + return out +} + +func quotaWindows(key policyplus.KeyRecord, usage map[string]float64) map[string]map[string]any { + limits := map[string]*float64{ + policyplus.Range5H: key.FiveHourUSD, + policyplus.Range24H: key.DailyLimitUSD, + policyplus.Range7D: key.WeeklyLimitUSD, + policyplus.RangeMonth: key.MonthlyLimitUSD, + } + out := map[string]map[string]any{} + for _, name := range []string{policyplus.Range5H, policyplus.Range24H, policyplus.Range7D, policyplus.RangeMonth} { + used := usage[name] + row := map[string]any{ + "used_usd": used, + "limit_usd": nil, + "remaining_usd": nil, + "percent": nil, + } + if limit := limits[name]; limit != nil { + remaining := *limit - used + if remaining < 0 { + remaining = 0 + } + percent := 0.0 + if *limit > 0 { + percent = used / *limit + } + row["limit_usd"] = *limit + row["remaining_usd"] = remaining + row["percent"] = percent + } + out[name] = row + } + return out +} + +func codexRequestsForKey(key policyplus.KeyRecord, limit int) ([]map[string]any, string) { + if requests, ok := fetchExecutorCodexSummaries(key, limit); ok { + return requests, "codexcont_executor_store" + } + if requests, ok := fetchCodexContRequests(key, limit); ok { + return requests, "codexcont_admin" + } + store := loadedStore() + if store == nil { + return []map[string]any{}, "unavailable" + } + items, err := store.RecentCodexSummaries(context.Background(), key.ID, limit) + if err != nil { + return []map[string]any{}, "store_error" + } + out := make([]map[string]any, 0, len(items)) + for _, item := range items { + safe := safeCodexSummary(item.Summary, key) + if safe == nil { + continue + } + out = append(out, safe) + } + sortCodexSummariesNewestFirst(out) + return out, "governor_store" +} + +func fetchExecutorCodexSummaries(key policyplus.KeyRecord, limit int) ([]map[string]any, bool) { + cfg := loadedConfig() + path := strings.TrimSpace(cfg.CodexSummaryDBPath) + if path == "" { + return nil, false + } + items, err := policyplus.RecentCodexSummariesFromSQLite(context.Background(), path, key.ID, limit) + if err != nil { + return nil, false + } + out := make([]map[string]any, 0, len(items)) + for _, item := range items { + safe := safeCodexSummary(item.Summary, key) + if safe == nil { + continue + } + out = append(out, safe) + } + sortCodexSummariesNewestFirst(out) + return out, true +} + +func fetchCodexContRequests(key policyplus.KeyRecord, limit int) ([]map[string]any, bool) { + cfg := loadedConfig() + if !cfg.CodexContEnabled { + return nil, false + } + base := strings.TrimRight(strings.TrimSpace(cfg.CodexContURL), "/") + if base == "" { + return nil, false + } + parsed, err := url.Parse(base + "/admin/requests?limit=" + strconv.Itoa(limit)) + if err != nil || parsed.Scheme == "" || parsed.Host == "" { + return nil, false + } + client := http.Client{Timeout: 1200 * time.Millisecond} + resp, err := client.Get(parsed.String()) + if err != nil { + return nil, false + } + defer resp.Body.Close() + if resp.StatusCode < 200 || resp.StatusCode >= 300 { + return nil, false + } + var body struct { + Requests []map[string]any `json:"requests"` + } + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { + return nil, false + } + out := make([]map[string]any, 0, len(body.Requests)) + for _, req := range body.Requests { + safe := safeCodexSummary(req, key) + if safe == nil { + continue + } + out = append(out, safe) + } + sortCodexSummariesNewestFirst(out) + return out, true +} + +func sortCodexSummariesNewestFirst(items []map[string]any) { + sort.SliceStable(items, func(i, j int) bool { + left, leftOK := codexSummaryDisplayTime(items[i]) + right, rightOK := codexSummaryDisplayTime(items[j]) + if leftOK != rightOK { + return leftOK + } + if !leftOK { + return false + } + return left.After(right) + }) +} + +func codexSummaryDisplayTime(req map[string]any) (time.Time, bool) { + for _, field := range []string{"started_at", "updated_at", "ended_at"} { + if parsed, ok := parseCodexSummaryTime(req[field]); ok { + return parsed, true + } + } + return time.Time{}, false +} + +func parseCodexSummaryTime(value any) (time.Time, bool) { + switch v := value.(type) { + case time.Time: + if v.IsZero() { + return time.Time{}, false + } + return v, true + case string: + raw := strings.TrimSpace(v) + if raw == "" { + return time.Time{}, false + } + for _, layout := range []string{time.RFC3339Nano, time.RFC3339, "2006-01-02 15:04:05"} { + if parsed, err := time.Parse(layout, raw); err == nil { + return parsed, true + } + } + return time.Time{}, false + case json.Number: + if asInt, err := v.Int64(); err == nil { + return unixLikeTime(asInt) + } + if asFloat, err := v.Float64(); err == nil { + return unixLikeTime(int64(asFloat)) + } + case float64: + return unixLikeTime(int64(v)) + case int64: + return unixLikeTime(v) + case int: + return unixLikeTime(int64(v)) + } + return time.Time{}, false +} + +func unixLikeTime(raw int64) (time.Time, bool) { + if raw <= 0 { + return time.Time{}, false + } + if raw > 1_000_000_000_000 { + return time.UnixMilli(raw), true + } + return time.Unix(raw, 0), true +} + +func safeCodexSummary(req map[string]any, key policyplus.KeyRecord) map[string]any { + if req == nil { + return nil + } + identity, _ := req["key_identity"].(map[string]any) + if !codexIdentityMatches(identity, key) { + return nil + } + fields := []string{ + "request_id", "model", "path", "started_at", "updated_at", "ended_at", + "duration_ms", "status", "protection", "latest_round", + "latest_reasoning_tokens", "first_truncation_round", + "first_truncation_reasoning_tokens", "first_truncation_decision", + "continuation_count", "stopped_reason", "failure_reason", + "passthrough_reason", "rounds", + } + out := map[string]any{} + for _, field := range fields { + if value, ok := req[field]; ok { + out[field] = value + } + } + out["key_identity"] = key.Safe() + return out +} + +func codexIdentityMatches(identity map[string]any, key policyplus.KeyRecord) bool { + if strings.TrimSpace(key.ID) == "" || identity == nil { + return false + } + id := strings.TrimSpace(fmt.Sprint(identity["id"])) + if id != "" && id == key.ID { + return true + } + preview := strings.TrimSpace(fmt.Sprint(identity["preview"])) + return preview != "" && preview == key.Preview +} + +func forwardHostStream(targetStreamID string, sourceStreamID string) { + defer func() { + _, _ = callHost(methodHostModelStreamClose, hostModelStreamCloseRequest{StreamID: sourceStreamID}) + _, _ = callHost(methodHostStreamClose, hostStreamCloseRequest{StreamID: targetStreamID}) + }() + for { + result, err := callHost(methodHostModelStreamRead, hostModelStreamReadRequest{StreamID: sourceStreamID}) + if err != nil { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Error: policyplus.Brief(err.Error(), 400), + }) + return + } + var chunk hostModelStreamReadResponse + if err := json.Unmarshal(result, &chunk); err != nil { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Error: "decode host stream chunk: " + policyplus.Brief(err.Error(), 300), + }) + return + } + if len(chunk.Payload) > 0 { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Payload: chunk.Payload, + }) + } + if chunk.Error != "" { + _, _ = callHost(methodHostStreamEmit, hostStreamEmitRequest{ + StreamID: targetStreamID, + Error: policyplus.Brief(chunk.Error, 400), + }) + return + } + if chunk.Done { + return + } + } +} + +func keyFromSession(req managementRequest) (policyplus.KeyRecord, bool) { + _ = syncNativeKeysFromLoadedConfig() + _ = refreshKeyPolicyState(false) + tokens := sessionTokensFromCookie(headerFirst(req.Headers, "Cookie")) + cfg := loadedConfig() + for _, token := range tokens { + if key, ok := keyFromSessionToken(token, cfg.SessionSecret); ok { + return key, true + } + } + return policyplus.KeyRecord{}, false +} + +func sessionTokensFromCookie(cookie string) []string { + var tokens []string + for _, part := range strings.Split(cookie, ";") { + part = strings.TrimSpace(part) + if strings.HasPrefix(part, plusSessionCookieName+"=") { + tokens = append(tokens, strings.TrimPrefix(part, plusSessionCookieName+"=")) + continue + } + if strings.HasPrefix(part, "cpa_governor_session=") { + tokens = append(tokens, strings.TrimPrefix(part, "cpa_governor_session=")) + } + } + return tokens +} + +func keyFromSessionToken(token string, secret string) (policyplus.KeyRecord, bool) { + payload, ok := policyplus.VerifySession(token, secret, time.Now()) + if !ok { + return policyplus.KeyRecord{}, false + } + store := loadedStore() + if store == nil { + return policyplus.KeyRecord{}, false + } + keys, err := store.ListKeys(context.Background()) + if err != nil { + return policyplus.KeyRecord{}, false + } + for _, key := range keys { + currentHash, errCurrent := policyplus.NormalizeHash(key.KeyHash) + sessionHash, errSession := policyplus.NormalizeHash(payload.KeyHash) + if key.ID == payload.KeyID && key.Enabled && !key.Archived && errCurrent == nil && errSession == nil && currentHash == sessionHash { + return key, true + } + } + return policyplus.KeyRecord{}, false +} + +func codexcontStatus() map[string]any { + cfg := loadedConfig() + status := map[string]any{ + "enabled": cfg.CodexContEnabled, + "route": cfg.CodexContRoute, + "url": cfg.CodexContURL, + "fail_mode": cfg.FailMode, + "mode": "passive_until_executor_cutover", + } + if cfg.CodexContEnabled { + health := probeCodexContHealth(cfg.CodexContURL) + for key, value := range health { + status[key] = value + } + } + return status +} + +func probeCodexContHealth(baseURL string) map[string]any { + out := map[string]any{ + "health_ok": false, + } + parsed, err := url.Parse(strings.TrimRight(baseURL, "/") + "/engine/healthz") + if err != nil || parsed.Scheme == "" || parsed.Host == "" { + out["health_error"] = "invalid_engine_url" + return out + } + client := http.Client{Timeout: 800 * time.Millisecond} + resp, err := client.Get(parsed.String()) + if err != nil { + out["health_error"] = policyplus.Brief(err.Error(), 200) + return out + } + defer resp.Body.Close() + out["health_status"] = resp.StatusCode + if resp.StatusCode >= 200 && resp.StatusCode < 300 { + out["health_ok"] = true + } + return out +} + +func bearer(value string) string { + value = strings.TrimSpace(value) + if strings.HasPrefix(strings.ToLower(value), "bearer ") { + return strings.TrimSpace(value[7:]) + } + return "" +} + +func headerFirst(headers http.Header, name string) string { + if headers == nil { + return "" + } + if value := strings.TrimSpace(headers.Get(name)); value != "" { + return value + } + for key, values := range headers { + if !strings.EqualFold(key, name) { + continue + } + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + } + return "" +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return strings.TrimSpace(value) + } + } + return "" +} + +func cloneHeader(headers http.Header) http.Header { + if headers == nil { + return nil + } + cloned := make(http.Header, len(headers)) + for key, values := range headers { + cloned[key] = append([]string(nil), values...) + } + return cloned +} + +func cloneValues(values map[string][]string) map[string][]string { + if values == nil { + return nil + } + cloned := make(map[string][]string, len(values)) + for key, items := range values { + cloned[key] = append([]string(nil), items...) + } + return cloned +} + +func okEnvelope(v any) ([]byte, error) { + raw, err := json.Marshal(v) + if err != nil { + return nil, err + } + return json.Marshal(envelope{OK: true, Result: raw}) +} + +func errorEnvelope(code, message string) []byte { + raw, _ := json.Marshal(envelope{OK: false, Error: &envelopeError{Code: code, Message: message}}) + return raw +} + +func jsonResponse(status int, v any) ([]byte, error) { + body, err := json.Marshal(v) + if err != nil { + return nil, err + } + return okEnvelope(managementResponse{ + StatusCode: status, + Headers: http.Header{ + "Content-Type": []string{"application/json; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + }, + Body: body, + }) +} + +func managementHTML(html string) ([]byte, error) { + return okEnvelope(managementResponse{ + StatusCode: http.StatusOK, + Headers: http.Header{ + "Content-Type": []string{"text/html; charset=utf-8"}, + "Cache-Control": []string{"no-store"}, + }, + Body: []byte(html), + }) +} + +var hostCall = func(method string, payload any) (json.RawMessage, error) { + _ = payload + return nil, fmt.Errorf("host callback %s is unavailable", method) +} + +func callHost(method string, payload any) (json.RawMessage, error) { + return hostCall(method, payload) +} + +func adminHTML() string { + return renderHTML(adminHTMLTemplate) +} + +func userHTML() string { + return renderHTML(userHTMLTemplate) +} + +func sharedCSS() string { + return sharedCSSTemplate +} + +func renderHTML(tpl string) string { + tpl = strings.ReplaceAll(tpl, "{{SHARED_CSS}}", sharedCSS()) + return strings.ReplaceAll(tpl, "{{CSS}}", sharedCSS()) +} diff --git a/cpa_key_policy_plus_plugin/go/main_test.go b/cpa_key_policy_plus_plugin/go/main_test.go new file mode 100644 index 0000000..476d48a --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/main_test.go @@ -0,0 +1,1116 @@ +package main + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "net/http" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "codexcont/cpa-key-policy-plus-plugin/internal/policyplus" + _ "modernc.org/sqlite" +) + +func setupTestState(t *testing.T) policyplus.KeyRecord { + t.Helper() + state.mu.Lock() + state.cfg = policyplus.DefaultConfig() + state.cfg.StateDBPath = filepath.Join(t.TempDir(), "policyplus.sqlite") + state.cfg.SessionSecret = "test-secret" + state.cfg.ExclusiveAuth = true + state.keyState = policyplus.KeyPolicyState{} + state.keyStatePath = "" + state.rpmBuckets = map[string][]time.Time{} + old := state.store + state.store = nil + state.mu.Unlock() + if old != nil { + _ = old.Close() + } + store, err := policyplus.OpenStore(state.cfg.StateDBPath) + if err != nil { + t.Fatal(err) + } + key := policyplus.KeyRecord{ + ID: "alice-key", + Name: "Alice", + KeyHash: "sha256:" + policyplus.SHA256Hex("sk-alice-secret"), + Enabled: true, + Preview: policyplus.HashPreview(policyplus.SHA256Hex("sk-alice-secret")), + Source: policyplus.NativeCPASource, + SourcePresent: true, + RPM: 10, + Concurrency: 2, + MaxActiveSessions: 1, + Models: []string{"gpt-5.5"}, + DailyLimitUSD: floatPtr(10), + WeeklyLimitUSD: floatPtr(50), + } + if err := store.UpsertKey(context.Background(), key); err != nil { + t.Fatal(err) + } + state.mu.Lock() + state.store = store + state.mu.Unlock() + t.Cleanup(func() { + state.mu.Lock() + if state.store != nil { + _ = state.store.Close() + } + state.store = nil + state.mu.Unlock() + }) + return key +} + +func TestPluginRegistrationIsPolicyPlusExclusiveAuth(t *testing.T) { + setupTestState(t) + raw, err := okEnvelope(pluginRegistration()) + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var reg registration + if err := json.Unmarshal(env.Result, ®); err != nil { + t.Fatal(err) + } + if reg.Metadata.Name != pluginID { + t.Fatalf("plugin name = %s", reg.Metadata.Name) + } + if !reg.Capabilities.FrontendAuthProvider || !reg.Capabilities.FrontendAuthProviderExclusive { + t.Fatalf("frontend auth capabilities = %#v", reg.Capabilities) + } + if !reg.Capabilities.ModelRouter || !reg.Capabilities.Executor { + t.Fatalf("plus must expose deny-only model route/executor for explicit policy 429s: %#v", reg.Capabilities) + } + if reg.Capabilities.ExecutorModelScope != executorModelScopeBoth { + t.Fatalf("executor model scope = %q", reg.Capabilities.ExecutorModelScope) + } + if len(reg.Capabilities.ExecutorInputFormats) != 1 || reg.Capabilities.ExecutorInputFormats[0] != executorFormatOpenAIResponse { + t.Fatalf("executor input formats = %#v", reg.Capabilities.ExecutorInputFormats) + } + if len(reg.Capabilities.ExecutorOutputFormats) != 1 || reg.Capabilities.ExecutorOutputFormats[0] != executorFormatOpenAIResponse { + t.Fatalf("executor output formats = %#v", reg.Capabilities.ExecutorOutputFormats) + } +} + +func TestFrontendAuthIgnoresRetiredActiveSessionLimit(t *testing.T) { + setupTestState(t) + req := frontendAuthRequest{ + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + Body: []byte(`{"model":"gpt-5.5","prompt_cache_key":"window-a"}`), + } + if !authOK(t, req) { + t.Fatal("first session should authenticate") + } + req.Body = []byte(`{"model":"gpt-5.5","prompt_cache_key":"window-a"}`) + if !authOK(t, req) { + t.Fatal("same session should refresh and authenticate") + } + req.Body = []byte(`{"model":"gpt-5.5","prompt_cache_key":"window-b"}`) + if !authOK(t, req) { + t.Fatal("retired Codex window limit should not reject a second session") + } +} + +func TestFrontendAuthAllowsMissingSessionAndAudits(t *testing.T) { + setupTestState(t) + req := frontendAuthRequest{ + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + Body: []byte(`{"model":"gpt-5.5"}`), + } + if !authOK(t, req) { + t.Fatal("missing session identity should not be rejected in v1") + } +} + +func TestAdminSaveKeysPersistsUnifiedLimits(t *testing.T) { + setupTestState(t) + enabled := false + body := map[string]any{"keys": []map[string]any{{ + "id": "alice-key", + "name": "Alice Plus", + "enabled": enabled, + "rpm": 5, + "concurrency": 1, + "max_active_sessions": 3, + "models": []string{"gpt-5.4", "custom-unknown"}, + "prices": map[string]any{ + "custom-unknown": map[string]any{"input_per_million": 1.2}, + }, + "five_hour_usd": 1.25, + "daily_usd": 2.5, + "weekly_usd": 7.5, + "monthly_usd": 20.0, + }}} + rawBody, _ := json.Marshal(body) + raw, err := adminSaveKeys(managementRequest{Body: rawBody}) + if err != nil { + t.Fatal(err) + } + bodyBytes := decodeManagementBody(t, raw) + if strings.Contains(string(bodyBytes), "Alice Plus") || !strings.Contains(string(bodyBytes), "Alice") { + t.Fatalf("save response should keep CPA/CPAMP readonly alias: %s", bodyBytes) + } + store := loadedStore() + keys, err := store.ListKeys(context.Background()) + if err != nil || len(keys) != 1 { + t.Fatalf("ListKeys err=%v keys=%#v", err, keys) + } + got := keys[0] + if got.Enabled || got.RPM != 5 || got.Concurrency != 0 || got.MaxActiveSessions != 0 { + t.Fatalf("updated key = %#v", got) + } + if got.Name != "Alice" { + t.Fatalf("Plus save must not rename native key alias, got %q", got.Name) + } + if got.FiveHourUSD == nil || *got.FiveHourUSD != 1.25 || got.MonthlyLimitUSD == nil || *got.MonthlyLimitUSD != 20 { + t.Fatalf("limits not persisted: %#v", got) + } + if len(got.Models) != 2 || got.Models[1] != "custom-unknown" { + t.Fatalf("models should preserve unknown selected model: %#v", got.Models) + } + if price, ok := got.Prices["custom-unknown"]; !ok || price.Model != "custom-unknown" || price.InputPerMillion != 1.2 { + t.Fatalf("custom price not normalized: %#v", got.Prices) + } +} + +func TestAdminCreateKeyRetiredToCPAMP(t *testing.T) { + setupTestState(t) + raw, err := adminCreateKey(managementRequest{Body: []byte(`{"name":"Bob","enabled":false,"rpm":9,"concurrency":4,"max_active_sessions":3,"models":["gpt-5.4-mini"]}`)}) + if err != nil { + t.Fatal(err) + } + resp := decodeManagementResponse(t, raw) + bodyBytes := resp.Body + keys, err := loadedStore().ListKeys(context.Background()) + if err != nil { + t.Fatal(err) + } + if resp.StatusCode != http.StatusGone || !strings.Contains(string(bodyBytes), "native_key_lifecycle_owned_by_cpa") { + t.Fatalf("create should be retired to CPA/CPAMP: status=%d body=%s", resp.StatusCode, bodyBytes) + } + if strings.Contains(string(bodyBytes), "raw_key") || len(keys) != 1 { + t.Fatalf("retired create must not return raw keys or persist Bob: keys=%#v body=%s", keys, bodyBytes) + } +} + +func TestManagementAliasCreateSaveAndReset(t *testing.T) { + setupTestState(t) + createRaw, _ := json.Marshal(map[string]any{"name": "Alias", "models": []string{"gpt-5.4"}}) + raw, err := managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodPost, + Path: "/key-policy-plus/api/keys/create", + Body: createRaw, + })) + if err != nil { + t.Fatal(err) + } + createResp := decodeManagementResponse(t, raw) + createBody := createResp.Body + if createResp.StatusCode != http.StatusGone || !strings.Contains(string(createBody), "native_key_lifecycle_owned_by_cpa") || strings.Contains(string(createBody), `"raw_key"`) { + t.Fatalf("create alias should be retired: status=%d body=%s", createResp.StatusCode, createBody) + } + saveRaw, _ := json.Marshal(map[string]any{"keys": []map[string]any{{ + "id": "alice-key", + "name": "Alias Saved", + "enabled": true, + "rpm": 11, + "concurrency": 2, + "max_active_sessions": 1, + "models": []string{"gpt-5.5"}, + }}}) + raw, err = managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodPut, + Path: "/key-policy-plus/api/keys/save", + Body: saveRaw, + })) + if err != nil { + t.Fatal(err) + } + saveBody := decodeManagementBody(t, raw) + if strings.Contains(string(saveBody), "Alias Saved") || !strings.Contains(string(saveBody), "Alice") { + t.Fatalf("save via alias should update strategy but not readonly alias: %s", saveBody) + } + keys, err := loadedStore().ListKeys(context.Background()) + if err != nil { + t.Fatal(err) + } + for _, key := range keys { + if key.ID == "alice-key" && (key.Concurrency != 0 || key.MaxActiveSessions != 0) { + t.Fatalf("alias save should force retired limits to zero: %#v", key) + } + } + resetRaw, _ := json.Marshal(map[string]any{"id": "alice-key", "window": "5h"}) + raw, err = managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodPost, + Path: "/key-policy-plus/api/keys/reset", + Body: resetRaw, + })) + if err != nil { + t.Fatal(err) + } + resetBody := decodeManagementBody(t, raw) + if !strings.Contains(string(resetBody), `"ok":true`) { + t.Fatalf("reset via alias failed: %s", resetBody) + } +} + +func TestArchiveRouteReturnsGone(t *testing.T) { + key := setupTestState(t) + archiveRaw, _ := json.Marshal(map[string]any{"id": key.ID, "archived": true}) + raw, err := managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodPost, + Path: "/key-policy-plus/api/keys/archive", + Body: archiveRaw, + })) + if err != nil { + t.Fatal(err) + } + body := decodeManagementBody(t, raw) + if !strings.Contains(string(body), `"native_key_lifecycle_owned_by_cpa"`) { + t.Fatalf("archive route should return CPA/CPAMP lifecycle guidance: %s", body) + } +} + +func TestDeleteKeyRetiredAndDoesNotDisableNativePolicy(t *testing.T) { + key := setupTestState(t) + deleteRaw, _ := json.Marshal(map[string]any{"id": key.ID, "confirm": "delete"}) + raw, err := managementHandle(mustJSON(t, managementRequest{ + Method: http.MethodPost, + Path: "/key-policy-plus/api/keys/delete", + Body: deleteRaw, + })) + if err != nil { + t.Fatal(err) + } + resp := decodeManagementResponse(t, raw) + body := resp.Body + if resp.StatusCode != http.StatusGone || !strings.Contains(string(body), `"native_key_lifecycle_owned_by_cpa"`) { + t.Fatalf("delete should be retired to CPA/CPAMP: status=%d body=%s", resp.StatusCode, body) + } + if !authOK(t, frontendAuthRequest{ + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + Body: []byte(`{"model":"gpt-5.5","prompt_cache_key":"window-a"}`), + }) { + t.Fatal("retired Plus delete should not disable native policy") + } + raw, err = userSession(managementRequest{Headers: http.Header{"X-CPA-Key-Policy-Plus-Key": []string{"sk-alice-secret"}}}) + if err != nil { + t.Fatal(err) + } + body = decodeManagementBody(t, raw) + if !strings.Contains(string(body), `"ok":true`) { + t.Fatalf("retired Plus delete should not break user session: %s", body) + } +} + +func TestAdminKeysMirrorsCurrentNativeCPAKeys(t *testing.T) { + setupTestState(t) + dir := t.TempDir() + configPath := filepath.Join(dir, "config.yaml") + aliasPath := filepath.Join(dir, "cpamp.sqlite") + writeNativeConfig := func(keys ...string) { + t.Helper() + body := "api-keys:\n" + for _, key := range keys { + body += " - " + key + "\n" + } + if err := os.WriteFile(configPath, []byte(body), 0o600); err != nil { + t.Fatal(err) + } + } + db, err := sql.Open("sqlite", aliasPath) + if err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`create table api_key_aliases(api_key_hash text primary key, alias text, updated_at_ms integer)`); err != nil { + t.Fatal(err) + } + upsertAlias := func(rawKey, alias string) { + t.Helper() + if _, err := db.Exec(`insert into api_key_aliases(api_key_hash, alias, updated_at_ms) values(?, ?, ?) + on conflict(api_key_hash) do update set alias=excluded.alias, updated_at_ms=excluded.updated_at_ms`, + "sha256:"+policyplus.SHA256Hex(rawKey), alias, time.Now().UnixMilli()); err != nil { + t.Fatal(err) + } + } + upsertAlias("sk-native-qq", "QQ专用") + upsertAlias("sk-native-wei", "阿伟专用") + if err := db.Close(); err != nil { + t.Fatal(err) + } + writeNativeConfig("sk-native-qq", "sk-native-wei") + + legacy := policyplus.KeyRecord{ + ID: "cpa_legacy_policy", + Name: "旧 CPI 下划线 Key", + KeyHash: "sha256:" + policyplus.SHA256Hex("cpa_legacy_policy"), + Enabled: true, + Preview: policyplus.HashPreview(policyplus.SHA256Hex("cpa_legacy_policy")), + Source: policyplus.LegacyPlusSource, + SourcePresent: false, + Hidden: true, + } + if err := loadedStore().UpsertKey(context.Background(), legacy); err != nil { + t.Fatal(err) + } + state.mu.Lock() + state.cfg.NativeKeysConfigPath = configPath + state.cfg.CPAMPAliasDBPath = aliasPath + state.mu.Unlock() + + raw, err := adminKeys(managementRequest{}) + if err != nil { + t.Fatal(err) + } + var first struct { + OK bool `json:"ok"` + Keys []map[string]any `json:"keys"` + Meta map[string]json.RawMessage `json:"codexcont"` + } + if err := json.Unmarshal(decodeManagementBody(t, raw), &first); err != nil { + t.Fatal(err) + } + if len(first.Keys) != 2 { + t.Fatalf("default admin list should mirror only current native keys: %#v", first.Keys) + } + names := map[string]bool{} + for _, key := range first.Keys { + names[fmt.Sprint(key["name"])] = true + if key["source"] != policyplus.NativeCPASource || key["source_present"] != true || key["hidden"] == true { + t.Fatalf("default row should be active native key only: %#v", key) + } + } + if !names["QQ专用"] || !names["阿伟专用"] || names["旧 CPI 下划线 Key"] { + t.Fatalf("unexpected admin key names: %#v rows=%#v", names, first.Keys) + } + for _, key := range first.Keys { + if key["enabled"] != true { + t.Fatalf("new official native keys should be enabled by default: %#v", key) + } + } + + db, err = sql.Open("sqlite", aliasPath) + if err != nil { + t.Fatal(err) + } + if _, err := db.Exec(`update api_key_aliases set alias=? where api_key_hash=?`, "QQ官Key改名", "sha256:"+policyplus.SHA256Hex("sk-native-qq")); err != nil { + t.Fatal(err) + } + if err := db.Close(); err != nil { + t.Fatal(err) + } + writeNativeConfig("sk-native-qq") + raw, err = adminKeys(managementRequest{}) + if err != nil { + t.Fatal(err) + } + var second struct { + Keys []map[string]any `json:"keys"` + } + if err := json.Unmarshal(decodeManagementBody(t, raw), &second); err != nil { + t.Fatal(err) + } + if len(second.Keys) != 1 || second.Keys[0]["name"] != "QQ官Key改名" { + t.Fatalf("admin list should refresh alias changes and official deletions: %#v", second.Keys) + } + + raw, err = adminKeys(managementRequest{Query: map[string][]string{"include_removed": {"1"}}}) + if err != nil { + t.Fatal(err) + } + var withRemoved struct { + Keys []map[string]any `json:"keys"` + } + if err := json.Unmarshal(decodeManagementBody(t, raw), &withRemoved); err != nil { + t.Fatal(err) + } + if len(withRemoved.Keys) < 2 { + t.Fatalf("include_removed should expose removed native diagnostics: %#v", withRemoved.Keys) + } + seenRemovedQQ := false + for _, key := range withRemoved.Keys { + if key["source"] != policyplus.NativeCPASource || key["name"] == "旧 CPI 下划线 Key" { + t.Fatalf("include_removed leaked non-native row: %#v", key) + } + if key["name"] == "阿伟专用" && key["source_present"] == false { + seenRemovedQQ = true + } + } + if !seenRemovedQQ { + t.Fatalf("include_removed did not expose the official key removed by sync: %#v", withRemoved.Keys) + } +} + +func TestAdminKeysUsesAliasFallbackPath(t *testing.T) { + setupTestState(t) + dir := t.TempDir() + configPath := filepath.Join(dir, "config.yaml") + emptyAliasPath := filepath.Join(dir, "empty-cpamp.sqlite") + realAliasPath := filepath.Join(dir, "real-cpamp.sqlite") + if err := os.WriteFile(configPath, []byte("api-keys:\n - sk-alice\n"), 0o600); err != nil { + t.Fatal(err) + } + emptyDB, err := sql.Open("sqlite", emptyAliasPath) + if err != nil { + t.Fatal(err) + } + if _, err := emptyDB.Exec(`create table unrelated(id text)`); err != nil { + t.Fatal(err) + } + if err := emptyDB.Close(); err != nil { + t.Fatal(err) + } + realDB, err := sql.Open("sqlite", realAliasPath) + if err != nil { + t.Fatal(err) + } + if _, err := realDB.Exec(`create table api_key_aliases(api_key_hash text primary key, alias text, updated_at_ms integer)`); err != nil { + t.Fatal(err) + } + if _, err := realDB.Exec(`insert into api_key_aliases(api_key_hash, alias, updated_at_ms) values(?, ?, ?)`, "sha256:"+policyplus.SHA256Hex("sk-alice"), "alicea", time.Now().UnixMilli()); err != nil { + t.Fatal(err) + } + if err := realDB.Close(); err != nil { + t.Fatal(err) + } + + state.mu.Lock() + state.cfg.NativeKeysConfigPath = configPath + state.cfg.CPAMPAliasDBPath = emptyAliasPath + state.cfg.CPAMPAliasDBPaths = realAliasPath + state.mu.Unlock() + + raw, err := adminKeys(managementRequest{}) + if err != nil { + t.Fatal(err) + } + var resp struct { + Keys []map[string]any `json:"keys"` + } + if err := json.Unmarshal(decodeManagementBody(t, raw), &resp); err != nil { + t.Fatal(err) + } + if len(resp.Keys) != 1 || resp.Keys[0]["name"] != "alicea" || resp.Keys[0]["alias"] != "alicea" { + t.Fatalf("admin list should use fallback CPAMP alias source: %#v", resp.Keys) + } + if preview := fmt.Sprint(resp.Keys[0]["preview"]); preview == "" || strings.Contains(preview, "sk-alice") { + t.Fatalf("preview should stay safe and derived from the same native key: %#v", resp.Keys[0]) + } +} + +func TestPolicyDecisionDenialsUseExplicitCodes(t *testing.T) { + tests := []struct { + name string + mutate func(*testing.T, *policyplus.KeyRecord) + model string + code string + param string + }{ + { + name: "disabled", + mutate: func(t *testing.T, key *policyplus.KeyRecord) { + key.Enabled = false + }, + model: "gpt-5.5", + code: "api_key_disabled", + param: "disabled", + }, + { + name: "source removed", + mutate: func(t *testing.T, key *policyplus.KeyRecord) { + key.SourcePresent = false + }, + model: "gpt-5.5", + code: "api_key_source_removed", + param: "source_removed", + }, + { + name: "model not allowed", + model: "gpt-5.4", + code: "model_not_allowed", + param: "model", + }, + { + name: "rpm", + mutate: func(t *testing.T, key *policyplus.KeyRecord) { + key.RPM = 1 + }, + model: "gpt-5.5", + code: "rpm_rate_limit_exceeded", + param: "rpm", + }, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + key := setupTestState(t) + if tc.mutate != nil { + tc.mutate(t, &key) + } + if key.SourcePresent { + if err := loadedStore().SaveKeySettings(context.Background(), key); err != nil { + t.Fatal(err) + } + } + if tc.name == "source removed" { + if err := loadedStore().SyncNativeKeys(context.Background(), nil); err != nil { + t.Fatal(err) + } + } + if tc.name == "rpm" { + first := evaluatePolicy(key, tc.model, true) + if !first.Allowed { + t.Fatalf("first RPM request should pass: %#v", first) + } + } + decision := evaluatePolicy(key, tc.model, tc.name == "rpm") + if decision.Allowed || decision.StatusCode != http.StatusTooManyRequests || decision.Code != tc.code || decision.Param != tc.param || !strings.Contains(decision.Message, "CPA Key Policy+") { + t.Fatalf("decision = %#v", decision) + } + }) + } +} + +func TestPolicyDecisionQuotaWindowsUsePostAccountingOrder(t *testing.T) { + windows := []struct { + name string + set func(*policyplus.KeyRecord) + code string + param string + }{ + {policyplus.Range5H, func(k *policyplus.KeyRecord) { k.FiveHourUSD = floatPtr(1) }, "five_hour_quota_exceeded", policyplus.Range5H}, + {policyplus.Range24H, func(k *policyplus.KeyRecord) { k.DailyLimitUSD = floatPtr(1) }, "daily_quota_exceeded", policyplus.Range24H}, + {policyplus.Range7D, func(k *policyplus.KeyRecord) { k.WeeklyLimitUSD = floatPtr(1) }, "weekly_quota_exceeded", policyplus.Range7D}, + {policyplus.RangeMonth, func(k *policyplus.KeyRecord) { k.MonthlyLimitUSD = floatPtr(1) }, "monthly_quota_exceeded", policyplus.RangeMonth}, + } + for _, item := range windows { + t.Run(item.name, func(t *testing.T) { + key := setupTestState(t) + key.FiveHourUSD = nil + key.DailyLimitUSD = nil + key.WeeklyLimitUSD = nil + key.MonthlyLimitUSD = nil + item.set(&key) + if err := loadedStore().SaveKeySettings(context.Background(), key); err != nil { + t.Fatal(err) + } + if err := loadedStore().InsertUsage(context.Background(), policyplus.UsageEvent{ + RequestID: "quota-" + item.name, + KeyID: key.ID, + RequestedAt: time.Now(), + Cost: 1, + }); err != nil { + t.Fatal(err) + } + decision := evaluatePolicy(key, "gpt-5.5", false) + if decision.Allowed || decision.Code != item.code || decision.Param != item.param || decision.Window != item.name || decision.UsedUSD != 1 || decision.LimitUSD != 1 { + t.Fatalf("quota decision = %#v", decision) + } + if !strings.Contains(decision.Message, "已用 $1.00 / 上限 $1.00") { + t.Fatalf("quota message should expose used/limit: %s", decision.Message) + } + }) + } +} + +func TestRouteAndExecutorReturnPolicyDenied429(t *testing.T) { + key := setupTestState(t) + key.Enabled = false + if err := loadedStore().SaveKeySettings(context.Background(), key); err != nil { + t.Fatal(err) + } + routeRaw, err := routeModel(mustJSON(t, modelRouteRequest{ + RequestedModel: "gpt-5.5", + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + Body: []byte(`{"model":"gpt-5.5"}`), + })) + if err != nil { + t.Fatal(err) + } + var routeEnv envelope + if err := json.Unmarshal(routeRaw, &routeEnv); err != nil { + t.Fatal(err) + } + var routeResp modelRouteResponse + if err := json.Unmarshal(routeEnv.Result, &routeResp); err != nil { + t.Fatal(err) + } + if !routeResp.Handled || routeResp.TargetKind != routeTargetSelf || routeResp.Reason != "cpa_key_policy_plus_policy_denied" { + t.Fatalf("route response = %#v", routeResp) + } + + execRaw, err := executorExecute(mustJSON(t, executorCallRequest{ + executorRequest: executorRequest{ + Model: "gpt-5.5", + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + Payload: []byte(`{"model":"gpt-5.5"}`), + }, + })) + if err != nil { + t.Fatal(err) + } + var execEnv envelope + if err := json.Unmarshal(execRaw, &execEnv); err != nil { + t.Fatal(err) + } + var execResp executorResponse + if err := json.Unmarshal(execEnv.Result, &execResp); err != nil { + t.Fatal(err) + } + if execResp.Headers.Get("X-CPA-Policy-Reason") != "api_key_disabled" || execResp.Headers.Get("Retry-After") == "" { + t.Fatalf("executor deny response = %#v", execResp) + } + if execResp.Headers.Get("X-CPA-Policy-Window") != "disabled" { + t.Fatalf("executor deny headers should include safe policy window/param: %#v", execResp.Headers) + } + var body struct { + Error struct { + Message string `json:"message"` + Type string `json:"type"` + Code string `json:"code"` + Param string `json:"param"` + } `json:"error"` + } + if err := json.Unmarshal(execResp.Payload, &body); err != nil { + t.Fatal(err) + } + if body.Error.Code != "api_key_disabled" || body.Error.Message == "" || body.Error.Param != "disabled" { + t.Fatalf("deny body = %s", execResp.Payload) + } +} + +func TestFrontendAuthOnlySurfacesDeniedNativeKeysForModelRequests(t *testing.T) { + key := setupTestState(t) + key.Enabled = false + if err := loadedStore().SaveKeySettings(context.Background(), key); err != nil { + t.Fatal(err) + } + raw, err := frontendAuth(mustJSON(t, frontendAuthRequest{ + Path: "/v1/models", + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + })) + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var resp frontendAuthResponse + if err := json.Unmarshal(env.Result, &resp); err != nil { + t.Fatal(err) + } + if resp.Authenticated { + t.Fatalf("non-model denied request should fail closed through normal auth path: %#v", resp) + } + + raw, err = frontendAuth(mustJSON(t, frontendAuthRequest{ + Path: "/v1/responses", + Headers: http.Header{"Authorization": []string{"Bearer sk-alice-secret"}}, + Body: []byte(`{"model":"gpt-5.5"}`), + })) + if err != nil { + t.Fatal(err) + } + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + if err := json.Unmarshal(env.Result, &resp); err != nil { + t.Fatal(err) + } + if !resp.Authenticated || resp.Metadata[policyDenyMetadataPrefix+"code"] != "api_key_disabled" { + t.Fatalf("responses denied request should route to explicit policy body: %#v", resp) + } +} + +func TestExecutorRequestNormalizationKeepsNestedPayload(t *testing.T) { + body := []byte(`{"model":"gpt-5.5","stream":true}`) + req := normalizedExecutorRequest(executorCallRequest{ + NestedExecutorRequest: executorRequest{ + Model: "gpt-5.5", + Payload: body, + }, + }) + if req.Model != "gpt-5.5" || string(req.Payload) != string(body) { + t.Fatalf("normalized nested request = %#v", req) + } +} + +func TestAdminKeysIncludesQuotaProjection(t *testing.T) { + setupTestState(t) + store := loadedStore() + if err := store.InsertUsage(context.Background(), policyplus.UsageEvent{ + RequestID: "usage-a", + KeyID: "alice-key", + RequestedAt: time.Now().Add(-time.Hour), + Cost: 2.5, + }); err != nil { + t.Fatal(err) + } + raw, err := adminKeys(managementRequest{}) + if err != nil { + t.Fatal(err) + } + body := decodeManagementBody(t, raw) + text := string(body) + for _, want := range []string{`"usage"`, `"quota"`, `"24h"`, `"remaining_usd"`} { + if !strings.Contains(text, want) { + t.Fatalf("admin key projection missing %s: %s", want, body) + } + } +} + +func TestUsageHandleMapsExecutorRecordsByAuthID(t *testing.T) { + key := setupTestState(t) + key.Models = []string{"gpt-5.4"} + key.Prices = map[string]policyplus.ModelPrice{ + "gpt-5.4": {Model: "gpt-5.4", InputPerMillion: 10, OutputPerMillion: 20}, + } + if err := loadedStore().SaveKeySettings(context.Background(), key); err != nil { + t.Fatal(err) + } + body, _ := json.Marshal(usageRecord{ + Provider: "codex-account-3.json", + ExecutorType: "codex", + Model: "gpt-5.3-codex-spark", + AuthID: key.ID, + Source: "codex-account-3.json", + RequestedAt: time.Now(), + Detail: usageDetail{ + InputTokens: 10, + OutputTokens: 5, + ReasoningTokens: 2, + TotalTokens: 15, + }, + }) + raw, err := usageHandle(body) + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + if !env.OK { + t.Fatalf("usage response = %s", raw) + } + events, err := loadedStore().RecentEvents(context.Background(), key.ID, 10) + if err != nil { + t.Fatal(err) + } + if len(events) != 1 { + t.Fatalf("events = %#v", events) + } + got := events[0] + if got.KeyID != key.ID || got.Model != "gpt-5.4" || got.RequestedModel != "gpt-5.4" || got.ActualModel != "gpt-5.3-codex-spark" { + t.Fatalf("usage event did not preserve auth/alias mapping: %#v", got) + } + if got.Cost <= 0 { + t.Fatalf("usage event should use visible model price book, got cost=%f event=%#v", got.Cost, got) + } +} + +func TestVisibleUsageModelDoesNotGuessForMultiModelKeys(t *testing.T) { + key := policyplus.KeyRecord{ID: "key-1", Models: []string{"gpt-5.4", "gpt-5.5"}} + got := visibleUsageModel(key, usageRecord{Model: "provider-internal-unknown"}) + if got != "provider-internal-unknown" { + t.Fatalf("multi-model key should not guess visible alias, got %q", got) + } + got = visibleUsageModel(key, usageRecord{Model: "gpt-5.3-codex-spark"}) + if got != "gpt-5.4" { + t.Fatalf("known executor alias should be projected to visible model, got %q", got) + } + got = visibleUsageModel(key, usageRecord{Model: "gpt-5.3-codex-spark", Alias: "gpt-5.3-codex-spark"}) + if got != "gpt-5.4" { + t.Fatalf("internal alias field should be projected to visible model, got %q", got) + } + got = visibleUsageModel(key, usageRecord{Model: "gpt-5.3-codex-spark", Alias: "gpt-5.4"}) + if got != "gpt-5.4" { + t.Fatalf("explicit alias should win, got %q", got) + } +} + +func TestAdminModelsFallsBackToConfiguredModels(t *testing.T) { + key := setupTestState(t) + key.Models = []string{"gpt-5.5", "legacy-custom"} + key.Prices = map[string]policyplus.ModelPrice{ + "price-only": {Model: "price-only", InputPerMillion: 1}, + } + if err := loadedStore().SaveKeySettings(context.Background(), key); err != nil { + t.Fatal(err) + } + raw, err := adminModels(managementRequest{}) + if err != nil { + t.Fatal(err) + } + body := decodeManagementBody(t, raw) + text := string(body) + for _, want := range []string{"gpt-5.5", "legacy-custom", "price-only"} { + if !strings.Contains(text, want) { + t.Fatalf("model catalog missing %s: %s", want, text) + } + } +} + +func TestUserHTMLUsesPolicyPlusHeader(t *testing.T) { + html := userHTML() + if !strings.Contains(html, "X-CPA-Key-Policy-Plus-Key") { + t.Fatal("user page must send the Plus login header") + } + if strings.Contains(html, "X-CPA-Governor-Key") { + t.Fatal("user page should not keep the Governor login header") + } + for _, want := range []string{"完整原生 sk- Key", "旧的 cpa_ Key 已迁移下线", "只接受 CPA 原生 sk- Key"} { + if !strings.Contains(html, want) { + t.Fatalf("user page missing native-key guidance %q", want) + } + } +} + +func TestUserHTMLFixedRangeUX(t *testing.T) { + html := userHTML() + for _, removed := range []string{`id="rangeSelect"`, "rangeSelect", "state.range"} { + if strings.Contains(html, removed) { + t.Fatalf("user page should not keep mutable range control %q", removed) + } + } + for _, want := range []string{ + `const PRIMARY_RANGE = "24h";`, + "${rangeLabel(PRIMARY_RANGE)}费用", + "${rangeLabel(PRIMARY_RANGE)} · 最近", + "24H / 7D", + "5H / 本月", + "api(`/usage?range=${encodeURIComponent(PRIMARY_RANGE)}`", + "api(`/events?range=${encodeURIComponent(PRIMARY_RANGE)}&limit=100`", + } { + if !strings.Contains(html, want) { + t.Fatalf("user fixed-range page missing %q", want) + } + } +} + +func TestUserHTMLIgnoresRefreshCancelNoise(t *testing.T) { + html := userHTML() + for _, want := range []string{ + "function isRefreshCancel", + `abort("refresh_cancelled")`, + `controller.abort("refresh_timeout")`, + } { + if !strings.Contains(html, want) { + t.Fatalf("user refresh cancel handling missing %q", want) + } + } + usageCancel := strings.Index(html, "if (isRefreshCancel(controller.signal)) return;") + usageError := strings.Index(html, `state.errors.usage = "用量接口同步失败："`) + if usageCancel < 0 || usageError < 0 || usageCancel > usageError { + t.Fatal("usage refresh cancel must be ignored before setting a sync error") + } + protectionError := strings.Index(html, `state.errors.protection = "思维链保护同步失败："`) + if protectionError < 0 { + t.Fatal("protection sync error assignment missing") + } + protectionCancel := strings.LastIndex(html[:protectionError], "if (isRefreshCancel(controller.signal)) return;") + if protectionCancel < 0 { + t.Fatal("protection refresh cancel must be ignored before setting a sync error") + } +} + +func TestCodexRequestsPreferExecutorSummaryBridge(t *testing.T) { + key := setupTestState(t) + execPath := filepath.Join(t.TempDir(), "executor.sqlite") + execStore, err := policyplus.OpenStore(execPath) + if err != nil { + t.Fatal(err) + } + if err := execStore.SaveCodexSummary(context.Background(), "exec-a", key.ID, "gpt-5.5", "auto_continued", map[string]any{ + "request_id": "exec-a", + "model": "gpt-5.5", + "protection": "auto_continued", + "key_identity": map[string]any{"known": true, "id": key.ID, "preview": key.Preview}, + "continuation_count": 1, + }); err != nil { + t.Fatal(err) + } + if err := execStore.SaveCodexSummary(context.Background(), "exec-b", "bob-key", "gpt-5.5", "protected_clean", map[string]any{ + "request_id": "exec-b", + "model": "gpt-5.5", + "protection": "protected_clean", + "key_identity": map[string]any{"known": true, "id": "bob-key"}, + }); err != nil { + t.Fatal(err) + } + if err := execStore.Close(); err != nil { + t.Fatal(err) + } + state.mu.Lock() + state.cfg.CodexSummaryDBPath = execPath + state.cfg.CodexContEnabled = false + state.mu.Unlock() + requests, source := codexRequestsForKey(key, 10) + if source != "codexcont_executor_store" || len(requests) != 1 || requests[0]["request_id"] != "exec-a" { + t.Fatalf("source=%s requests=%#v", source, requests) + } +} + +func TestAdminHTMLHasRenderedSharedCSS(t *testing.T) { + html := adminHTML() + if strings.Contains(html, "{{CSS}}") || strings.Contains(html, "{{SHARED_CSS}}") { + t.Fatalf("admin html still has css placeholder") + } + if !strings.Contains(html, "--panel") || !strings.Contains(html, "CPA Key Policy+") { + t.Fatal("admin html should embed shared style and key policy UI") + } + if strings.Contains(html, "/v0/resource/plugins/cpa-key-policy-plus/admin/api") { + t.Fatal("admin html must not send mutating requests through GET-only resource routes") + } + if !strings.Contains(html, "/key-policy-plus/api") || !strings.Contains(html, "编辑模型/价格") { + t.Fatal("admin html should use the management alias and structured model editor") + } + for _, removed := range []string{"请求并发", "Codex窗口", "显示归档", "归档隐藏", "恢复 Key"} { + if strings.Contains(html, removed) { + t.Fatalf("admin html should not contain retired control %q", removed) + } + } + for _, removed := range []string{"新建 Key", "删除 Key", "复制完整 Key", "raw-key", "/keys/create", "/keys/delete"} { + if strings.Contains(html, removed) { + t.Fatalf("admin html should not expose Plus-side key lifecycle control %q", removed) + } + } + for _, want := range []string{"Key 策略", "保存策略", "当前官方 Key", "新原生 Key 默认启用", "未配置限额"} { + if !strings.Contains(html, want) { + t.Fatalf("admin html missing native policy UI marker %q", want) + } + } + if strings.Contains(html, "新原生 Key 默认禁用") || strings.Contains(html, "策略总数") || strings.Contains(html, "显示官方已移除") || strings.Contains(html, "include_removed") { + t.Fatal("admin html should not advertise old disabled/default, historical count, or removed-row controls") + } +} + +func TestSessionInvalidatedWhenKeyHashChanges(t *testing.T) { + key := setupTestState(t) + req := managementRequest{Headers: http.Header{"X-CPA-Key-Policy-Plus-Key": []string{"sk-alice-secret"}}} + raw, err := userSession(req) + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var resp managementResponse + if err := json.Unmarshal(env.Result, &resp); err != nil { + t.Fatal(err) + } + cookie := strings.Join(resp.Headers.Values("Set-Cookie"), "; ") + if cookie == "" { + t.Fatal("session did not set cookie") + } + if _, ok := keyFromSession(managementRequest{Headers: http.Header{"Cookie": []string{cookie}}}); !ok { + t.Fatal("fresh session should resolve") + } + key.KeyHash = "sha256:" + policyplus.SHA256Hex("sk-rotated-secret") + if err := loadedStore().UpsertKey(context.Background(), key); err != nil { + t.Fatal(err) + } + if _, ok := keyFromSession(managementRequest{Headers: http.Header{"Cookie": []string{cookie}}}); ok { + t.Fatal("old session should not survive key rotation") + } +} + +func TestSessionCookiePathCompatibility(t *testing.T) { + setupTestState(t) + raw, err := userSession(managementRequest{Headers: http.Header{"X-CPA-Key-Policy-Plus-Key": []string{"sk-alice-secret"}}}) + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var resp managementResponse + if err := json.Unmarshal(env.Result, &resp); err != nil { + t.Fatal(err) + } + cookies := resp.Headers.Values("Set-Cookie") + for _, want := range []string{ + "Path=/;", + "Path=/v0/resource/plugins/cpa-key-policy-plus/user;", + "Path=/key-policy-plus-user;", + } { + if !strings.Contains(strings.Join(cookies, "\n"), want) { + t.Fatalf("session response should set compatible cookie path %s: %#v", want, cookies) + } + } + valid := "" + for _, rawCookie := range cookies { + if strings.Contains(rawCookie, "Path=/;") { + valid = strings.SplitN(rawCookie, ";", 2)[0] + break + } + } + if valid == "" { + t.Fatalf("root session cookie not found: %#v", cookies) + } + staleFirst := "cpa_key_policy_plus_session=stale-invalid-token; " + valid + if _, ok := keyFromSession(managementRequest{Headers: http.Header{"Cookie": []string{staleFirst}}}); !ok { + t.Fatal("fresh session should resolve even when a stale path-specific cookie is sent first") + } +} + +func authOK(t *testing.T, req frontendAuthRequest) bool { + t.Helper() + rawReq, _ := json.Marshal(req) + raw, err := frontendAuth(rawReq) + if err != nil { + t.Fatal(err) + } + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var resp frontendAuthResponse + if err := json.Unmarshal(env.Result, &resp); err != nil { + t.Fatal(err) + } + return resp.Authenticated +} + +func decodeManagementBody(t *testing.T, raw []byte) []byte { + t.Helper() + return decodeManagementResponse(t, raw).Body +} + +func decodeManagementResponse(t *testing.T, raw []byte) managementResponse { + t.Helper() + var env envelope + if err := json.Unmarshal(raw, &env); err != nil { + t.Fatal(err) + } + var resp managementResponse + if err := json.Unmarshal(env.Result, &resp); err != nil { + t.Fatal(err) + } + return resp +} + +func mustJSON(t *testing.T, value any) []byte { + t.Helper() + raw, err := json.Marshal(value) + if err != nil { + t.Fatal(err) + } + return raw +} + +func TestMain(m *testing.M) { + os.Exit(m.Run()) +} diff --git a/cpa_key_policy_plus_plugin/go/plugin_export.go b/cpa_key_policy_plus_plugin/go/plugin_export.go new file mode 100644 index 0000000..f0db1d4 --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/plugin_export.go @@ -0,0 +1,166 @@ +//go:build cliproxy_plugin + +package main + +/* +#include +#include + +typedef struct { + void* ptr; + size_t len; +} cliproxy_buffer; + +typedef int (*cliproxy_host_call_fn)(void*, const char*, const uint8_t*, size_t, cliproxy_buffer*); +typedef void (*cliproxy_host_free_fn)(void*, size_t); + +typedef struct { + uint32_t abi_version; + void* host_ctx; + cliproxy_host_call_fn call; + cliproxy_host_free_fn free_buffer; +} cliproxy_host_api; + +typedef int (*cliproxy_plugin_call_fn)(char*, uint8_t*, size_t, cliproxy_buffer*); +typedef void (*cliproxy_plugin_free_fn)(void*, size_t); +typedef void (*cliproxy_plugin_shutdown_fn)(void); + +typedef struct { + uint32_t abi_version; + cliproxy_plugin_call_fn call; + cliproxy_plugin_free_fn free_buffer; + cliproxy_plugin_shutdown_fn shutdown; +} cliproxy_plugin_api; + +extern int cliproxyPluginCall(char*, uint8_t*, size_t, cliproxy_buffer*); +extern void cliproxyPluginFree(void*, size_t); +extern void cliproxyPluginShutdown(void); + +static const cliproxy_host_api* stored_host; + +static void store_host_api(const cliproxy_host_api* host) { + stored_host = host; +} + +static int call_host_api(const char* method, const uint8_t* request, size_t request_len, cliproxy_buffer* response) { + if (stored_host == NULL || stored_host->call == NULL) { + return 1; + } + return stored_host->call(stored_host->host_ctx, method, request, request_len, response); +} + +static void free_host_buffer(void* ptr, size_t len) { + if (stored_host != NULL && stored_host->free_buffer != NULL && ptr != NULL) { + stored_host->free_buffer(ptr, len); + } +} +*/ +import "C" + +import ( + "encoding/json" + "fmt" + "unsafe" +) + +//export cliproxy_plugin_init +func cliproxy_plugin_init(host *C.cliproxy_host_api, plugin *C.cliproxy_plugin_api) C.int { + if plugin == nil { + return 1 + } + C.store_host_api(host) + hostCall = cgoHostCall + plugin.abi_version = C.uint32_t(abiVersion) + plugin.call = C.cliproxy_plugin_call_fn(C.cliproxyPluginCall) + plugin.free_buffer = C.cliproxy_plugin_free_fn(C.cliproxyPluginFree) + plugin.shutdown = C.cliproxy_plugin_shutdown_fn(C.cliproxyPluginShutdown) + return 0 +} + +//export cliproxyPluginCall +func cliproxyPluginCall(method *C.char, request *C.uint8_t, requestLen C.size_t, response *C.cliproxy_buffer) C.int { + if response != nil { + response.ptr = nil + response.len = 0 + } + if method == nil { + writeCResponse(response, errorEnvelope("invalid_method", "method is required")) + return 1 + } + var requestBytes []byte + if request != nil && requestLen > 0 { + requestBytes = C.GoBytes(unsafe.Pointer(request), C.int(requestLen)) + } + raw, errHandle := handleMethod(C.GoString(method), requestBytes) + if errHandle != nil { + writeCResponse(response, errorEnvelope("plugin_error", errHandle.Error())) + return 1 + } + writeCResponse(response, raw) + return 0 +} + +//export cliproxyPluginFree +func cliproxyPluginFree(ptr unsafe.Pointer, _ C.size_t) { + if ptr != nil { + C.free(ptr) + } +} + +//export cliproxyPluginShutdown +func cliproxyPluginShutdown() { + shutdownPlugin() +} + +func writeCResponse(response *C.cliproxy_buffer, raw []byte) { + if response == nil || len(raw) == 0 { + return + } + ptr := C.CBytes(raw) + if ptr == nil { + return + } + response.ptr = ptr + response.len = C.size_t(len(raw)) +} + +func cgoHostCall(method string, payload any) (json.RawMessage, error) { + rawPayload, err := json.Marshal(payload) + if err != nil { + return nil, err + } + cMethod := C.CString(method) + defer C.free(unsafe.Pointer(cMethod)) + var response C.cliproxy_buffer + var requestPtr *C.uint8_t + if len(rawPayload) > 0 { + cPayload := C.CBytes(rawPayload) + if cPayload == nil { + return nil, fmt.Errorf("allocate host callback") + } + defer C.free(cPayload) + requestPtr = (*C.uint8_t)(cPayload) + } + callCode := C.call_host_api(cMethod, requestPtr, C.size_t(len(rawPayload)), &response) + var rawResponse []byte + if response.ptr != nil && response.len > 0 { + rawResponse = C.GoBytes(response.ptr, C.int(response.len)) + } + if response.ptr != nil { + C.free_host_buffer(response.ptr, response.len) + } + if callCode != 0 || len(rawResponse) == 0 { + return nil, fmt.Errorf("host callback %s returned code=%d", method, int(callCode)) + } + var env envelope + if err := json.Unmarshal(rawResponse, &env); err != nil { + return nil, err + } + if !env.OK { + if env.Error != nil { + return nil, fmt.Errorf("%s: %s", env.Error.Code, env.Error.Message) + } + return nil, fmt.Errorf("host callback failed") + } + return env.Result, nil +} diff --git a/cpa_key_policy_plus_plugin/go/plugin_types.go b/cpa_key_policy_plus_plugin/go/plugin_types.go new file mode 100644 index 0000000..a04e40a --- /dev/null +++ b/cpa_key_policy_plus_plugin/go/plugin_types.go @@ -0,0 +1,234 @@ +package main + +import ( + "net/http" + "net/url" + "time" +) + +const ( + abiVersion uint32 = 1 + schemaVersion uint32 = 1 + + methodPluginRegister = "plugin.register" + methodPluginReconfigure = "plugin.reconfigure" + methodFrontendAuthIdentifier = "frontend_auth.identifier" + methodFrontendAuthAuthenticate = "frontend_auth.authenticate" + methodModelRoute = "model.route" + methodExecutorIdentifier = "executor.identifier" + methodExecutorExecute = "executor.execute" + methodExecutorExecuteStream = "executor.execute_stream" + methodExecutorCountTokens = "executor.count_tokens" + methodUsageHandle = "usage.handle" + methodManagementRegister = "management.register" + methodManagementHandle = "management.handle" + methodHostModelExecute = "host.model.execute" + methodHostModelExecuteStream = "host.model.execute_stream" + methodHostModelStreamRead = "host.model.stream_read" + methodHostModelStreamClose = "host.model.stream_close" + methodHostStreamEmit = "host.stream.emit" + methodHostStreamClose = "host.stream.close" + methodHostAuthList = "host.auth.list" +) + +const ( + configString = "string" + configBoolean = "boolean" + configEnum = "enum" + + routeTargetSelf = "self" +) + +type configField struct { + Name string `json:"Name"` + Type string `json:"Type"` + EnumValues []string `json:"EnumValues,omitempty"` + Description string `json:"Description"` +} + +type frontendAuthRequest struct { + Method string `json:"Method"` + Path string `json:"Path"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` +} + +type frontendAuthResponse struct { + Authenticated bool `json:"Authenticated"` + Principal string `json:"Principal,omitempty"` + Metadata map[string]string `json:"Metadata,omitempty"` +} + +type modelRouteRequest struct { + PluginID string `json:"PluginID"` + SourceFormat string `json:"SourceFormat"` + RequestedModel string `json:"RequestedModel"` + Stream bool `json:"Stream"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` + Metadata map[string]any `json:"Metadata"` + AvailableProviders []string `json:"AvailableProviders"` +} + +type modelRouteResponse struct { + Handled bool `json:"Handled"` + TargetKind string `json:"TargetKind,omitempty"` + Target string `json:"Target,omitempty"` + TargetModel string `json:"TargetModel,omitempty"` + Reason string `json:"Reason,omitempty"` +} + +type managementRegistrationResponse struct { + Routes []managementRoute `json:"routes,omitempty"` + Resources []resourceRoute `json:"resources,omitempty"` +} + +type managementRoute struct { + Method string `json:"Method"` + Path string `json:"Path"` + Menu string `json:"Menu,omitempty"` + Description string `json:"Description,omitempty"` +} + +type resourceRoute struct { + Path string `json:"Path"` + Menu string `json:"Menu,omitempty"` + Description string `json:"Description,omitempty"` +} + +type managementRequest struct { + Method string `json:"Method"` + Path string `json:"Path"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + Body []byte `json:"Body"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type managementResponse struct { + StatusCode int `json:"StatusCode"` + Headers http.Header `json:"Headers"` + Body []byte `json:"Body"` +} + +type executorResponse struct { + Payload []byte `json:"Payload"` + Headers http.Header `json:"Headers,omitempty"` + Metadata map[string]any `json:"Metadata,omitempty"` +} + +type usageRecord struct { + Provider string `json:"Provider"` + ExecutorType string `json:"ExecutorType"` + Model string `json:"Model"` + Alias string `json:"Alias"` + APIKey string `json:"APIKey"` + AuthID string `json:"AuthID"` + AuthIndex string `json:"AuthIndex"` + AuthType string `json:"AuthType"` + Source string `json:"Source"` + ReasoningEffort string `json:"ReasoningEffort"` + ServiceTier string `json:"ServiceTier"` + RequestedAt time.Time `json:"RequestedAt"` + Latency time.Duration `json:"Latency"` + TTFT time.Duration `json:"TTFT"` + Failed bool `json:"Failed"` + Failure usageFailure `json:"Failure"` + Detail usageDetail `json:"Detail"` + ResponseHeaders http.Header `json:"ResponseHeaders"` +} + +type executorRequest struct { + AuthID string `json:"AuthID"` + AuthProvider string `json:"AuthProvider"` + Model string `json:"Model"` + Format string `json:"Format"` + Stream bool `json:"Stream"` + Alt string `json:"Alt"` + Headers http.Header `json:"Headers"` + Query url.Values `json:"Query"` + OriginalRequest []byte `json:"OriginalRequest"` + SourceFormat string `json:"SourceFormat"` + Payload []byte `json:"Payload"` + Metadata map[string]any `json:"Metadata"` + StorageJSON []byte `json:"StorageJSON"` + AuthMetadata map[string]any `json:"AuthMetadata"` + AuthAttributes map[string]string `json:"AuthAttributes"` +} + +type executorCallRequest struct { + executorRequest + NestedExecutorRequest executorRequest `json:"ExecutorRequest,omitempty"` + StreamID string `json:"stream_id,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type executorStreamResponse struct { + Headers http.Header `json:"headers,omitempty"` +} + +type hostModelExecutionRequest struct { + EntryProtocol string `json:"entry_protocol"` + ExitProtocol string `json:"exit_protocol"` + Model string `json:"model"` + Stream bool `json:"stream"` + Body []byte `json:"body"` + Headers http.Header `json:"headers"` + Query url.Values `json:"query"` + Alt string `json:"alt,omitempty"` + HostCallbackID string `json:"host_callback_id,omitempty"` +} + +type hostModelExecutionResponse struct { + StatusCode int `json:"status_code"` + Headers http.Header `json:"headers"` + Body []byte `json:"body"` +} + +type hostModelStreamResponse struct { + StatusCode int `json:"status_code"` + Headers http.Header `json:"headers"` + StreamID string `json:"stream_id"` +} + +type hostModelStreamReadRequest struct { + StreamID string `json:"stream_id"` +} + +type hostModelStreamReadResponse struct { + Payload []byte `json:"payload"` + Error string `json:"error"` + Done bool `json:"done"` +} + +type hostModelStreamCloseRequest struct { + StreamID string `json:"stream_id"` +} + +type hostStreamEmitRequest struct { + StreamID string `json:"stream_id"` + Payload []byte `json:"payload,omitempty"` + Error string `json:"error,omitempty"` +} + +type hostStreamCloseRequest struct { + StreamID string `json:"stream_id"` + Error string `json:"error,omitempty"` +} + +type usageFailure struct { + StatusCode int `json:"StatusCode"` + Body string `json:"Body"` +} + +type usageDetail struct { + InputTokens int64 `json:"InputTokens"` + OutputTokens int64 `json:"OutputTokens"` + ReasoningTokens int64 `json:"ReasoningTokens"` + CachedTokens int64 `json:"CachedTokens"` + CacheReadTokens int64 `json:"CacheReadTokens"` + CacheCreationTokens int64 `json:"CacheCreationTokens"` + TotalTokens int64 `json:"TotalTokens"` +} diff --git a/cpa_usage_portal/__init__.py b/cpa_usage_portal/__init__.py new file mode 100644 index 0000000..647bff3 --- /dev/null +++ b/cpa_usage_portal/__init__.py @@ -0,0 +1,6 @@ +"""Self-service CPA usage portal.""" + +from .app import create_app +from .config import PortalConfig, load_config_from_env + +__all__ = ["PortalConfig", "create_app", "load_config_from_env"] diff --git a/cpa_usage_portal/app.py b/cpa_usage_portal/app.py new file mode 100644 index 0000000..e5a2ff5 --- /dev/null +++ b/cpa_usage_portal/app.py @@ -0,0 +1,707 @@ +"""Starlette app for the self-service CPA usage portal.""" +from __future__ import annotations + +import asyncio +import contextlib +import json +import math +from pathlib import Path +from typing import Any + +import httpx +from starlette.applications import Starlette +from starlette.requests import Request +from starlette.responses import HTMLResponse, JSONResponse, Response, StreamingResponse +from starlette.routing import Route + +from .config import PortalConfig +from .cpamp import CPAMPClient, SUPPORTED_RANGES, range_window +from .key_policy import KeyPolicyState, KeyRecord +from .pricing import apply_event_pricing, apply_key_policy_pricing +from .quota_state import QuotaState, RESET_WINDOWS +from .redaction import safe_api_key_stats, safe_events +from .security import sha256_hex, sign_session, verify_session + +_STATIC = Path(__file__).with_name("static") +_DASHBOARD = _STATIC / "dashboard.html" +_ADMIN_DASHBOARD = _STATIC / "admin.html" + + +class AuthError(Exception): + pass + + +def _json_error(message: str, status_code: int) -> JSONResponse: + return JSONResponse({"error": message}, status_code=status_code) + + +def _load_key_state(request: Request) -> KeyPolicyState: + cfg: PortalConfig = request.app.state.cfg + return KeyPolicyState.load(cfg.key_policy_state_path) + + +def _session_payload(request: Request) -> dict[str, Any]: + cfg: PortalConfig = request.app.state.cfg + token = request.cookies.get(cfg.session_cookie_name, "") + payload = verify_session(token, cfg.session_secret) + if payload is None: + raise AuthError("not_authenticated") + return payload + + +def _current_key(request: Request): + payload = _session_payload(request) + state = _load_key_state(request) + record = state.get_by_raw_hash(str(payload.get("key_hash") or "")) + if record is None or not record.enabled: + raise AuthError("key_not_available") + return record + + +def _quota(request: Request) -> QuotaState: + return request.app.state.quota + + +def _record_id(record: KeyRecord) -> str: + return record.policy_id or record.raw_key_hash + + +def _safe_record(record: KeyRecord, quota: QuotaState) -> dict[str, Any]: + safe = record.safe_dict() + local_limits = quota.get_limits(_record_id(record)) + limits = dict(safe.get("limits") or {}) + limits.update(local_limits.safe_dict()) + safe["limits"] = limits + safe["reset_points"] = quota.get_reset_points(_record_id(record)) + return safe + + +def _safe_key_summary(record: KeyRecord) -> dict[str, Any]: + safe = record.safe_dict() + return { + "id": safe.get("id") or _record_id(record), + "name": safe.get("name") or "", + "preview": safe.get("preview") or "", + "enabled": bool(safe.get("enabled")), + } + + +def _find_record(state: KeyPolicyState, key_id: str) -> KeyRecord | None: + for record in state.keys: + if _record_id(record) == key_id: + return record + return None + + +def _parse_before(request: Request) -> tuple[int | None, int | None]: + before_ms = request.query_params.get("before_ms") + before_id = request.query_params.get("before_id") + if request.query_params.get("before") and (not before_ms and not before_id): + parts = request.query_params["before"].split(":", 1) + before_ms = parts[0] if parts else None + before_id = parts[1] if len(parts) > 1 else None + try: + parsed_ms = int(before_ms) if before_ms else None + except ValueError: + parsed_ms = None + try: + parsed_id = int(before_id) if before_id else None + except ValueError: + parsed_id = None + return parsed_ms, parsed_id + + +def _parse_range(request: Request, *, default: str) -> str: + value = request.query_params.get("range", default) + return value if value in SUPPORTED_RANGES else default + + +def _parse_float_limit(value: Any) -> float | None: + if value is None or value == "": + return None + try: + parsed = float(value) + except (TypeError, ValueError): + raise ValueError("invalid_limit") + if parsed < 0 or not math.isfinite(parsed): + raise ValueError("invalid_limit") + return parsed + + +def _actor(request: Request) -> str: + return ( + request.headers.get("cf-access-authenticated-user-email") + or request.headers.get("x-usage-admin-actor") + or "admin" + ) + + +def _admin_allowed(request: Request) -> bool: + cfg: PortalConfig = request.app.state.cfg + return request.headers.get(cfg.admin_header_name) == cfg.admin_header_value + + +def _admin_guard(request: Request) -> JSONResponse | None: + if _admin_allowed(request): + return None + return _json_error("not_found", 404) + + +def _limit_for(record: KeyRecord, quota: QuotaState, range_name: str) -> float | None: + local = quota.get_limits(_record_id(record)) + if range_name == "5h": + return local.five_hour_usd + if range_name == "24h": + return record.daily_limit_usd + if range_name == "7d": + return record.weekly_limit_usd + if range_name == "month": + return local.monthly_usd + return None + + +def _quota_projection( + record: KeyRecord, + quota: QuotaState, + *, + range_name: str, + window_from_ms: int, + window_to_ms: int, + reset_at_ms: int | None, + used_usd: Any, +) -> dict[str, Any]: + limit = _limit_for(record, quota, range_name) + used = _float(used_usd) + remaining = None + percent = None + if limit is not None and limit > 0: + remaining = max(limit - used, 0.0) + percent = used / limit + return { + "range": range_name, + "from_ms": window_from_ms, + "to_ms": window_to_ms, + "reset_at_ms": reset_at_ms, + "limit_usd": limit, + "used_usd": used, + "remaining_usd": remaining, + "used_percent": percent, + } + + +def _float(value: Any) -> float: + try: + return float(value or 0) + except (TypeError, ValueError): + return 0.0 + + +def _attach_event_accounting( + events: list[dict[str, Any]], + *, + record: KeyRecord, + quota: QuotaState, + selected_range: str, + selected_quota: dict[str, Any], + now_ms_value: int, +) -> list[dict[str, Any]]: + effective_windows: dict[str, tuple[int, int, int | None]] = {} + key_id = _record_id(record) + for name in RESET_WINDOWS: + window, reset_at = quota.effective_window(key_id, name, now_ms_value=now_ms_value) + effective_windows[name] = (window.from_ms, window.to_ms, reset_at) + + projected: list[dict[str, Any]] = [] + for event in events: + row = dict(event) + ts = int(row.get("timestamp_ms") or 0) + included = [ + name + for name, (from_ms, to_ms, _reset_at) in effective_windows.items() + if ts and from_ms <= ts <= to_ms + ] + window_from, window_to, reset_at = effective_windows.get(selected_range, (0, 0, None)) + row["accounting"] = { + "selected_range": selected_range, + "included_windows": included, + "window_from_ms": window_from, + "window_to_ms": window_to, + "reset_at_ms": reset_at, + "current_window_remaining_usd": selected_quota.get("remaining_usd"), + "current_window_limit_usd": selected_quota.get("limit_usd"), + } + projected.append(row) + return projected + + +async def _events_for_record( + request: Request, + record: KeyRecord, + *, + range_name: str, + limit: int, +) -> tuple[list[dict[str, Any]], dict[str, Any]]: + quota_state = _quota(request) + window, reset_at = quota_state.effective_window(_record_id(record), range_name) + data = await request.app.state.cpamp.analytics( + api_key_hash=record.cpamp_hash, + window=window, + include_events=True, + include_model_stats=True, + event_limit=limit, + ) + data = apply_key_policy_pricing(data, record.model_prices) + quota_summary = _quota_projection( + record, + quota_state, + range_name=range_name, + window_from_ms=window.from_ms, + window_to_ms=window.to_ms, + reset_at_ms=reset_at, + used_usd=(data.get("summary") or {}).get("total_cost"), + ) + page = data.get("events") or {} + items = apply_event_pricing( + safe_events(page.get("items") or [], expected_hash=record.cpamp_hash), + record.model_prices, + ) + items = _attach_event_accounting( + items, + record=record, + quota=quota_state, + selected_range=range_name, + selected_quota=quota_summary, + now_ms_value=window.to_ms, + ) + key_summary = _safe_key_summary(record) + for item in items: + item["key"] = key_summary + return items, { + "range": range_name, + "from_ms": window.from_ms, + "to_ms": window.to_ms, + "reset_at_ms": reset_at, + "quota": quota_summary, + } + + +async def dashboard(_request: Request) -> HTMLResponse: + return HTMLResponse(_DASHBOARD.read_text(encoding="utf-8")) + + +async def healthz(request: Request) -> JSONResponse: + cfg: PortalConfig = request.app.state.cfg + state_ok = Path(cfg.key_policy_state_path).exists() + cpamp_ok: bool | None = None + try: + await request.app.state.cpamp.health() + cpamp_ok = True + except Exception: + cpamp_ok = False + return JSONResponse({"ok": state_ok and bool(cpamp_ok), "key_policy_state": state_ok, "cpamp": cpamp_ok}) + + +async def create_session(request: Request) -> JSONResponse: + try: + body = await request.json() + except json.JSONDecodeError: + return _json_error("invalid_json", 400) + api_key = str((body or {}).get("api_key") or "").strip() + if not api_key: + return _json_error("api_key_required", 400) + + key_hash = sha256_hex(api_key) + state = _load_key_state(request) + record = state.get_by_raw_hash(key_hash) + if record is None: + return _json_error("invalid_api_key", 401) + if not record.enabled: + return _json_error("api_key_disabled", 403) + + cfg: PortalConfig = request.app.state.cfg + token = sign_session( + {"key_hash": record.raw_key_hash, "cpamp_hash": record.cpamp_hash, "key_name": record.name}, + cfg.session_secret, + ttl_seconds=cfg.session_ttl_seconds, + ) + response = JSONResponse({"me": _safe_record(record, _quota(request))}) + response.set_cookie( + cfg.session_cookie_name, + token, + max_age=cfg.session_ttl_seconds, + httponly=True, + secure=cfg.cookie_secure, + samesite="lax", + path="/", + ) + return response + + +async def delete_session(request: Request) -> JSONResponse: + cfg: PortalConfig = request.app.state.cfg + response = JSONResponse({"ok": True}) + response.delete_cookie(cfg.session_cookie_name, path="/") + return response + + +async def me(request: Request) -> JSONResponse: + try: + record = _current_key(request) + except AuthError as exc: + return _json_error(str(exc), 401) + return JSONResponse({"me": _safe_record(record, _quota(request))}) + + +async def usage(request: Request) -> JSONResponse: + try: + record = _current_key(request) + except AuthError as exc: + return _json_error(str(exc), 401) + range_name = _parse_range(request, default="24h") + quota_state = _quota(request) + window, reset_at = quota_state.effective_window(_record_id(record), range_name) + data = await request.app.state.cpamp.analytics( + api_key_hash=record.cpamp_hash, + window=window, + include_events=False, + include_model_stats=True, + ) + data = apply_key_policy_pricing(data, record.model_prices) + quota_summary = _quota_projection( + record, + quota_state, + range_name=range_name, + window_from_ms=window.from_ms, + window_to_ms=window.to_ms, + reset_at_ms=reset_at, + used_usd=(data.get("summary") or {}).get("total_cost"), + ) + return JSONResponse({ + "range": range_name, + "from_ms": window.from_ms, + "to_ms": window.to_ms, + "reset_at_ms": reset_at, + "limits": _safe_record(record, quota_state).get("limits") or {}, + "reset_points": quota_state.get_reset_points(_record_id(record)), + "quota": quota_summary, + "summary": data.get("summary") or {}, + "timeline": data.get("timeline") or [], + "model_share": data.get("model_share") or [], + "model_stats": data.get("model_stats") or [], + "api_key_stats": safe_api_key_stats(data.get("api_key_stats") or [], expected_hash=record.cpamp_hash), + }) + + +async def events(request: Request) -> JSONResponse: + try: + record = _current_key(request) + except AuthError as exc: + return _json_error(str(exc), 401) + limit_raw = request.query_params.get("limit", "100") + try: + limit = max(1, min(int(limit_raw), 200)) + except ValueError: + limit = 100 + before_ms, before_id = _parse_before(request) + range_name = _parse_range(request, default="7d") + quota_state = _quota(request) + window, reset_at = quota_state.effective_window(_record_id(record), range_name) + data = await request.app.state.cpamp.analytics( + api_key_hash=record.cpamp_hash, + window=window, + include_events=True, + include_model_stats=True, + event_limit=limit, + before_ms=before_ms, + before_id=before_id, + ) + data = apply_key_policy_pricing(data, record.model_prices) + quota_summary = _quota_projection( + record, + quota_state, + range_name=range_name, + window_from_ms=window.from_ms, + window_to_ms=window.to_ms, + reset_at_ms=reset_at, + used_usd=(data.get("summary") or {}).get("total_cost"), + ) + page = data.get("events") or {} + items = apply_event_pricing( + safe_events(page.get("items") or [], expected_hash=record.cpamp_hash), + record.model_prices, + ) + items = _attach_event_accounting( + items, + record=record, + quota=quota_state, + selected_range=range_name, + selected_quota=quota_summary, + now_ms_value=window.to_ms, + ) + return JSONResponse({ + "range": range_name, + "from_ms": window.from_ms, + "to_ms": window.to_ms, + "reset_at_ms": reset_at, + "quota": quota_summary, + "events": items, + "next_before_ms": page.get("next_before_ms") or 0, + "next_before_id": page.get("next_before_id") or 0, + "has_more": bool(page.get("has_more")), + "total_count": page.get("total_count") or len(items), + }) + + +def _sse(name: str, payload: Any) -> bytes: + data = json.dumps(payload, ensure_ascii=False, separators=(",", ":")) + return f"event: {name}\ndata: {data}\n\n".encode("utf-8") + + +async def events_stream(request: Request) -> StreamingResponse: + try: + record = _current_key(request) + except AuthError as exc: + return StreamingResponse(iter([_sse("error", {"error": str(exc)})]), media_type="text/event-stream") + + cfg: PortalConfig = request.app.state.cfg + + async def stream(): + seen: set[str] = set() + yield _sse("ready", {"ok": True}) + while True: + if await request.is_disconnected(): + break + try: + data = await request.app.state.cpamp.analytics( + api_key_hash=record.cpamp_hash, + window=_quota(request).effective_window(_record_id(record), "24h")[0], + include_events=True, + event_limit=20, + ) + page = data.get("events") or {} + items = apply_event_pricing( + safe_events(page.get("items") or [], expected_hash=record.cpamp_hash), + record.model_prices, + ) + for item in reversed(items): + event_hash = str(item.get("event_hash") or "") + if event_hash and event_hash not in seen: + seen.add(event_hash) + yield _sse("event", item) + except Exception as exc: + yield _sse("error", {"error": type(exc).__name__}) + await asyncio.sleep(max(1.0, cfg.poll_seconds)) + + return StreamingResponse( + stream(), + media_type="text/event-stream", + headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, + ) + + +async def admin_dashboard(request: Request) -> Response: + blocked = _admin_guard(request) + if blocked is not None: + return blocked + return HTMLResponse(_ADMIN_DASHBOARD.read_text(encoding="utf-8")) + + +async def _analytics_for_quota(request: Request, record: KeyRecord, range_name: str) -> dict[str, Any]: + quota_state = _quota(request) + window, reset_at = quota_state.effective_window(_record_id(record), range_name) + data = await request.app.state.cpamp.analytics( + api_key_hash=record.cpamp_hash, + window=window, + include_events=False, + include_model_stats=True, + ) + data = apply_key_policy_pricing(data, record.model_prices) + summary = data.get("summary") or {} + return _quota_projection( + record, + quota_state, + range_name=range_name, + window_from_ms=window.from_ms, + window_to_ms=window.to_ms, + reset_at_ms=reset_at, + used_usd=summary.get("total_cost"), + ) + + +async def admin_keys(request: Request) -> JSONResponse: + blocked = _admin_guard(request) + if blocked is not None: + return blocked + state = _load_key_state(request) + quota_state = _quota(request) + keys = [] + for record in state.keys: + usage_windows: dict[str, Any] = {} + for name in RESET_WINDOWS: + try: + usage_windows[name] = await _analytics_for_quota(request, record, name) + except Exception as exc: + usage_windows[name] = {"range": name, "error": type(exc).__name__} + row = _safe_record(record, quota_state) + row["usage_windows"] = usage_windows + keys.append(row) + return JSONResponse({"keys": keys}) + + +async def admin_update_limits(request: Request) -> JSONResponse: + blocked = _admin_guard(request) + if blocked is not None: + return blocked + key_id = request.path_params["key_id"] + state = _load_key_state(request) + record = _find_record(state, key_id) + if record is None: + return _json_error("key_not_found", 404) + try: + body = await request.json() + five_hour = _parse_float_limit((body or {}).get("five_hour_usd")) + monthly = _parse_float_limit((body or {}).get("monthly_usd")) + except (json.JSONDecodeError, ValueError) as exc: + return _json_error(str(exc) or "invalid_json", 400) + quota_state = _quota(request) + quota_state.set_limits(_record_id(record), five_hour_usd=five_hour, monthly_usd=monthly, actor=_actor(request)) + return JSONResponse({"me": _safe_record(record, quota_state)}) + + +async def admin_update_limits_batch(request: Request) -> JSONResponse: + blocked = _admin_guard(request) + if blocked is not None: + return blocked + try: + body = await request.json() + except json.JSONDecodeError: + return _json_error("invalid_json", 400) + items = (body or {}).get("limits") + if not isinstance(items, list): + return _json_error("invalid_limits", 400) + + state = _load_key_state(request) + validated: list[tuple[KeyRecord, float | None, float | None]] = [] + for index, item in enumerate(items): + if not isinstance(item, dict): + return _json_error(f"invalid_limit_item:{index}", 400) + key_id = str(item.get("id") or "").strip() + record = _find_record(state, key_id) + if record is None: + return _json_error(f"key_not_found:{key_id or index}", 404) + try: + five_hour = _parse_float_limit(item.get("five_hour_usd")) + monthly = _parse_float_limit(item.get("monthly_usd")) + except ValueError as exc: + return _json_error(f"{str(exc)}:{key_id}", 400) + validated.append((record, five_hour, monthly)) + + quota_state = _quota(request) + actor = _actor(request) + for record, five_hour, monthly in validated: + quota_state.set_limits( + _record_id(record), + five_hour_usd=five_hour, + monthly_usd=monthly, + actor=actor, + ) + return JSONResponse({"ok": True, "keys": [_safe_record(record, quota_state) for record, _, _ in validated]}) + + +async def admin_reset_usage(request: Request) -> JSONResponse: + blocked = _admin_guard(request) + if blocked is not None: + return blocked + key_id = request.path_params["key_id"] + state = _load_key_state(request) + record = _find_record(state, key_id) + if record is None: + return _json_error("key_not_found", 404) + try: + body = await request.json() + except json.JSONDecodeError: + return _json_error("invalid_json", 400) + window = str((body or {}).get("window") or "all") + try: + points = _quota(request).reset(_record_id(record), window=window, actor=_actor(request)) + except ValueError as exc: + return _json_error(str(exc), 400) + return JSONResponse({"ok": True, "reset_points": points}) + + +async def admin_events(request: Request) -> JSONResponse: + blocked = _admin_guard(request) + if blocked is not None: + return blocked + key_id = request.query_params.get("key_id", "all") or "all" + state = _load_key_state(request) + limit_raw = request.query_params.get("limit", "100") + try: + limit = max(1, min(int(limit_raw), 200)) + except ValueError: + limit = 100 + range_name = _parse_range(request, default="24h") + if key_id == "all": + merged: list[dict[str, Any]] = [] + for record in state.enabled_keys(): + try: + items, _item_meta = await _events_for_record(request, record, range_name=range_name, limit=limit) + except Exception: + continue + merged.extend(items) + merged.sort(key=lambda item: int(item.get("timestamp_ms") or 0), reverse=True) + window = range_window(range_name) + meta = {"range": range_name, "from_ms": window.from_ms, "to_ms": window.to_ms, "reset_at_ms": None} + return JSONResponse({ + **meta, + "quota": None, + "key_id": "all", + "events": merged[:limit], + }) + + record = _find_record(state, key_id) + if record is None: + return _json_error("key_not_found", 404) + items, meta = await _events_for_record(request, record, range_name=range_name, limit=limit) + return JSONResponse({ + **meta, + "key_id": _record_id(record), + "events": items, + }) + + +def create_app(cfg: PortalConfig) -> Starlette: + if not cfg.session_secret: + raise ValueError("CPA_USAGE_PORTAL_SESSION_SECRET or file is required") + if not cfg.cpamp_admin_key: + raise ValueError("CPA_USAGE_PORTAL_CPAMP_ADMIN_KEY or file is required") + + @contextlib.asynccontextmanager + async def lifespan(app: Starlette): + client = httpx.AsyncClient() + app.state.cfg = cfg + app.state.client = client + app.state.cpamp = CPAMPClient(cfg.cpamp_base_url, cfg.cpamp_admin_key, client) + app.state.quota = QuotaState(cfg.local_state_db_path) + try: + yield + finally: + await client.aclose() + + routes = [ + Route("/", dashboard, methods=["GET"]), + Route("/healthz", healthz, methods=["GET"]), + Route("/api/session", create_session, methods=["POST"]), + Route("/api/session", delete_session, methods=["DELETE"]), + Route("/api/me", me, methods=["GET"]), + Route("/api/usage", usage, methods=["GET"]), + Route("/api/events", events, methods=["GET"]), + Route("/api/events/stream", events_stream, methods=["GET"]), + Route("/admin/", admin_dashboard, methods=["GET"]), + Route("/admin/api/keys", admin_keys, methods=["GET"]), + Route("/admin/api/keys/limits", admin_update_limits_batch, methods=["PUT"]), + Route("/admin/api/keys/{key_id:str}/limits", admin_update_limits, methods=["PUT"]), + Route("/admin/api/keys/{key_id:str}/reset", admin_reset_usage, methods=["POST"]), + Route("/admin/api/events", admin_events, methods=["GET"]), + ] + return Starlette(routes=routes, lifespan=lifespan) diff --git a/cpa_usage_portal/budget.py b/cpa_usage_portal/budget.py new file mode 100644 index 0000000..834a698 --- /dev/null +++ b/cpa_usage_portal/budget.py @@ -0,0 +1,42 @@ +"""Semi-automatic budget suggestion helpers.""" +from __future__ import annotations + +from dataclasses import dataclass + +from .key_policy import KeyRecord, has_model_prices + + +@dataclass(frozen=True) +class BudgetSuggestion: + enabled_key_count: int + per_key_daily_usd: float | None + per_key_weekly_usd: float | None + patches: list[dict] + + +def suggest_equal_budget( + keys: list[KeyRecord], + *, + total_daily_usd: float | None = None, + total_weekly_usd: float | None = None, + require_model_prices: bool = True, +) -> BudgetSuggestion: + enabled = [key for key in keys if key.enabled] + if not enabled: + return BudgetSuggestion(0, None, None, []) + if require_model_prices: + missing = [key.name for key in enabled if not has_model_prices(key)] + if missing: + raise ValueError("missing model prices for: " + ", ".join(missing)) + + daily = None if total_daily_usd is None else round(float(total_daily_usd) / len(enabled), 4) + weekly = None if total_weekly_usd is None else round(float(total_weekly_usd) / len(enabled), 4) + patches = [] + for key in enabled: + patch = {"key_hash": key.key_hash, "name": key.name} + if daily is not None: + patch["daily_limit_usd"] = daily + if weekly is not None: + patch["weekly_limit_usd"] = weekly + patches.append(patch) + return BudgetSuggestion(len(enabled), daily, weekly, patches) diff --git a/cpa_usage_portal/config.py b/cpa_usage_portal/config.py new file mode 100644 index 0000000..1a31bfd --- /dev/null +++ b/cpa_usage_portal/config.py @@ -0,0 +1,65 @@ +"""Environment configuration for the CPA usage portal.""" +from __future__ import annotations + +import os +from dataclasses import dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class PortalConfig: + host: str = "0.0.0.0" + port: int = 8797 + key_policy_state_path: str = "/data/cpa-key-policy-state.json" + local_state_db_path: str = "/data/portal/usage_portal.sqlite" + cpamp_base_url: str = "http://cpamp:18317" + cpamp_admin_key: str = "" + session_secret: str = "" + session_cookie_name: str = "cpa_usage_session" + session_ttl_seconds: int = 24 * 60 * 60 + cookie_secure: bool = True + poll_seconds: float = 3.0 + admin_header_name: str = "x-usage-admin" + admin_header_value: str = "1" + + +def _read_secret(value: str, file_path: str) -> str: + if value.strip(): + return value.strip() + if file_path.strip(): + return Path(file_path).read_text(encoding="utf-8").strip() + return "" + + +def _bool_env(name: str, default: bool) -> bool: + value = os.environ.get(name) + if value is None or value == "": + return default + return value.strip().lower() in {"1", "true", "yes", "on"} + + +def load_config_from_env() -> PortalConfig: + return PortalConfig( + host=os.environ.get("CPA_USAGE_PORTAL_HOST", "0.0.0.0"), + port=int(os.environ.get("CPA_USAGE_PORTAL_PORT", "8797")), + key_policy_state_path=os.environ.get( + "CPA_USAGE_PORTAL_KEY_POLICY_STATE", + "/data/cpa-key-policy-state.json", + ), + local_state_db_path=os.environ.get("CPA_USAGE_PORTAL_LOCAL_STATE_DB", "/data/portal/usage_portal.sqlite"), + cpamp_base_url=os.environ.get("CPA_USAGE_PORTAL_CPAMP_URL", "http://cpamp:18317"), + cpamp_admin_key=_read_secret( + os.environ.get("CPA_USAGE_PORTAL_CPAMP_ADMIN_KEY", ""), + os.environ.get("CPA_USAGE_PORTAL_CPAMP_ADMIN_KEY_FILE", ""), + ), + session_secret=_read_secret( + os.environ.get("CPA_USAGE_PORTAL_SESSION_SECRET", ""), + os.environ.get("CPA_USAGE_PORTAL_SESSION_SECRET_FILE", ""), + ), + session_cookie_name=os.environ.get("CPA_USAGE_PORTAL_SESSION_COOKIE", "cpa_usage_session"), + session_ttl_seconds=int(os.environ.get("CPA_USAGE_PORTAL_SESSION_TTL_SECONDS", str(24 * 60 * 60))), + cookie_secure=_bool_env("CPA_USAGE_PORTAL_COOKIE_SECURE", True), + poll_seconds=float(os.environ.get("CPA_USAGE_PORTAL_POLL_SECONDS", "3")), + admin_header_name=os.environ.get("CPA_USAGE_PORTAL_ADMIN_HEADER_NAME", "x-usage-admin"), + admin_header_value=os.environ.get("CPA_USAGE_PORTAL_ADMIN_HEADER_VALUE", "1"), + ) diff --git a/cpa_usage_portal/cpamp.py b/cpa_usage_portal/cpamp.py new file mode 100644 index 0000000..d67ab9e --- /dev/null +++ b/cpa_usage_portal/cpamp.py @@ -0,0 +1,105 @@ +"""CPAMP monitoring API client.""" +from __future__ import annotations + +import time +from dataclasses import dataclass +from datetime import datetime, timedelta, timezone +from typing import Any + +import httpx + + +@dataclass(frozen=True) +class AnalyticsWindow: + from_ms: int + to_ms: int + + +SUPPORTED_RANGES = {"5h", "24h", "7d", "month"} +_SHANGHAI_TZ = timezone(timedelta(hours=8)) + + +def now_ms() -> int: + return int(time.time() * 1000) + + +def range_window(range_name: str, *, now_ms_value: int | None = None) -> AnalyticsWindow: + current = int(now_ms_value if now_ms_value is not None else now_ms()) + if range_name == "5h": + delta = 5 * 60 * 60 * 1000 + return AnalyticsWindow(from_ms=current - delta, to_ms=current) + if range_name == "7d": + delta = 7 * 24 * 60 * 60 * 1000 + return AnalyticsWindow(from_ms=current - delta, to_ms=current) + if range_name == "month": + local_now = datetime.fromtimestamp(current / 1000, _SHANGHAI_TZ) + start = local_now.replace(day=1, hour=0, minute=0, second=0, microsecond=0) + return AnalyticsWindow(from_ms=int(start.timestamp() * 1000), to_ms=current) + else: + delta = 24 * 60 * 60 * 1000 + return AnalyticsWindow(from_ms=current - delta, to_ms=current) + + +class CPAMPClient: + def __init__(self, base_url: str, admin_key: str, client: httpx.AsyncClient) -> None: + self.base_url = base_url.rstrip("/") + self.admin_key = admin_key.strip() + self.client = client + + def _headers(self) -> dict[str, str]: + return {"Authorization": f"Bearer {self.admin_key}"} + + async def health(self) -> dict[str, Any]: + resp = await self.client.get(f"{self.base_url}/health", headers=self._headers(), timeout=5.0) + resp.raise_for_status() + data = resp.json() + return data if isinstance(data, dict) else {"ok": True} + + async def analytics( + self, + *, + api_key_hash: str, + window: AnalyticsWindow, + include_events: bool = False, + include_model_stats: bool = False, + event_limit: int = 100, + before_ms: int | None = None, + before_id: int | None = None, + ) -> dict[str, Any]: + include: dict[str, Any] = { + "summary": True, + "timeline": True, + "model_share": True, + "api_key_stats": True, + "granularity": "hour", + } + if include_model_stats: + include["model_stats"] = True + if include_events: + page: dict[str, Any] = {"limit": max(1, min(int(event_limit), 200))} + if before_ms is not None: + page["before_ms"] = int(before_ms) + if before_id is not None: + page["before_id"] = int(before_id) + include["events_page"] = page + + body = { + "from_ms": window.from_ms, + "to_ms": window.to_ms, + "now_ms": now_ms(), + "time_zone": "Asia/Shanghai", + "filters": { + "api_key_hashes": [api_key_hash], + "include_failed": True, + }, + "include": include, + } + resp = await self.client.post( + f"{self.base_url}/v0/management/monitoring/analytics", + headers=self._headers(), + json=body, + timeout=15.0, + ) + resp.raise_for_status() + data = resp.json() + return data if isinstance(data, dict) else {} diff --git a/cpa_usage_portal/key_policy.py b/cpa_usage_portal/key_policy.py new file mode 100644 index 0000000..b1e7c31 --- /dev/null +++ b/cpa_usage_portal/key_policy.py @@ -0,0 +1,338 @@ +"""Read-only projection of CPA Key Policy state.""" +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +from .security import hash_preview, key_policy_hash, normalize_key_hash + + +def _as_list(value: Any) -> list[Any]: + if isinstance(value, list): + return value + if isinstance(value, tuple): + return list(value) + if isinstance(value, str) and value.strip(): + return [item.strip() for item in value.split(",") if item.strip()] + return [] + + +def _first(raw: dict[str, Any], *names: str) -> Any: + for name in names: + if name in raw: + return raw[name] + return None + + +def _float_or_none(value: Any) -> float | None: + if value is None or value == "": + return None + try: + return float(value) + except (TypeError, ValueError): + return None + + +def _int_or_none(value: Any) -> int | None: + if value is None or value == "": + return None + try: + return int(value) + except (TypeError, ValueError): + return None + + +@dataclass(frozen=True) +class ModelPrice: + model: str + input_per_million: float = 0.0 + output_per_million: float = 0.0 + cache_read_per_million: float = 0.0 + cache_creation_per_million: float = 0.0 + target_model: str | None = None + provider: str | None = None + + def safe_dict(self) -> dict[str, Any]: + return { + "model": self.model, + "target_model": self.target_model, + "provider": self.provider, + "input_per_million": self.input_per_million, + "output_per_million": self.output_per_million, + "cache_read_per_million": self.cache_read_per_million, + "cache_creation_per_million": self.cache_creation_per_million, + } + + +@dataclass(frozen=True) +class KeyRecord: + key_hash: str + name: str + enabled: bool + policy_id: str | None = None + rpm: int | None = None + models: list[str] = field(default_factory=list) + model_prices: dict[str, ModelPrice] = field(default_factory=dict) + daily_limit_usd: float | None = None + weekly_limit_usd: float | None = None + daily_usage_usd: float | None = None + weekly_usage_usd: float | None = None + preview: str | None = None + raw: dict[str, Any] = field(default_factory=dict) + + @property + def raw_key_hash(self) -> str: + return normalize_key_hash(self.key_hash) + + @property + def cpamp_hash(self) -> str: + if self.policy_id: + # CPA Key Policy authenticates requests as key.ID. CPA Manager Plus + # then hashes that principal, not the original cpa_... key. + return hashlib.sha256(self.policy_id.strip().encode("utf-8")).hexdigest() + return self.raw_key_hash + + def safe_dict(self) -> dict[str, Any]: + return { + "id": self.policy_id or self.name or hash_preview(self.raw_key_hash), + "name": self.name, + "enabled": self.enabled, + "preview": self.preview or hash_preview(self.raw_key_hash), + "rpm": self.rpm, + "models": list(self.models), + "limits": { + "daily_usd": self.daily_limit_usd, + "weekly_usd": self.weekly_limit_usd, + }, + "usage": { + "daily_usd": self.daily_usage_usd, + "weekly_usd": self.weekly_usage_usd, + }, + "pricing": { + "priced_model_count": len(self.model_prices), + "models": [price.safe_dict() for price in self.model_prices.values()], + }, + } + + +class KeyPolicyState: + def __init__(self, keys: list[KeyRecord], *, source: Path | None = None) -> None: + self.keys = keys + self.source = source + self._by_raw_hash = {item.raw_key_hash: item for item in keys} + self._by_cpamp_hash = {item.cpamp_hash: item for item in keys} + + @classmethod + def load(cls, path: str | Path) -> "KeyPolicyState": + source = Path(path) + data = json.loads(source.read_text(encoding="utf-8")) + keys = [_parse_key(item) for item in _extract_keys(data)] + return cls([key for key in keys if key is not None], source=source) + + def get(self, key_hash: str) -> KeyRecord | None: + normalized = normalize_key_hash(key_hash) + return self._by_raw_hash.get(normalized) or self._by_cpamp_hash.get(normalized) + + def get_by_raw_hash(self, key_hash: str) -> KeyRecord | None: + return self._by_raw_hash.get(normalize_key_hash(key_hash)) + + def get_by_cpamp_hash(self, key_hash: str) -> KeyRecord | None: + return self._by_cpamp_hash.get(normalize_key_hash(key_hash)) + + def enabled_keys(self) -> list[KeyRecord]: + return [key for key in self.keys if key.enabled] + + +def _extract_keys(data: Any) -> list[dict[str, Any]]: + if isinstance(data, list): + return [item for item in data if isinstance(item, dict)] + if not isinstance(data, dict): + return [] + for path in ( + ("keys",), + ("state", "keys"), + ("data", "keys"), + ("config", "keys"), + ): + current: Any = data + for part in path: + if not isinstance(current, dict): + current = None + break + current = current.get(part) + if isinstance(current, list): + return [item for item in current if isinstance(item, dict)] + return [] + + +def _parse_key(raw: dict[str, Any]) -> KeyRecord | None: + raw_hash = _first(raw, "key_hash", "keyHash", "hash", "api_key_hash", "apiKeyHash") + if not isinstance(raw_hash, str) or not raw_hash.strip(): + return None + try: + normalized = normalize_key_hash(raw_hash) + except ValueError: + return None + + disabled = bool(_first(raw, "disabled", "is_disabled", "isDisabled") or False) + enabled_raw = _first(raw, "enabled", "is_enabled", "isEnabled") + enabled = bool(enabled_raw) if enabled_raw is not None else not disabled + model_items = _as_list(_first(raw, "models", "allowed_models", "allowedModels", "model_allowlist", "modelAllowlist", "aliases")) + models = _parse_models(model_items) + model_prices = _parse_model_prices(raw, model_items) + name = str(_first(raw, "name", "label", "alias", "description") or "").strip() + if not name: + name = hash_preview(normalized) + raw_id = _first(raw, "id", "key_id", "keyId") + policy_id = str(raw_id).strip() if raw_id is not None else None + + return KeyRecord( + key_hash=key_policy_hash(normalized), + name=name, + enabled=enabled and not disabled, + policy_id=policy_id or None, + rpm=_int_or_none(_first(raw, "rpm", "rpm_limit", "rpmLimit", "rpm_per_minute", "rpmPerMinute")), + models=models, + model_prices=model_prices, + daily_limit_usd=_float_or_none(_first( + raw, + "daily_limit_usd", + "dailyLimitUsd", + "daily_limit", + "dailyLimit", + "daily_usd", + "dailyUsd", + "daily_quota_usd", + "dailyQuotaUsd", + "daily_spend_limit_usd", + "dailySpendLimitUsd", + )), + weekly_limit_usd=_float_or_none(_first( + raw, + "weekly_limit_usd", + "weeklyLimitUsd", + "weekly_limit", + "weeklyLimit", + "weekly_usd", + "weeklyUsd", + "weekly_quota_usd", + "weeklyQuotaUsd", + "weekly_spend_limit_usd", + "weeklySpendLimitUsd", + )), + daily_usage_usd=_float_or_none(_first(raw, "daily_usage_usd", "dailyUsageUsd", "daily_usage", "dailyUsage")), + weekly_usage_usd=_float_or_none(_first(raw, "weekly_usage_usd", "weeklyUsageUsd", "weekly_usage", "weeklyUsage")), + preview=str(_first(raw, "preview", "key_preview", "keyPreview") or "").strip() or None, + raw=dict(raw), + ) + + +def has_model_prices(record: KeyRecord) -> bool: + return bool(record.model_prices) + + +def _parse_models(items: list[Any]) -> list[str]: + values: list[str] = [] + for item in items: + if isinstance(item, dict): + name = _model_name(item) + else: + name = str(item or "").strip() + if name and name not in values: + values.append(name) + return values + + +def _model_name(raw: dict[str, Any]) -> str: + return str(_first( + raw, + "alias", + "model", + "name", + "id", + "target_model", + "targetModel", + "upstream_model", + "upstreamModel", + ) or "").strip() + + +def _parse_model_prices(raw: dict[str, Any], model_items: list[Any]) -> dict[str, ModelPrice]: + prices: dict[str, ModelPrice] = {} + + for item in model_items: + if isinstance(item, dict): + price = _parse_price_entry(item) + if price is not None: + prices[price.model] = price + + raw_prices = _first(raw, "model_prices", "modelPrices", "prices") + if isinstance(raw_prices, dict): + for name, item in raw_prices.items(): + if isinstance(item, dict): + price = _parse_price_entry(item, default_model=str(name)) + else: + price = _parse_price_entry({"input": item}, default_model=str(name)) + if price is not None: + prices[price.model] = price + elif isinstance(raw_prices, list): + for item in raw_prices: + if isinstance(item, dict): + price = _parse_price_entry(item) + if price is not None: + prices[price.model] = price + + return prices + + +def _parse_price_entry(raw: dict[str, Any], *, default_model: str | None = None) -> ModelPrice | None: + model = _model_name(raw) or str(default_model or "").strip() + if not model: + return None + input_price = _price(raw, "input_price_per_million", "inputPricePerMillion", "input", "prompt", "prompt_price_per_million") + output_price = _price(raw, "output_price_per_million", "outputPricePerMillion", "output", "completion", "completion_price_per_million") + cache_read_price = _price( + raw, + "cache_read_price_per_million", + "cacheReadPricePerMillion", + "cache_price_per_million", + "cachePricePerMillion", + "cache_read", + "cacheRead", + "cache", + "input_cache_read", + "inputCacheRead", + ) + cache_creation_price = _price( + raw, + "cache_creation_price_per_million", + "cacheCreationPricePerMillion", + "cache_write_price_per_million", + "cacheWritePricePerMillion", + "cache_creation", + "cacheCreation", + "cache_write", + "cacheWrite", + "input_cache_write", + "inputCacheWrite", + ) + if not any((value or 0) > 0 for value in (input_price, output_price, cache_read_price, cache_creation_price)): + return None + target_model = str(_first(raw, "target_model", "targetModel", "upstream_model", "upstreamModel") or "").strip() or None + provider = str(_first(raw, "provider", "type") or "").strip() or None + return ModelPrice( + model=model, + input_per_million=input_price or 0.0, + output_per_million=output_price or 0.0, + cache_read_per_million=cache_read_price or 0.0, + cache_creation_per_million=cache_creation_price or 0.0, + target_model=target_model, + provider=provider, + ) + + +def _price(raw: dict[str, Any], *names: str) -> float | None: + return _float_or_none(_first(raw, *names)) diff --git a/cpa_usage_portal/pricing.py b/cpa_usage_portal/pricing.py new file mode 100644 index 0000000..2f7a7a6 --- /dev/null +++ b/cpa_usage_portal/pricing.py @@ -0,0 +1,307 @@ +"""Per-key local cost projection using CPA Key Policy prices.""" +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + +from .key_policy import ModelPrice + +PER_MILLION = 1_000_000.0 + + +@dataclass(frozen=True) +class ModelTokens: + input_tokens: int = 0 + output_tokens: int = 0 + cached_tokens: int = 0 + cache_tokens: int = 0 + cache_read_tokens: int = 0 + cache_creation_tokens: int = 0 + + +@dataclass(frozen=True) +class CacheProjection: + compatible_cached_tokens: int = 0 + raw_cached_tokens: int = 0 + raw_cache_tokens: int = 0 + cache_read_tokens: int = 0 + cache_creation_tokens: int = 0 + cache_hit_tokens: int = 0 + cache_input_side_tokens: int = 0 + cache_hit_rate: float = 0.0 + semantics: str = "cpamp_compatible_cached_tokens" + + +def _number(value: Any) -> float: + try: + return float(value or 0) + except (TypeError, ValueError): + return 0.0 + + +def _int(value: Any) -> int: + try: + return int(value or 0) + except (TypeError, ValueError): + return 0 + + +def tokens_from_row(row: dict[str, Any]) -> ModelTokens: + return ModelTokens( + input_tokens=_int(row.get("input_tokens") or row.get("inputTokens")), + output_tokens=_int(row.get("output_tokens") or row.get("outputTokens")), + cached_tokens=_int(row.get("cached_tokens") or row.get("cachedTokens")), + cache_tokens=_int(row.get("cache_tokens") or row.get("cacheTokens")), + cache_read_tokens=_int(row.get("cache_read_tokens") or row.get("cacheReadTokens")), + cache_creation_tokens=_int(row.get("cache_creation_tokens") or row.get("cacheCreationTokens")), + ) + + +def cache_projection_for_tokens(tokens: ModelTokens) -> CacheProjection: + input_tokens = max(tokens.input_tokens, 0) + raw_cached_tokens = max(tokens.cached_tokens, 0) + raw_cache_tokens = max(tokens.cache_tokens, 0) + cache_read_tokens = max(tokens.cache_read_tokens, 0) + cache_creation_tokens = max(tokens.cache_creation_tokens, 0) + if raw_cache_tokens > 0: + cached_base = max(raw_cached_tokens, raw_cache_tokens) + compatible_cached_tokens = max(cached_base - cache_read_tokens - cache_creation_tokens, 0) + semantics = "raw_cache_tokens_normalized_to_cpamp" + else: + # CPAMP Management API already projects cached_tokens with its + # compatibility expression, so do not subtract fine-grained fields again. + compatible_cached_tokens = raw_cached_tokens + semantics = "cpamp_compatible_cached_tokens" + cache_hit_tokens = compatible_cached_tokens + cache_read_tokens + cache_input_side_tokens = max(input_tokens, compatible_cached_tokens) + cache_read_tokens + cache_creation_tokens + cache_hit_rate = 0.0 + if cache_input_side_tokens > 0: + cache_hit_rate = min(max(cache_hit_tokens / cache_input_side_tokens, 0.0), 1.0) + return CacheProjection( + compatible_cached_tokens=compatible_cached_tokens, + raw_cached_tokens=raw_cached_tokens, + raw_cache_tokens=raw_cache_tokens, + cache_read_tokens=cache_read_tokens, + cache_creation_tokens=cache_creation_tokens, + cache_hit_tokens=cache_hit_tokens, + cache_input_side_tokens=cache_input_side_tokens, + cache_hit_rate=cache_hit_rate, + semantics=semantics, + ) + + +def price_for_model( + price_book: dict[str, ModelPrice], + *model_names: Any, +) -> tuple[str, ModelPrice] | None: + candidates = [str(item or "").strip() for item in model_names if str(item or "").strip()] + for candidate in candidates: + if candidate in price_book: + return candidate, price_book[candidate] + lower_index = {name.lower(): (name, price) for name, price in price_book.items()} + for candidate in candidates: + found = lower_index.get(candidate.lower()) + if found is not None: + return found + return None + + +def cost_for_tokens(price: ModelPrice, tokens: ModelTokens, *, model: str = "", service_tier: str = "") -> float: + return cost_breakdown_for_tokens(price, tokens, model=model, service_tier=service_tier)["costs"]["total"] + + +def cost_breakdown_for_tokens( + price: ModelPrice, + tokens: ModelTokens, + *, + model: str = "", + service_tier: str = "", + reasoning_tokens: Any = 0, +) -> dict[str, Any]: + input_tokens = max(tokens.input_tokens, 0) + output_tokens = max(tokens.output_tokens, 0) + cache_projection = cache_projection_for_tokens(tokens) + cached_tokens = cache_projection.compatible_cached_tokens + cache_read_tokens = cache_projection.cache_read_tokens + cache_creation_tokens = cache_projection.cache_creation_tokens + reasoning = max(_int(reasoning_tokens), 0) + prompt_tokens = max(input_tokens - cached_tokens, 0) + cache_read_price = price.cache_read_per_million or price.input_per_million + cache_creation_price = price.cache_creation_per_million or price.input_per_million + input_cost = prompt_tokens * price.input_per_million / PER_MILLION + cached_cost = cached_tokens * cache_read_price / PER_MILLION + cache_read_cost = cache_read_tokens * cache_read_price / PER_MILLION + cache_creation_cost = cache_creation_tokens * cache_creation_price / PER_MILLION + output_cost = output_tokens * price.output_per_million / PER_MILLION + subtotal = input_cost + cached_cost + cache_read_cost + cache_creation_cost + output_cost + multiplier = service_tier_multiplier(model or price.model, service_tier) + return { + "source": "key_policy", + "price_model": model or price.model, + "unit": "usd_per_1m_tokens", + "service_tier": service_tier or "", + "service_tier_multiplier": multiplier, + "prices": { + "input_per_million": price.input_per_million, + "output_per_million": price.output_per_million, + "cache_read_per_million": cache_read_price, + "cache_creation_per_million": cache_creation_price, + }, + "tokens": { + "input": input_tokens, + "cached_input": cached_tokens, + "cpamp_cached_input": cached_tokens, + "raw_cached_input": cache_projection.raw_cached_tokens, + "raw_cache_tokens": cache_projection.raw_cache_tokens, + "billable_uncached_input": prompt_tokens, + "cache_read": cache_read_tokens, + "cache_creation": cache_creation_tokens, + "fine_grained_cache_read": cache_read_tokens, + "fine_grained_cache_creation": cache_creation_tokens, + "cache_hit_input": cache_projection.cache_hit_tokens, + "effective_cache_read_for_hit_rate": cache_projection.cache_hit_tokens, + "cache_input_side": cache_projection.cache_input_side_tokens, + "cache_hit_rate": cache_projection.cache_hit_rate, + "cache_semantics": cache_projection.semantics, + "total_cache_activity": cached_tokens + cache_read_tokens + cache_creation_tokens, + "output": output_tokens, + "reasoning": reasoning, + "visible_output_estimate": max(output_tokens - reasoning, 0), + "total": max(_int(input_tokens + output_tokens), 0), + }, + "costs": { + "input": input_cost * multiplier, + "cached_input": cached_cost * multiplier, + "cache_read": cache_read_cost * multiplier, + "cache_creation": cache_creation_cost * multiplier, + "cache_total": (cached_cost + cache_read_cost + cache_creation_cost) * multiplier, + "output": output_cost * multiplier, + "subtotal": subtotal, + "total": subtotal * multiplier, + }, + } + + +def cost_for_row( + row: dict[str, Any], + price_book: dict[str, ModelPrice], + *, + model_fields: tuple[str, ...] = ("model", "resolved_model", "requested_model"), +) -> tuple[float, str] | None: + breakdown = cost_breakdown_for_row(row, price_book, model_fields=model_fields) + if breakdown is None: + return None + return _number((breakdown.get("costs") or {}).get("total")), str(breakdown.get("price_model") or "") + + +def cost_breakdown_for_row( + row: dict[str, Any], + price_book: dict[str, ModelPrice], + *, + model_fields: tuple[str, ...] = ("model", "resolved_model", "requested_model"), +) -> dict[str, Any] | None: + matched = price_for_model(price_book, *(row.get(field) for field in model_fields)) + if matched is None: + return None + model, price = matched + breakdown = cost_breakdown_for_tokens( + price, + tokens_from_row(row), + model=model, + service_tier=str(row.get("service_tier") or ""), + reasoning_tokens=row.get("reasoning_tokens"), + ) + explicit_total = _int(row.get("total_tokens") or row.get("totalTokens")) + if explicit_total: + breakdown = { + **breakdown, + "tokens": {**breakdown["tokens"], "total": explicit_total}, + } + return breakdown + + +def service_tier_multiplier(model_name: str, service_tier: str) -> float: + tier = str(service_tier or "").strip().lower() + if tier not in {"priority", "fast"}: + return 1.0 + model = str(model_name or "").strip().lower() + if _is_family(model, "gpt-5.5"): + return 2.5 + if _is_family(model, "gpt-5.4-mini"): + return 2.0 + if _is_family(model, "gpt-5.4"): + return 2.0 + if _is_family(model, "gpt-5.3-codex"): + return 2.0 + return 1.0 + + +def _is_family(model: str, family: str) -> bool: + return model == family or model.startswith(family + "-") + + +def apply_key_policy_pricing(data: dict[str, Any], price_book: dict[str, ModelPrice]) -> dict[str, Any]: + if not price_book: + return data + projected = dict(data) + model_stats = [dict(row) for row in projected.get("model_stats") or [] if isinstance(row, dict)] + cost_by_model: dict[str, float] = {} + unpriced: set[str] = set() + for row in model_stats: + model = str(row.get("model") or "").strip() + priced = cost_for_row(row, price_book, model_fields=("model",)) + if priced is None: + if model: + unpriced.add(model) + continue + cost, matched_model = priced + row["cost"] = cost + row["cost_source"] = "key_policy" + cost_by_model[model or matched_model] = cost_by_model.get(model or matched_model, 0.0) + cost + + model_share = [dict(row) for row in projected.get("model_share") or [] if isinstance(row, dict)] + for row in model_share: + model = str(row.get("model") or "").strip() + if model in cost_by_model: + row["cost"] = cost_by_model[model] + row["cost_source"] = "key_policy" + continue + priced = cost_for_row(row, price_book, model_fields=("model",)) + if priced is None: + if model: + unpriced.add(model) + continue + cost, _ = priced + row["cost"] = cost + row["cost_source"] = "key_policy" + cost_by_model[model] = cost_by_model.get(model, 0.0) + cost + + if model_stats: + total_cost = sum(cost_by_model.values()) + else: + total_cost = sum(_number(row.get("cost")) for row in model_share) + summary = dict(projected.get("summary") or {}) + summary["total_cost"] = total_cost + summary["cost_source"] = "key_policy" + summary["unpriced_models"] = sorted(unpriced) + + projected["summary"] = summary + projected["model_share"] = model_share + projected["model_stats"] = model_stats + return projected + + +def apply_event_pricing(events: list[dict[str, Any]], price_book: dict[str, ModelPrice]) -> list[dict[str, Any]]: + if not price_book: + return events + projected: list[dict[str, Any]] = [] + for event in events: + row = dict(event) + breakdown = cost_breakdown_for_row(row, price_book, model_fields=("model", "requested_model")) + if breakdown is not None: + row["cost"] = _number((breakdown.get("costs") or {}).get("total")) + row["cost_source"] = "key_policy" + row["price_model"] = breakdown.get("price_model") or "" + row["cost_breakdown"] = breakdown + projected.append(row) + return projected diff --git a/cpa_usage_portal/quota_state.py b/cpa_usage_portal/quota_state.py new file mode 100644 index 0000000..866b196 --- /dev/null +++ b/cpa_usage_portal/quota_state.py @@ -0,0 +1,241 @@ +"""Local quota metadata and soft-reset watermarks for the usage portal.""" +from __future__ import annotations + +import json +import sqlite3 +from contextlib import contextmanager +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from .cpamp import AnalyticsWindow, SUPPORTED_RANGES, now_ms, range_window + +RESET_WINDOWS = ("5h", "24h", "7d", "month") + + +@dataclass(frozen=True) +class LocalLimits: + five_hour_usd: float | None = None + monthly_usd: float | None = None + updated_at_ms: int | None = None + + def safe_dict(self) -> dict[str, Any]: + return { + "five_hour_usd": self.five_hour_usd, + "monthly_usd": self.monthly_usd, + "updated_at_ms": self.updated_at_ms, + } + + +class QuotaState: + """Small SQLite-backed state owned by the custom portal. + + The database stores only operator metadata. It never stores raw API keys, + request/response bodies, OAuth tokens, or CPAMP management secrets. + """ + + def __init__(self, path: str | Path) -> None: + self.path = Path(path) + self.path.parent.mkdir(parents=True, exist_ok=True) + self._ensure_schema() + + def _connect(self) -> sqlite3.Connection: + conn = sqlite3.connect(str(self.path), timeout=10) + conn.row_factory = sqlite3.Row + return conn + + @contextmanager + def _connection(self): + conn = self._connect() + try: + yield conn + conn.commit() + finally: + conn.close() + + def _ensure_schema(self) -> None: + with self._connection() as conn: + conn.execute("pragma journal_mode=wal") + conn.execute( + """ + create table if not exists key_limits ( + policy_id text primary key, + five_hour_limit_usd real, + monthly_limit_usd real, + updated_at_ms integer not null + ) + """ + ) + conn.execute( + """ + create table if not exists reset_watermarks ( + policy_id text not null, + window text not null, + reset_at_ms integer not null, + updated_at_ms integer not null, + primary key (policy_id, window) + ) + """ + ) + conn.execute( + """ + create table if not exists audit_log ( + id integer primary key autoincrement, + timestamp_ms integer not null, + actor text, + action text not null, + policy_id text not null, + window text, + before_json text, + after_json text + ) + """ + ) + + def get_limits(self, policy_id: str) -> LocalLimits: + with self._connection() as conn: + row = conn.execute( + "select five_hour_limit_usd, monthly_limit_usd, updated_at_ms from key_limits where policy_id = ?", + (policy_id,), + ).fetchone() + if row is None: + return LocalLimits() + return LocalLimits( + five_hour_usd=_float_or_none(row["five_hour_limit_usd"]), + monthly_usd=_float_or_none(row["monthly_limit_usd"]), + updated_at_ms=_int_or_none(row["updated_at_ms"]), + ) + + def set_limits( + self, + policy_id: str, + *, + five_hour_usd: float | None, + monthly_usd: float | None, + actor: str = "admin", + ) -> LocalLimits: + before = self.get_limits(policy_id).safe_dict() + updated = now_ms() + with self._connection() as conn: + conn.execute( + """ + insert into key_limits(policy_id, five_hour_limit_usd, monthly_limit_usd, updated_at_ms) + values (?, ?, ?, ?) + on conflict(policy_id) do update set + five_hour_limit_usd = excluded.five_hour_limit_usd, + monthly_limit_usd = excluded.monthly_limit_usd, + updated_at_ms = excluded.updated_at_ms + """, + (policy_id, five_hour_usd, monthly_usd, updated), + ) + after = LocalLimits(five_hour_usd=five_hour_usd, monthly_usd=monthly_usd, updated_at_ms=updated).safe_dict() + self._insert_audit( + conn, + actor=actor, + action="set_limits", + policy_id=policy_id, + window=None, + before=before, + after=after, + ) + return LocalLimits(five_hour_usd=five_hour_usd, monthly_usd=monthly_usd, updated_at_ms=updated) + + def get_reset_points(self, policy_id: str) -> dict[str, int | None]: + points: dict[str, int | None] = {name: None for name in RESET_WINDOWS} + with self._connection() as conn: + rows = conn.execute( + "select window, reset_at_ms from reset_watermarks where policy_id = ?", + (policy_id,), + ).fetchall() + for row in rows: + window = str(row["window"] or "") + if window in points: + points[window] = _int_or_none(row["reset_at_ms"]) + return points + + def reset(self, policy_id: str, *, window: str, actor: str = "admin", reset_at_ms: int | None = None) -> dict[str, int | None]: + windows = list(RESET_WINDOWS) if window == "all" else [window] + invalid = [item for item in windows if item not in RESET_WINDOWS] + if invalid: + raise ValueError("invalid_reset_window") + reset_at = int(reset_at_ms if reset_at_ms is not None else now_ms()) + before = self.get_reset_points(policy_id) + with self._connection() as conn: + for item in windows: + conn.execute( + """ + insert into reset_watermarks(policy_id, window, reset_at_ms, updated_at_ms) + values (?, ?, ?, ?) + on conflict(policy_id, window) do update set + reset_at_ms = excluded.reset_at_ms, + updated_at_ms = excluded.updated_at_ms + """, + (policy_id, item, reset_at, reset_at), + ) + after = dict(before) + for item in windows: + after[item] = reset_at + self._insert_audit( + conn, + actor=actor, + action="reset_usage", + policy_id=policy_id, + window=window, + before=before, + after=after, + ) + return self.get_reset_points(policy_id) + + def effective_window(self, policy_id: str, range_name: str, *, now_ms_value: int | None = None) -> tuple[AnalyticsWindow, int | None]: + if range_name not in SUPPORTED_RANGES: + range_name = "24h" + base = range_window(range_name, now_ms_value=now_ms_value) + reset_at = self.get_reset_points(policy_id).get(range_name) + if reset_at is None or reset_at <= base.from_ms: + return base, reset_at + return AnalyticsWindow(from_ms=min(reset_at, base.to_ms), to_ms=base.to_ms), reset_at + + def _insert_audit( + self, + conn: sqlite3.Connection, + *, + actor: str, + action: str, + policy_id: str, + window: str | None, + before: dict[str, Any], + after: dict[str, Any], + ) -> None: + conn.execute( + """ + insert into audit_log(timestamp_ms, actor, action, policy_id, window, before_json, after_json) + values (?, ?, ?, ?, ?, ?, ?) + """, + ( + now_ms(), + actor[:160], + action, + policy_id, + window, + json.dumps(before, ensure_ascii=False, sort_keys=True), + json.dumps(after, ensure_ascii=False, sort_keys=True), + ), + ) + + +def _float_or_none(value: Any) -> float | None: + if value is None or value == "": + return None + try: + return float(value) + except (TypeError, ValueError): + return None + + +def _int_or_none(value: Any) -> int | None: + if value is None or value == "": + return None + try: + return int(value) + except (TypeError, ValueError): + return None diff --git a/cpa_usage_portal/redaction.py b/cpa_usage_portal/redaction.py new file mode 100644 index 0000000..924b40d --- /dev/null +++ b/cpa_usage_portal/redaction.py @@ -0,0 +1,213 @@ +"""Safe projections for user-facing usage data.""" +from __future__ import annotations + +import json +import re +from typing import Any + +from .security import hash_preview, normalize_key_hash + +SECRET_KEYWORDS = ( + "authorization", + "api_key", + "apikey", + "access_token", + "refresh_token", + "id_token", + "cookie", + "set-cookie", + "oauth", + "secret", + "encrypted_content", + "management_key", + "cpamp", +) + +_BEARER_RE = re.compile(r"(?i)\bBearer\s+[A-Za-z0-9._~+/=-]+") +_KEY_RE = re.compile( + r"(?i)\b(api[_-]?key|access[_-]?token|refresh[_-]?token|authorization|cookie|secret)" + r"\s*[:=]\s*['\"]?[^'\"\s,;]+" +) +_SPACE_RE = re.compile(r"\s+") +_HEADER_BLOB_MARKERS = ( + "cf-cache-status", + "set-cookie", + "strict-transport-security", + "cross-origin-opener-policy", + "x-codex-", + "x-openai-", + "report-to", +) +MAX_FAILURE_BRIEF = 120 +MAX_FAILURE_DETAIL = 600 + + +def redact(value: Any, *, key: str = "") -> Any: + key_l = key.lower() + if any(part in key_l for part in SECRET_KEYWORDS): + return "[REDACTED]" + if isinstance(value, str): + text = _BEARER_RE.sub("Bearer [REDACTED]", value) + return _KEY_RE.sub(lambda m: f"{m.group(1)}=[REDACTED]", text) + if isinstance(value, dict): + return {str(k): redact(v, key=str(k)) for k, v in value.items()} + if isinstance(value, list): + return [redact(item) for item in value] + if isinstance(value, (int, float, bool)) or value is None: + return value + return str(value) + + +def _as_text(value: Any) -> str: + if value is None or value == "": + return "" + redacted = redact(value) + if isinstance(redacted, str): + return redacted + try: + return json.dumps(redacted, ensure_ascii=False, sort_keys=True) + except TypeError: + return str(redacted) + + +def _compact(value: str) -> str: + return _SPACE_RE.sub(" ", value).strip() + + +def _truncate(value: str, limit: int) -> str: + text = _compact(value) + if len(text) <= limit: + return text + if limit <= 3: + return "." * max(0, limit) + return text[: max(0, limit - 3)].rstrip() + "..." + + +def _looks_like_response_headers(value: str) -> bool: + lower = value.lower() + return sum(1 for marker in _HEADER_BLOB_MARKERS if marker in lower) >= 2 + + +def _failure_projection(raw: Any, *, failed: bool, status_code: Any) -> tuple[str, str]: + text = _as_text(raw) + if not text: + if failed and status_code: + return f"HTTP {status_code}", f"HTTP {status_code}" + return "", "" + if _looks_like_response_headers(text): + if not failed: + return "", "" + text = "Upstream response headers omitted; inspect status code and quota fields." + detail = _truncate(text, MAX_FAILURE_DETAIL) + brief_source = detail.split("|", 1)[0].split("\n", 1)[0] + brief = _truncate(brief_source, MAX_FAILURE_BRIEF) + if not brief and failed and status_code: + brief = f"HTTP {status_code}" + return brief, detail + + +def safe_event(event: dict[str, Any], *, expected_hash: str) -> dict[str, Any] | None: + api_key_hash = str(event.get("api_key_hash") or "").strip() + try: + normalized = normalize_key_hash(api_key_hash) + expected = normalize_key_hash(expected_hash) + except ValueError: + return None + if normalized != expected: + return None + + status_code = event.get("fail_status_code") + failed = bool(event.get("failed")) + failure_brief, failure_detail = _failure_projection( + event.get("fail_summary") or "", + failed=failed, + status_code=status_code, + ) + return { + "request_id": event.get("request_id") or "", + "event_hash": event.get("event_hash") or "", + "timestamp_ms": event.get("timestamp_ms") or 0, + "model": event.get("resolved_model") or event.get("model") or "", + "requested_model": event.get("model") or "", + "endpoint": event.get("endpoint") or event.get("path") or "", + "status": "failed" if failed else "success", + "failed": failed, + "status_code": status_code, + "latency_ms": event.get("latency_ms"), + "ttft_ms": event.get("ttft_ms"), + "input_tokens": event.get("input_tokens") or 0, + "output_tokens": event.get("output_tokens") or 0, + "cached_tokens": event.get("cached_tokens") or 0, + "cache_read_tokens": event.get("cache_read_tokens") or 0, + "cache_creation_tokens": event.get("cache_creation_tokens") or 0, + "reasoning_tokens": event.get("reasoning_tokens") or 0, + "total_tokens": event.get("total_tokens") or 0, + "cost": event.get("cost") or 0, + "service_tier": event.get("service_tier") or "", + "reasoning_effort": event.get("reasoning_effort") or "", + "api_key_preview": hash_preview(expected), + "failure_brief": failure_brief, + "failure": failure_detail, + "quota": { + "used_percent": event.get("header_quota_used_percent"), + "recover_at_ms": event.get("header_quota_recover_at_ms"), + "plan": event.get("header_quota_plan_type") or "", + "error_kind": event.get("header_error_kind") or "", + "error_code": event.get("header_error_code") or "", + }, + } + + +def safe_events(events: list[dict[str, Any]], *, expected_hash: str) -> list[dict[str, Any]]: + projected: list[dict[str, Any]] = [] + for event in events: + if isinstance(event, dict): + safe = safe_event(event, expected_hash=expected_hash) + if safe is not None: + projected.append(safe) + return projected + + +_SAFE_STAT_FIELDS = { + "calls", + "requests", + "total_calls", + "success_calls", + "failure_calls", + "success_rate", + "input_tokens", + "output_tokens", + "cached_tokens", + "cache_read_tokens", + "cache_creation_tokens", + "reasoning_tokens", + "total_tokens", + "cost", + "total_cost", + "latency_ms", + "avg_latency_ms", + "ttft_ms", + "avg_ttft_ms", +} + + +def safe_api_key_stats(stats: list[dict[str, Any]], *, expected_hash: str) -> list[dict[str, Any]]: + try: + expected = normalize_key_hash(expected_hash) + except ValueError: + return [] + projected: list[dict[str, Any]] = [] + for stat in stats: + if not isinstance(stat, dict): + continue + row_hash = str(stat.get("api_key_hash") or "").strip() + if row_hash: + try: + if normalize_key_hash(row_hash) != expected: + continue + except ValueError: + continue + safe = {key: stat.get(key) for key in _SAFE_STAT_FIELDS if key in stat} + safe["api_key_preview"] = hash_preview(expected) + projected.append(safe) + return projected diff --git a/cpa_usage_portal/retention.py b/cpa_usage_portal/retention.py new file mode 100644 index 0000000..07357de --- /dev/null +++ b/cpa_usage_portal/retention.py @@ -0,0 +1,61 @@ +"""Small, explicit retention helper for CPAMP SQLite history.""" +from __future__ import annotations + +import sqlite3 +import time +from dataclasses import dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class RetentionResult: + deleted: int + cutoff_ms: int + batches: int + + +def _has_required_usage_table(conn: sqlite3.Connection) -> bool: + row = conn.execute( + "select name from sqlite_schema where type = 'table' and name = 'usage_events'" + ).fetchone() + if row is None: + return False + columns = {item[1] for item in conn.execute("pragma table_info(usage_events)").fetchall()} + return {"id", "timestamp_ms"}.issubset(columns) + + +def enforce_usage_retention( + db_path: str | Path, + *, + keep_days: int = 7, + batch_size: int = 500, + now_ms_value: int | None = None, +) -> RetentionResult: + keep_ms = max(1, int(keep_days)) * 24 * 60 * 60 * 1000 + cutoff_ms = (now_ms_value if now_ms_value is not None else int(time.time() * 1000)) - keep_ms + deleted = 0 + batches = 0 + conn = sqlite3.connect(str(db_path)) + try: + conn.execute("pragma busy_timeout = 5000") + if not _has_required_usage_table(conn): + return RetentionResult(0, cutoff_ms, 0) + while True: + rows = conn.execute( + "select id from usage_events where timestamp_ms < ? order by timestamp_ms limit ?", + (cutoff_ms, max(1, int(batch_size))), + ).fetchall() + if not rows: + break + ids = [row[0] for row in rows] + placeholders = ",".join("?" for _ in ids) + conn.execute(f"delete from usage_events where id in ({placeholders})", ids) + conn.commit() + deleted += len(ids) + batches += 1 + if len(ids) < batch_size: + break + conn.execute("pragma wal_checkpoint(TRUNCATE)").fetchall() + finally: + conn.close() + return RetentionResult(deleted, cutoff_ms, batches) diff --git a/cpa_usage_portal/security.py b/cpa_usage_portal/security.py new file mode 100644 index 0000000..656ba03 --- /dev/null +++ b/cpa_usage_portal/security.py @@ -0,0 +1,81 @@ +"""Hashing and signed-session helpers for the CPA usage portal.""" +from __future__ import annotations + +import base64 +import hashlib +import hmac +import json +import time +from typing import Any + +HASH_PREFIX = "sha256:" + + +def sha256_hex(value: str) -> str: + return hashlib.sha256(value.strip().encode("utf-8")).hexdigest() + + +def normalize_key_hash(value: str) -> str: + text = (value or "").strip().lower() + if text.startswith(HASH_PREFIX): + text = text[len(HASH_PREFIX):] + if len(text) != 64 or any(ch not in "0123456789abcdef" for ch in text): + raise ValueError("invalid sha256 key hash") + return text + + +def key_policy_hash(hex_hash: str) -> str: + return f"{HASH_PREFIX}{normalize_key_hash(hex_hash)}" + + +def hash_preview(hex_hash: str) -> str: + normalized = normalize_key_hash(hex_hash) + return f"{normalized[:8]}...{normalized[-4:]}" + + +def _b64(data: bytes) -> str: + return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii") + + +def _unb64(data: str) -> bytes: + pad = "=" * (-len(data) % 4) + return base64.urlsafe_b64decode(data + pad) + + +def sign_session(payload: dict[str, Any], secret: str, *, ttl_seconds: int) -> str: + now = int(time.time()) + safe_payload = { + **payload, + "iat": now, + "exp": now + max(1, int(ttl_seconds)), + } + encoded = _b64(json.dumps(safe_payload, sort_keys=True, separators=(",", ":")).encode("utf-8")) + signature = hmac.new(secret.encode("utf-8"), encoded.encode("ascii"), hashlib.sha256).digest() + return f"{encoded}.{_b64(signature)}" + + +def verify_session(token: str, secret: str) -> dict[str, Any] | None: + try: + encoded, signature = token.split(".", 1) + except ValueError: + return None + want = hmac.new(secret.encode("utf-8"), encoded.encode("ascii"), hashlib.sha256).digest() + try: + got = _unb64(signature) + except Exception: + return None + if not hmac.compare_digest(got, want): + return None + try: + payload = json.loads(_unb64(encoded)) + except Exception: + return None + if not isinstance(payload, dict): + return None + if int(payload.get("exp") or 0) < int(time.time()): + return None + try: + payload["key_hash"] = normalize_key_hash(str(payload.get("key_hash") or "")) + except ValueError: + return None + return payload diff --git a/cpa_usage_portal/static/admin.html b/cpa_usage_portal/static/admin.html new file mode 100644 index 0000000..1471496 --- /dev/null +++ b/cpa_usage_portal/static/admin.html @@ -0,0 +1,617 @@ + + + + + + CPA 用量管理 + + + +
+
+
+
U
+
+

CPA 用量管理

+
自有门户的限额、估算用量与软清零；不修改 CPAMP 原始请求记录。
+
+
+
+ 等待刷新 + + +
+
+ +
+
+
Key 数量
+
0
+
启用 0
+
+
+
所选窗口总费用
+
$0.0000
+
-
+
+
+
超限风险
+
0
+
按门户估算，不是官方账单
+
+
+
清零方式
+
Soft
+
只写 reset watermark
+
+
+ +
+
+
+

Key 限额与窗口用量

+

日限/周限来自 Key Policy；5H/月限和清零点保存在自有 SQLite。

+
+
+ 无改动 + +
+
+
+ + + + + + + + + + + + + + +
Key状态5H日周月本地限额操作
正在读取...
+
+
+ +
+
+
+

最近请求

+

默认显示全部 Key，可按单个用户/Key 排障。

+
+ +
+
+ + + + + + + + + + + + + + + +
时间用户/Key状态模型延迟TokensReasoning费用详情
暂无请求记录
+
+
+
+ + + + diff --git a/cpa_usage_portal/static/dashboard.html b/cpa_usage_portal/static/dashboard.html new file mode 100644 index 0000000..e396219 --- /dev/null +++ b/cpa_usage_portal/static/dashboard.html @@ -0,0 +1,945 @@ + + + + + + CPA 用量自助页 + + + +
+
+
+
C
+
+

CPA 用量自助页

+
使用 Key Policy 的 cpa_... Key 登录，只查看自己的用量。
+
+
+
+ 实时未连接 + + + +
+
+ + + + +
+ + + + diff --git a/deploy/cpa-usage-portal/Dockerfile b/deploy/cpa-usage-portal/Dockerfile new file mode 100644 index 0000000..b5b58bd --- /dev/null +++ b/deploy/cpa-usage-portal/Dockerfile @@ -0,0 +1,9 @@ +FROM python:3.12-slim + +WORKDIR /app +RUN pip install --no-cache-dir "httpx>=0.27" "starlette>=0.37" "uvicorn>=0.30" + +COPY cpa_usage_portal ./cpa_usage_portal +COPY run_usage_portal.py ./run_usage_portal.py + +CMD ["python", "run_usage_portal.py"] diff --git a/deploy/cpa-usage-portal/docker-compose.example.yaml b/deploy/cpa-usage-portal/docker-compose.example.yaml new file mode 100644 index 0000000..e9dac4f --- /dev/null +++ b/deploy/cpa-usage-portal/docker-compose.example.yaml @@ -0,0 +1,25 @@ +services: + cpa-usage-portal: + build: + context: ../.. + dockerfile: deploy/cpa-usage-portal/Dockerfile + container_name: cpa-usage-portal + restart: unless-stopped + networks: + - cpa_net + environment: + CPA_USAGE_PORTAL_KEY_POLICY_STATE: /data/plugin-state/cpa-key-policy-state.json + CPA_USAGE_PORTAL_LOCAL_STATE_DB: /data/portal/usage_portal.sqlite + CPA_USAGE_PORTAL_CPAMP_URL: http://cpamp:18317 + CPA_USAGE_PORTAL_CPAMP_ADMIN_KEY_FILE: /run/secrets/cpamp_admin_key + CPA_USAGE_PORTAL_SESSION_SECRET_FILE: /run/secrets/session_secret + CPA_USAGE_PORTAL_COOKIE_SECURE: "true" + volumes: + - /opt/codex-stacks/cpa/plugin-state:/data/plugin-state:ro + - ./data:/data/portal + - ./secrets/cpamp_admin_key:/run/secrets/cpamp_admin_key:ro + - ./secrets/session_secret:/run/secrets/session_secret:ro + +networks: + cpa_net: + external: true diff --git a/middleware/admin.py b/middleware/admin.py new file mode 100644 index 0000000..cbcc170 --- /dev/null +++ b/middleware/admin.py @@ -0,0 +1,160 @@ +"""Read-only admin routes for the CodexCont dashboard.""" +from __future__ import annotations + +import asyncio +import json +from pathlib import Path +from typing import Any +from urllib.parse import urlsplit + +import httpx +from starlette.requests import Request +from starlette.responses import HTMLResponse, JSONResponse, RedirectResponse, StreamingResponse + +from .config import Config + +_DASHBOARD_HTML = Path(__file__).with_name("dashboard.html") + + +def _admin_diag(request: Request): + return request.app.state.diagnostics + + +def _safe_config(cfg: Config, diagnostics_max_events: int) -> dict[str, Any]: + parsed = urlsplit(cfg.upstream.url) + return { + "listen_paths": list(cfg.server.listen_paths), + "upstream_mode": cfg.upstream.mode, + "upstream_host": parsed.netloc, + "upstream_path": parsed.path, + "auth_mode": cfg.auth.mode, + "continuation_enabled": cfg.cont.enabled, + "continuation_method": cfg.cont.method, + "max_continue": cfg.cont.max_continue, + "truncation_step": cfg.cont.truncation_step, + "log_retention": diagnostics_max_events, + "key_identity_configured": bool(cfg.admin.key_policy_state_path), + } + + +def _upstream_health_url(cfg: Config) -> str | None: + parsed = urlsplit(cfg.upstream.url) + if not parsed.scheme or not parsed.netloc: + return None + return f"{parsed.scheme}://{parsed.netloc}/healthz" + + +async def _probe_upstream(client: httpx.AsyncClient, cfg: Config) -> dict[str, Any]: + url = _upstream_health_url(cfg) + if not url: + return {"ok": False, "status": "invalid_upstream_url"} + try: + resp = await client.get(url, timeout=3.0) + return { + "ok": 200 <= resp.status_code < 400, + "status": "http", + "status_code": resp.status_code, + "url": url, + } + except Exception as exc: + return { + "ok": False, + "status": "error", + "error": type(exc).__name__, + "url": url, + } + + +async def admin_healthz(request: Request) -> JSONResponse: + return JSONResponse(_admin_diag(request).health()) + + +async def admin_status(request: Request) -> JSONResponse: + cfg: Config = request.app.state.cfg + upstream = await _probe_upstream(request.app.state.client, cfg) + diag = _admin_diag(request) + return JSONResponse( + diag.snapshot(upstream=upstream, config=_safe_config(cfg, diag.max_events)) + ) + + +async def admin_logs(request: Request) -> JSONResponse: + raw_limit = request.query_params.get("limit", "200") + try: + limit = int(raw_limit) + except ValueError: + limit = 200 + diag = _admin_diag(request) + return JSONResponse({"events": diag.recent(limit=limit), "max_events": diag.max_events}) + + +async def admin_requests(request: Request) -> JSONResponse: + raw_limit = request.query_params.get("limit", "100") + try: + limit = int(raw_limit) + except ValueError: + limit = 100 + diag = _admin_diag(request) + return JSONResponse( + {"requests": diag.recent_requests(limit=limit), "max_requests": diag.max_requests} + ) + + +def _sse_event(name: str, data: Any) -> bytes: + body = json.dumps(data, ensure_ascii=False, separators=(",", ":")) + return f"event: {name}\ndata: {body}\n\n".encode("utf-8") + + +async def admin_logs_stream(request: Request) -> StreamingResponse: + diag = _admin_diag(request) + log_queue = diag.subscribe() + request_queue = diag.subscribe_requests() + once = request.query_params.get("once") == "1" + + async def events(): + try: + yield _sse_event("ready", {"ok": True}) + for item in diag.recent_requests(limit=50): + yield _sse_event("request", item) + for item in diag.recent(limit=50): + yield _sse_event("log", item) + if once: + return + while True: + if await request.is_disconnected(): + break + log_task = asyncio.create_task(log_queue.get()) + req_task = asyncio.create_task(request_queue.get()) + done, pending = await asyncio.wait( + {log_task, req_task}, + timeout=15.0, + return_when=asyncio.FIRST_COMPLETED, + ) + for task in pending: + task.cancel() + if pending: + await asyncio.gather(*pending, return_exceptions=True) + if not done: + yield b": keepalive\n\n" + continue + for task in done: + item = task.result() + event_name = "log" if task is log_task else "request" + yield _sse_event(event_name, item) + finally: + diag.unsubscribe(log_queue) + diag.unsubscribe_requests(request_queue) + + return StreamingResponse( + events(), + media_type="text/event-stream", + headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, + ) + + +async def admin_dashboard(_request: Request) -> HTMLResponse: + return HTMLResponse(_DASHBOARD_HTML.read_text(encoding="utf-8")) + + +async def admin_redirect(_request: Request) -> RedirectResponse: + return RedirectResponse(url="./admin/", status_code=307) diff --git a/middleware/app.py b/middleware/app.py index 5c0c3ac..37526ec 100644 --- a/middleware/app.py +++ b/middleware/app.py @@ -7,16 +7,28 @@ from __future__ import annotations import contextlib +import io import json import logging from typing import Any import httpx +import zstandard as zstd from starlette.applications import Starlette from starlette.requests import Request from starlette.responses import JSONResponse, Response, StreamingResponse from starlette.routing import Route +from .admin import ( + admin_dashboard, + admin_healthz, + admin_logs, + admin_logs_stream, + admin_requests, + admin_redirect, + admin_status, +) +from .engine import engine_analyze, engine_healthz from .codex import ( build_round_payload, declares_continue_tool, @@ -25,12 +37,32 @@ ) from .config import Config from .creds import build_upstream_headers, would_inject_authorization +from .diagnostics import Diagnostics +from .key_identity import KeyIdentityResolver from .proxy import fold_stream, open_passthrough, open_round from .store import IdStore log = logging.getLogger("middleware.app") +class BodyDecodeError(ValueError): + pass + + +def _decode_request_body(raw: bytes, encoding: str | None) -> bytes: + enc = (encoding or "").strip().lower() + if not enc or enc == "identity": + return raw + if enc != "zstd": + raise BodyDecodeError(f"unsupported request content-encoding: {enc}") + + try: + with zstd.ZstdDecompressor().stream_reader(io.BytesIO(raw)) as reader: + return reader.read() + except zstd.ZstdError as exc: + raise BodyDecodeError("invalid zstd request body") from exc + + def _header_base(request: Request) -> str | None: """The non-blank Responses-API-Base header value, or None (case-insensitive).""" v = request.headers.get("responses-api-base") @@ -72,17 +104,36 @@ def _url_is_from_header(cfg: Config, request: Request) -> bool: async def _passthrough( - client: httpx.AsyncClient, cfg: Config, request: Request, raw: bytes, url: str + client: httpx.AsyncClient, + cfg: Config, + request: Request, + raw: bytes, + url: str, + diagnostics: Diagnostics | None = None, + request_id: str | None = None, ): """Pure proxy: forward the raw request and stream the raw response back.""" headers = build_upstream_headers(request.headers.items(), cfg) - resp = await open_passthrough(client, url, raw, headers) + try: + resp = await open_passthrough(client, url, raw, headers) + except Exception as exc: + if diagnostics and request_id: + diagnostics.request_failed(request_id, reason="passthrough_open_error", detail=repr(exc)) + raise async def body_iter(): + failed = False try: async for chunk in resp.aiter_bytes(): yield chunk + except Exception as exc: + failed = True + if diagnostics and request_id: + diagnostics.request_failed(request_id, reason="passthrough_stream_error", detail=repr(exc)) + raise finally: + if diagnostics and request_id and not failed: + diagnostics.request_finished(request_id, status=f"passthrough:{resp.status_code}") await resp.aclose() return StreamingResponse( @@ -95,21 +146,50 @@ async def body_iter(): async def handle_responses(request: Request) -> Response: cfg: Config = request.app.state.cfg client: httpx.AsyncClient = request.app.state.client + diagnostics: Diagnostics = request.app.state.diagnostics + key_identity = request.app.state.key_identity.identify_authorization( + request.headers.get("authorization") + ) + request_id = diagnostics.request_started(path=request.url.path, key_identity=key_identity) + + wire_raw = await request.body() + try: + raw = _decode_request_body(wire_raw, request.headers.get("content-encoding")) + except BodyDecodeError as exc: + log.warning( + "request body decode failed: content-type=%s content-encoding=%s len=%d error=%s", + request.headers.get("content-type"), + request.headers.get("content-encoding"), + len(wire_raw), + exc, + ) + diagnostics.request_failed(request_id, reason="body_decode_error", detail=str(exc)) + return JSONResponse({"error": str(exc)}, status_code=400) - raw = await request.body() try: body: dict[str, Any] = json.loads(raw) except (json.JSONDecodeError, UnicodeDecodeError): + log.warning( + "invalid JSON body: content-type=%s content-encoding=%s len=%d", + request.headers.get("content-type"), + request.headers.get("content-encoding"), + len(raw), + ) + diagnostics.request_failed(request_id, reason="invalid_json_body") return JSONResponse({"error": "invalid JSON body"}, status_code=400) if not isinstance(body, dict): + diagnostics.request_failed(request_id, reason="non_object_body") return JSONResponse({"error": "body must be a JSON object"}, status_code=400) url = _resolve_upstream_url(cfg, request) if url is None: + diagnostics.request_update(request_id, model=body.get("model")) + diagnostics.request_failed(request_id, reason="missing_responses_api_base") return JSONResponse( {"error": "Responses-API-Base header is required (upstream mode=header_required)"}, status_code=400, ) + diagnostics.request_update(request_id, model=body.get("model"), upstream_url=url) # Safety: never send the proxy's configured credentials to a URL the request # itself supplied. If the base came from the header, the request must carry @@ -119,6 +199,7 @@ async def handle_responses(request: Request) -> Response: ): log.warning("blocked: Responses-API-Base override without own auth (model=%s)", body.get("model")) + diagnostics.request_failed(request_id, reason="blocked_header_override_without_own_auth") return JSONResponse( {"error": "When overriding the upstream base (Responses-API-Base), the request must " "provide its own Authorization; the proxy will not send its configured " @@ -148,10 +229,14 @@ async def handle_responses(request: Request) -> Response: else "declares-continue_thinking") log.info("passthrough (%s): model=%s path=%s url=%s", why, body.get("model"), request.url.path, url) - return await _passthrough(client, cfg, request, raw, url) + diagnostics.mark_passthrough(request_id, reason=why, model=body.get("model")) + return await _passthrough(client, cfg, request, raw, url, diagnostics, request_id) log.info("fold start: model=%s path=%s url=%s input_items=%d", body.get("model"), request.url.path, url, len(body.get("input") or [])) + diagnostics.mark_fold_start( + request_id, model=body.get("model"), path=request.url.path, upstream_url=url + ) # repair_followup="stateful": re-insert tool_pair continue pairs after recorded # ids (tool_pair only — commentary preserves cross-turn structure via forward_marker). @@ -180,12 +265,25 @@ async def handle_responses(request: Request) -> Response: if resp.status_code >= 400: err = await resp.aread() await resp.aclose() + diagnostics.request_failed( + request_id, reason="upstream_http_error", detail={"status_code": resp.status_code} + ) return Response( err, status_code=resp.status_code, media_type=resp.headers.get("content-type") ) return StreamingResponse( - fold_stream(client, cfg, body, headers, resp, request.app.state.id_store, url=url), + fold_stream( + client, + cfg, + body, + headers, + resp, + request.app.state.id_store, + url=url, + diagnostics=diagnostics, + request_id=request_id, + ), media_type="text/event-stream", ) @@ -205,6 +303,8 @@ def create_app(cfg: Config) -> Starlette: @contextlib.asynccontextmanager async def lifespan(app: Starlette): app.state.cfg = cfg + app.state.diagnostics = Diagnostics(max_events=cfg.admin.max_log_events) + app.state.key_identity = KeyIdentityResolver(cfg.admin.key_policy_state_path) app.state.client = _make_client() app.state.id_store = IdStore() try: @@ -213,6 +313,16 @@ async def lifespan(app: Starlette): await app.state.client.aclose() routes = [ + Route("/engine/healthz", engine_healthz, methods=["GET"]), + Route("/engine/v1/responses/analyze", engine_analyze, methods=["POST"]), + Route("/admin", admin_redirect, methods=["GET"]), + Route("/admin/", admin_dashboard, methods=["GET"]), + Route("/admin/healthz", admin_healthz, methods=["GET"]), + Route("/admin/status", admin_status, methods=["GET"]), + Route("/admin/requests", admin_requests, methods=["GET"]), + Route("/admin/logs", admin_logs, methods=["GET"]), + Route("/admin/logs/stream", admin_logs_stream, methods=["GET"]), + ] + [ Route(path, handle_responses, methods=["POST"]) for path in cfg.server.listen_paths ] return Starlette(routes=routes, lifespan=lifespan) diff --git a/middleware/config.py b/middleware/config.py index 57bb15b..fc4e8db 100644 --- a/middleware/config.py +++ b/middleware/config.py @@ -71,6 +71,12 @@ class LogCfg: dump_rounds_dir: str = "" +@dataclass(frozen=True) +class AdminCfg: + max_log_events: int = 800 + key_policy_state_path: str = "" + + @dataclass(frozen=True) class Config: server: ServerCfg = field(default_factory=ServerCfg) @@ -79,6 +85,7 @@ class Config: cont: ContinueCfg = field(default_factory=ContinueCfg) stream: StreamCfg = field(default_factory=StreamCfg) log: LogCfg = field(default_factory=LogCfg) + admin: AdminCfg = field(default_factory=AdminCfg) # Directory config.toml lived in (for resolving relative paths if needed). root: Path = field(default_factory=lambda: Path.cwd()) @@ -108,6 +115,7 @@ def load_config(path: str | Path) -> Config: cont = _section(data, "continue") stream = _section(data, "stream") log = _section(data, "log") + admin = _section(data, "admin") # listen_paths is a list in TOML; store as tuple. if "listen_paths" in server and isinstance(server["listen_paths"], list): @@ -125,6 +133,7 @@ def load_config(path: str | Path) -> Config: cont=ContinueCfg(**_only_known(ContinueCfg, cont)), stream=StreamCfg(**_only_known(StreamCfg, stream)), log=LogCfg(**_only_known(LogCfg, log)), + admin=AdminCfg(**_only_known(AdminCfg, admin)), root=path.resolve().parent if path.exists() else Path.cwd(), ) diff --git a/middleware/creds.py b/middleware/creds.py index 0c261ac..88e9251 100644 --- a/middleware/creds.py +++ b/middleware/creds.py @@ -5,6 +5,8 @@ its own). Two exceptions: 1. client-owned headers (Host, Content-Length, ...) are dropped so httpx sets them correctly — the body length changes when we merge `include`. + Content-Encoding is also dropped because encoded agent bodies are decoded + before the middleware inspects or forwards them. 2. credentials (Authorization, chatgpt-account-id) follow the auth mode, with the token / account id supplied directly from config.toml `[auth]`. """ @@ -25,6 +27,7 @@ "proxy-connection", "transfer-encoding", "accept-encoding", + "content-encoding", } _AUTH = "authorization" diff --git a/middleware/dashboard.html b/middleware/dashboard.html new file mode 100644 index 0000000..c7b2f37 --- /dev/null +++ b/middleware/dashboard.html @@ -0,0 +1,954 @@ + + + + + + + CodexCont 保护状态面板 + + + +
+
+
+
C
+
+

CodexCont 保护状态面板

+
正在读取运行状态
+
+
+
+ 连接中 + 活跃 0 + +
+
+ +
+
+
#总请求
+
0
+
-
+
+
+
P进入保护链
+
0
+
由 CodexCont 折叠处理
+
+
+
C自动续写
+
0
+
-
+
+
+
516疑似截断
+
0
+
516 / 518n-2 指纹
+
+
+
!失败
+
0
+
-
+
+
+ +
+
+
+

最近请求

+

保护结果、命中轮和末轮 reasoning 是主视图。

+
+
+ + + 0 条 +
+
+
+ + + + + + + + + + + + + + + + +
时间用户/Key保护结果模型轮次命中轮末轮思考量续写最终结果详情
暂无请求
+
+
+ +
+ + 高级日志 + 原始脱敏事件 + +
+
+
+ + +
+
+ + + +
+
+
+ + + + + + + + + + + +
时间级别事件请求字段
暂无日志
+
+
+
+
+ + + + diff --git a/middleware/diagnostics.py b/middleware/diagnostics.py new file mode 100644 index 0000000..1221f3b --- /dev/null +++ b/middleware/diagnostics.py @@ -0,0 +1,503 @@ +"""In-process diagnostics for the CodexCont admin dashboard.""" +from __future__ import annotations + +import asyncio +from copy import deepcopy +import re +import threading +import time +import uuid +from collections import deque +from datetime import UTC, datetime +from typing import Any + +_SECRET_KEYWORDS = ( + "authorization", + "api-key", + "api_key", + "apikey", + "access_token", + "refresh_token", + "id_token", + "tunnel_token", + "bearer_token", + "oauth", + "secret", + "encrypted_content", +) +_BEARER_RE = re.compile(r"(?i)\bBearer\s+[A-Za-z0-9._~+/=-]+") +_KEY_VALUE_RE = re.compile( + r"(?i)\b(api[_-]?key|access[_-]?token|refresh[_-]?token|authorization|secret)" + r"\s*[:=]\s*['\"]?[^'\"\s,;]+" +) + + +def utc_now_iso() -> str: + return datetime.now(UTC).isoformat().replace("+00:00", "Z") + + +def redact_value(value: Any, *, key: str = "") -> Any: + """Return a JSON-safe value with obvious secrets removed.""" + key_l = key.lower() + if key_l == "token" or key_l.endswith("_token") or any(part in key_l for part in _SECRET_KEYWORDS): + return "[REDACTED]" + if isinstance(value, str): + text = _BEARER_RE.sub("Bearer [REDACTED]", value) + return _KEY_VALUE_RE.sub(lambda m: f"{m.group(1)}=[REDACTED]", text) + if isinstance(value, dict): + return {str(k): redact_value(v, key=str(k)) for k, v in value.items()} + if isinstance(value, (list, tuple)): + return [redact_value(v) for v in value] + if isinstance(value, (int, float, bool)) or value is None: + return value + return str(value) + + +_RISK_STOP_REASONS = { + "no_encrypted_content", + "max_continue", + "max_total_output_tokens", + "tier_out_of_window", +} + + +def _public_request_summary(summary: dict[str, Any]) -> dict[str, Any]: + public = deepcopy(summary) + public.pop("_started_perf", None) + return redact_value(public) + + +class Diagnostics: + """Small memory-only metrics and event hub. + + This intentionally does not write to disk. It is safe for the small SJC VPS + and simple enough to keep CodexCont independent from CPA internals. + """ + + def __init__(self, *, max_events: int = 800, max_requests: int = 200) -> None: + self.max_events = max(1, int(max_events)) + self.max_requests = max(1, int(max_requests)) + self._events: deque[dict[str, Any]] = deque(maxlen=self.max_events) + self._subscribers: set[asyncio.Queue[dict[str, Any]]] = set() + self._request_subscribers: set[asyncio.Queue[dict[str, Any]]] = set() + self._lock = threading.RLock() + self._seq = 0 + self._started_wall = utc_now_iso() + self._started_perf = time.monotonic() + self._active_ids: set[str] = set() + self._counters: dict[str, int] = { + "total_requests": 0, + "active_requests": 0, + "folded_requests": 0, + "passthrough_requests": 0, + "continuations": 0, + "truncation_hits": 0, + "failures": 0, + } + self._last_request_at: str | None = None + self._last_continuation_at: str | None = None + self._last_error_at: str | None = None + self._last_error: dict[str, Any] | None = None + self._request_meta: dict[str, dict[str, Any]] = {} + self._request_summaries: dict[str, dict[str, Any]] = {} + + def recent(self, *, limit: int | None = None) -> list[dict[str, Any]]: + with self._lock: + events = list(self._events) + if limit is None: + return events + limit = max(0, min(int(limit), self.max_events)) + return events[-limit:] + + def subscribe(self) -> asyncio.Queue[dict[str, Any]]: + queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue(maxsize=200) + with self._lock: + self._subscribers.add(queue) + return queue + + def unsubscribe(self, queue: asyncio.Queue[dict[str, Any]]) -> None: + with self._lock: + self._subscribers.discard(queue) + + def subscribe_requests(self) -> asyncio.Queue[dict[str, Any]]: + queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue(maxsize=200) + with self._lock: + self._request_subscribers.add(queue) + return queue + + def unsubscribe_requests(self, queue: asyncio.Queue[dict[str, Any]]) -> None: + with self._lock: + self._request_subscribers.discard(queue) + + def recent_requests(self, *, limit: int | None = None) -> list[dict[str, Any]]: + with self._lock: + summaries = [_public_request_summary(item) for item in self._request_summaries.values()] + if limit is None: + return summaries + limit = max(0, min(int(limit), self.max_requests)) + return summaries[-limit:] + + def record(self, level: str, event: str, message: str = "", **fields: Any) -> dict[str, Any]: + item = { + "seq": 0, + "ts": utc_now_iso(), + "level": (level or "info").lower(), + "event": event, + "message": redact_value(message), + "fields": redact_value(fields), + } + with self._lock: + self._seq += 1 + item["seq"] = self._seq + self._events.append(item) + subscribers = list(self._subscribers) + + for queue in subscribers: + try: + queue.put_nowait(item) + except asyncio.QueueFull: + try: + queue.get_nowait() + except asyncio.QueueEmpty: + pass + try: + queue.put_nowait(item) + except asyncio.QueueFull: + pass + return item + + def _trim_requests_locked(self) -> None: + while len(self._request_summaries) > self.max_requests: + removed = False + for request_id in list(self._request_summaries.keys()): + if request_id not in self._active_ids: + self._request_summaries.pop(request_id, None) + removed = True + break + if not removed: + break + + def _request_copy_locked(self, request_id: str) -> dict[str, Any] | None: + summary = self._request_summaries.get(request_id) + if summary is None: + return None + return _public_request_summary(summary) + + def _publish_request(self, summary: dict[str, Any] | None) -> None: + if summary is None: + return + with self._lock: + subscribers = list(self._request_subscribers) + for queue in subscribers: + try: + queue.put_nowait(summary) + except asyncio.QueueFull: + try: + queue.get_nowait() + except asyncio.QueueEmpty: + pass + try: + queue.put_nowait(summary) + except asyncio.QueueFull: + pass + + def _set_request_result_locked(self, summary: dict[str, Any]) -> None: + if summary.get("status") == "failed": + summary["protection"] = "failed" + return + if summary.get("status") == "incomplete": + summary["protection"] = "incomplete" + return + if summary.get("passthrough"): + summary["protection"] = "passthrough" + return + if summary.get("continuation_count", 0) > 0: + summary["protection"] = "auto_continued" + return + if summary.get("truncation_match") or summary.get("stopped_reason") in _RISK_STOP_REASONS: + summary["protection"] = "risk_uncontinued" + return + if summary.get("folded"): + summary["protection"] = "protected_clean" + return + summary["protection"] = "processing" + + def _finish_request_locked( + self, + request_id: str, + *, + status: str, + stopped_reason: str | None = None, + failure_reason: str | None = None, + failure_detail: Any = None, + ) -> dict[str, Any] | None: + now = utc_now_iso() + summary = self._request_summaries.get(request_id) + if summary is None: + return None + summary["updated_at"] = now + summary["ended_at"] = now + started_perf = summary.pop("_started_perf", None) + if isinstance(started_perf, (int, float)): + summary["duration_ms"] = round((time.monotonic() - started_perf) * 1000) + summary["final_status"] = status + summary["stopped_reason"] = stopped_reason + if failure_reason is not None: + summary["status"] = "failed" + summary["failure_reason"] = failure_reason + summary["failure_detail"] = redact_value(failure_detail) + elif status == "incomplete" or status == "closed": + summary["status"] = "incomplete" + else: + summary["status"] = "completed" + self._set_request_result_locked(summary) + return _public_request_summary(summary) + + def request_started( + self, + *, + path: str, + model: str | None = None, + key_identity: dict[str, Any] | None = None, + ) -> str: + request_id = uuid.uuid4().hex[:12] + now = utc_now_iso() + perf = time.monotonic() + safe_identity = redact_value(key_identity or { + "known": False, + "source": "unset", + "name": "未识别 Key", + "preview": "", + }) + with self._lock: + self._active_ids.add(request_id) + self._counters["total_requests"] += 1 + self._counters["active_requests"] = len(self._active_ids) + self._last_request_at = now + self._request_meta[request_id] = { + "request_id": request_id, + "path": path, + "model": model, + "key_identity": safe_identity, + "started_at": now, + } + self._request_summaries[request_id] = { + "request_id": request_id, + "model": model, + "path": path, + "key_identity": safe_identity, + "started_at": now, + "updated_at": now, + "ended_at": None, + "duration_ms": None, + "status": "processing", + "protection": "processing", + "folded": False, + "passthrough": False, + "passthrough_reason": None, + "rounds": [], + "latest_round": None, + "latest_reasoning_tokens": None, + "first_truncation_round": None, + "first_truncation_reasoning_tokens": None, + "first_truncation_n": None, + "first_truncation_decision": None, + "continuation_count": 0, + "truncation_match": False, + "final_status": None, + "stopped_reason": None, + "failure_reason": None, + "failure_detail": None, + "_started_perf": perf, + } + self._trim_requests_locked() + summary = self._request_copy_locked(request_id) + self._publish_request(summary) + self.record("info", "request_started", "Responses request received", + request_id=request_id, path=path, model=model, key_identity=safe_identity) + return request_id + + def request_update(self, request_id: str, **fields: Any) -> None: + summary = None + with self._lock: + meta = self._request_meta.get(request_id) + if meta is not None: + meta.update({k: v for k, v in fields.items() if v is not None}) + req = self._request_summaries.get(request_id) + if req is not None: + safe_updates = {k: v for k, v in fields.items() if k in {"model", "path"} and v is not None} + if safe_updates: + req.update(safe_updates) + req["updated_at"] = utc_now_iso() + summary = self._request_copy_locked(request_id) + self._publish_request(summary) + + def mark_fold_start(self, request_id: str, *, model: Any, path: str, upstream_url: str) -> None: + summary = None + with self._lock: + self._counters["folded_requests"] += 1 + req = self._request_summaries.get(request_id) + if req is not None: + req["folded"] = True + req["model"] = model + req["path"] = path + req["updated_at"] = utc_now_iso() + req["protection"] = "processing" + summary = self._request_copy_locked(request_id) + self._publish_request(summary) + self.record("info", "fold_start", "Folded Responses stream started", + request_id=request_id, model=model, path=path, upstream_url=upstream_url) + + def mark_passthrough(self, request_id: str, *, reason: str, model: Any) -> None: + summary = None + with self._lock: + self._counters["passthrough_requests"] += 1 + req = self._request_summaries.get(request_id) + if req is not None: + req["passthrough"] = True + req["passthrough_reason"] = reason + req["model"] = model + req["updated_at"] = utc_now_iso() + req["protection"] = "passthrough" + summary = self._request_copy_locked(request_id) + self._publish_request(summary) + self.record("info", "passthrough", "Request passed through without folding", + request_id=request_id, reason=reason, model=model) + + def round_decision( + self, + request_id: str, + *, + round_no: int, + reasoning_tokens: int | None, + n: int | None, + decision: str, + buffered: list[str], + truncation_match: bool, + ) -> None: + summary = None + if truncation_match: + with self._lock: + self._counters["truncation_hits"] += 1 + with self._lock: + req = self._request_summaries.get(request_id) + if req is not None: + round_summary = { + "round": round_no, + "reasoning_tokens": reasoning_tokens, + "n": n, + "decision": decision, + "buffered": list(buffered), + "truncation_match": truncation_match, + } + req["rounds"].append(round_summary) + req["latest_round"] = round_no + req["latest_reasoning_tokens"] = reasoning_tokens + if truncation_match and req.get("first_truncation_round") is None: + req["first_truncation_round"] = round_no + req["first_truncation_reasoning_tokens"] = reasoning_tokens + req["first_truncation_n"] = n + req["first_truncation_decision"] = decision + req["truncation_match"] = bool(req.get("truncation_match") or truncation_match) + req["updated_at"] = utc_now_iso() + if truncation_match and decision != "continue" and req.get("continuation_count", 0) == 0: + req["protection"] = "risk_uncontinued" + req["stopped_reason"] = decision + summary = self._request_copy_locked(request_id) + self._publish_request(summary) + self.record( + "info", + "round_decision", + "Round finished and continuation decision was made", + request_id=request_id, + round=round_no, + reasoning_tokens=reasoning_tokens, + n=n, + decision=decision, + buffered=buffered, + truncation_match=truncation_match, + ) + + def continuation_opened(self, request_id: str, *, from_round: int, next_round: int, method: str) -> None: + now = utc_now_iso() + summary = None + with self._lock: + self._counters["continuations"] += 1 + self._last_continuation_at = now + req = self._request_summaries.get(request_id) + if req is not None: + req["continuation_count"] = int(req.get("continuation_count") or 0) + 1 + req["updated_at"] = now + req["protection"] = "auto_continued" + summary = self._request_copy_locked(request_id) + self._publish_request(summary) + self.record("info", "continuation_opened", "Opened hidden continuation round", + request_id=request_id, from_round=from_round, next_round=next_round, method=method) + + def request_finished(self, request_id: str, *, status: str, stopped_reason: str | None = None) -> None: + summary = None + with self._lock: + self._active_ids.discard(request_id) + self._counters["active_requests"] = len(self._active_ids) + self._request_meta.pop(request_id, None) + summary = self._finish_request_locked( + request_id, status=status, stopped_reason=stopped_reason + ) + self._trim_requests_locked() + self._publish_request(summary) + self.record("info", "request_finished", "Responses request finished", + request_id=request_id, status=status, stopped_reason=stopped_reason) + + def request_failed(self, request_id: str, *, reason: str, detail: Any = None) -> None: + now = utc_now_iso() + error = {"request_id": request_id, "reason": reason, "detail": redact_value(detail)} + summary = None + with self._lock: + self._active_ids.discard(request_id) + self._counters["active_requests"] = len(self._active_ids) + self._counters["failures"] += 1 + self._last_error_at = now + self._last_error = error + self._request_meta.pop(request_id, None) + summary = self._finish_request_locked( + request_id, + status="failed", + failure_reason=reason, + failure_detail=detail, + ) + self._trim_requests_locked() + self._publish_request(summary) + self.record("warning", "request_failed", "Responses request failed", **error) + + def health(self) -> dict[str, Any]: + return { + "ok": True, + "started_at": self._started_wall, + "uptime_seconds": round(time.monotonic() - self._started_perf, 3), + } + + def snapshot( + self, + *, + upstream: dict[str, Any] | None = None, + config: dict[str, Any] | None = None, + ) -> dict[str, Any]: + with self._lock: + counters = dict(self._counters) + active = list(self._request_meta.values()) + last_error = dict(self._last_error) if self._last_error else None + last_request_at = self._last_request_at + last_continuation_at = self._last_continuation_at + last_error_at = self._last_error_at + return { + **self.health(), + "counters": counters, + "active_requests": active, + "recent_requests": self.recent_requests(limit=10), + "last_request_at": last_request_at, + "last_continuation_at": last_continuation_at, + "last_error_at": last_error_at, + "last_error": last_error, + "upstream": upstream or {"ok": None, "status": "not_checked"}, + "config": config or {}, + } diff --git a/middleware/engine.py b/middleware/engine.py new file mode 100644 index 0000000..382180a --- /dev/null +++ b/middleware/engine.py @@ -0,0 +1,159 @@ +"""Internal CodexCont engine API. + +The engine endpoints are deliberately narrower than the public proxy path. They +return only safe protection summaries that a CPA plugin can persist or display. +""" +from __future__ import annotations + +import json +import time +from dataclasses import asdict, dataclass +from typing import Any + +from starlette.requests import Request +from starlette.responses import JSONResponse + +from .codex import is_truncation_pattern, tier_n +from .diagnostics import redact_value + + +STARTED_AT = time.time() + + +@dataclass(frozen=True) +class EngineRoundSummary: + round: int + reasoning_tokens: int | None = None + truncation_match: bool = False + truncation_n: int | None = None + decision: str = "unknown" + + +def _int_or_none(value: Any) -> int | None: + try: + if value is None or value == "": + return None + return int(value) + except (TypeError, ValueError): + return None + + +def _string(value: Any, default: str = "") -> str: + text = str(value or "").strip() + return text or default + + +def _extract_rounds(payload: dict[str, Any]) -> list[EngineRoundSummary]: + raw_rounds = payload.get("rounds") + rounds: list[EngineRoundSummary] = [] + if isinstance(raw_rounds, list): + for idx, item in enumerate(raw_rounds, start=1): + if not isinstance(item, dict): + continue + round_no = _int_or_none(item.get("round") or item.get("round_no")) or idx + tokens = _int_or_none( + item.get("reasoning_tokens") + or item.get("reasoningTokens") + or item.get("output_tokens_details", {}).get("reasoning_tokens") + ) + trunc = bool(item.get("truncation_match") or is_truncation_pattern(tokens)) + rounds.append( + EngineRoundSummary( + round=round_no, + reasoning_tokens=tokens, + truncation_match=trunc, + truncation_n=tier_n(tokens) if trunc else None, + decision=_string(item.get("decision"), "continue" if trunc else "clean"), + ) + ) + if not rounds: + tokens = _int_or_none( + payload.get("reasoning_tokens") + or payload.get("reasoningTokens") + or payload.get("usage", {}) + .get("output_tokens_details", {}) + .get("reasoning_tokens") + ) + trunc = bool(payload.get("truncation_match") or is_truncation_pattern(tokens)) + rounds.append( + EngineRoundSummary( + round=1, + reasoning_tokens=tokens, + truncation_match=trunc, + truncation_n=tier_n(tokens) if trunc else None, + decision=_string(payload.get("decision"), "continue" if trunc else "clean"), + ) + ) + return rounds + + +def summarize_engine_payload(payload: dict[str, Any]) -> dict[str, Any]: + """Project a safe CodexCont protection summary from engine input. + + This is intentionally tolerant. The CPA plugin can send already-known round + summaries, while tests and smoke probes can send only a usage object. + """ + rounds = _extract_rounds(payload) + latest = rounds[-1] if rounds else EngineRoundSummary(round=1) + first_hit = next((item for item in rounds if item.truncation_match), None) + continuation_count = max(0, int(payload.get("continuation_count") or 0)) + if continuation_count == 0 and len(rounds) > 1: + continuation_count = len(rounds) - 1 + + failure = _string(payload.get("failure_reason") or payload.get("failure")) + stopped_reason = _string(payload.get("stopped_reason") or payload.get("stop_reason")) + folded = bool(payload.get("folded") or continuation_count > 0 or len(rounds) > 1) + passthrough = bool(payload.get("passthrough")) + + if failure: + protection = "failed" + elif passthrough: + protection = "passthrough" + elif continuation_count > 0: + protection = "auto_continued" + elif first_hit is not None: + protection = "risk_uncontinued" + else: + protection = "protected_clean" + + summary = { + "ok": True, + "request_id": _string(payload.get("request_id")), + "model": _string(payload.get("model")), + "protection": protection, + "folded": folded, + "passthrough": passthrough, + "rounds": [asdict(item) for item in rounds], + "latest_round": latest.round, + "latest_reasoning_tokens": latest.reasoning_tokens, + "first_truncation_round": first_hit.round if first_hit else None, + "first_truncation_reasoning_tokens": first_hit.reasoning_tokens if first_hit else None, + "first_truncation_n": first_hit.truncation_n if first_hit else None, + "continuation_count": continuation_count, + "stopped_reason": stopped_reason or None, + "failure_reason": failure or None, + "safe": True, + } + return redact_value(summary) + + +async def engine_healthz(request: Request) -> JSONResponse: + _ = request + return JSONResponse( + { + "ok": True, + "mode": "codexcont-engine", + "uptime_seconds": round(time.time() - STARTED_AT, 3), + } + ) + + +async def engine_analyze(request: Request) -> JSONResponse: + raw = await request.body() + try: + body = json.loads(raw or b"{}") + except (json.JSONDecodeError, UnicodeDecodeError): + return JSONResponse({"ok": False, "error": "invalid_json_body"}, status_code=400) + if not isinstance(body, dict): + return JSONResponse({"ok": False, "error": "body_must_be_object"}, status_code=400) + return JSONResponse(summarize_engine_payload(body)) diff --git a/middleware/key_identity.py b/middleware/key_identity.py new file mode 100644 index 0000000..51e6276 --- /dev/null +++ b/middleware/key_identity.py @@ -0,0 +1,124 @@ +"""Read-only Key Policy identity projection for CodexCont diagnostics.""" +from __future__ import annotations + +import hashlib +import json +from pathlib import Path +from typing import Any + + +def _sha256_hex(value: str) -> str: + return hashlib.sha256(value.strip().encode("utf-8")).hexdigest() + + +def _normalize_hash(value: Any) -> str: + text = str(value or "").strip() + if text.startswith("sha256:"): + text = text.split(":", 1)[1] + if len(text) != 64 or any(ch not in "0123456789abcdefABCDEF" for ch in text): + return "" + return text.lower() + + +def _preview(value: str) -> str: + normalized = _normalize_hash(value) + if not normalized: + return "" + return f"{normalized[:8]}...{normalized[-6:]}" + + +def _first(raw: dict[str, Any], *names: str) -> Any: + for name in names: + if name in raw: + return raw[name] + return None + + +def _extract_keys(data: Any) -> list[dict[str, Any]]: + if isinstance(data, list): + return [item for item in data if isinstance(item, dict)] + if not isinstance(data, dict): + return [] + for path in ( + ("keys",), + ("state", "keys"), + ("data", "keys"), + ("config", "keys"), + ): + current: Any = data + for part in path: + if not isinstance(current, dict): + current = None + break + current = current.get(part) + if isinstance(current, list): + return [item for item in current if isinstance(item, dict)] + return [] + + +def _safe_record(raw: dict[str, Any], key_hash: str) -> dict[str, Any]: + disabled = bool(_first(raw, "disabled", "is_disabled", "isDisabled") or False) + enabled_raw = _first(raw, "enabled", "is_enabled", "isEnabled") + enabled = bool(enabled_raw) if enabled_raw is not None else not disabled + name = str(_first(raw, "name", "label", "alias", "description") or "").strip() or _preview(key_hash) + raw_id = _first(raw, "id", "key_id", "keyId") + return { + "known": True, + "source": "key_policy_state", + "id": str(raw_id).strip() if raw_id is not None else _preview(key_hash), + "name": name, + "preview": str(_first(raw, "preview", "key_preview", "keyPreview") or "").strip() or _preview(key_hash), + "enabled": enabled and not disabled, + } + + +class KeyIdentityResolver: + def __init__(self, state_path: str = "") -> None: + self.state_path = str(state_path or "").strip() + + def identify_authorization(self, authorization: str | None) -> dict[str, Any]: + bearer = _bearer_token(authorization) + if not bearer: + return { + "known": False, + "source": "authorization", + "name": "未携带 Key", + "preview": "", + } + key_hash = _sha256_hex(bearer) + if not self.state_path: + return { + "known": False, + "source": "unconfigured", + "name": "未配置身份表", + "preview": _preview(key_hash), + } + try: + data = json.loads(Path(self.state_path).read_text(encoding="utf-8")) + except Exception: + return { + "known": False, + "source": "key_policy_state_unavailable", + "name": "身份表不可读", + "preview": _preview(key_hash), + } + for item in _extract_keys(data): + stored = _normalize_hash(_first(item, "key_hash", "keyHash", "hash", "api_key_hash", "apiKeyHash")) + if stored and stored == key_hash: + return _safe_record(item, key_hash) + return { + "known": False, + "source": "key_policy_state", + "name": "未识别 Key", + "preview": _preview(key_hash), + } + + +def _bearer_token(authorization: str | None) -> str: + text = str(authorization or "").strip() + if not text: + return "" + parts = text.split(None, 1) + if len(parts) != 2 or parts[0].lower() != "bearer": + return "" + return parts[1].strip() diff --git a/middleware/proxy.py b/middleware/proxy.py index da4b406..dcd3e83 100644 --- a/middleware/proxy.py +++ b/middleware/proxy.py @@ -25,6 +25,7 @@ tier_n, ) from .config import Config +from .diagnostics import Diagnostics from .sse import DONE, incremental_sse, serialize_done, serialize_event log = logging.getLogger("middleware.proxy") @@ -294,6 +295,8 @@ async def fold_stream( first_response: httpx.Response, id_store: Any | None = None, url: str | None = None, + diagnostics: Diagnostics | None = None, + request_id: str | None = None, ) -> AsyncIterator[bytes]: """Yield the folded downstream SSE byte stream. `first_response` is the already-opened (2xx) round-1 upstream response; later rounds are opened here @@ -317,6 +320,7 @@ async def fold_stream( response = first_response round_no = 0 + closed = False try: while True: @@ -434,12 +438,29 @@ async def fold_stream( else "upstream_eof" if not saw_terminal else stopped_reason or "clean" ) + if diagnostics and request_id: + diagnostics.round_decision( + request_id, + round_no=round_no, + reasoning_tokens=rt, + n=n, + decision=decision, + buffered=buffered, + truncation_match=is_truncation_pattern(rt, cont.truncation_step), + ) log.info("round %d: %s | n=%s buffered=%s -> %s", round_no, _fmt_usage(usage), n, buffered or "[]", decision) await response.aclose() if do_continue: + if diagnostics and request_id: + diagnostics.continuation_opened( + request_id, + from_round=round_no, + next_round=round_no + 1, + method=cont.method, + ) last_id = round_reasoning[-1].get("id") or "" if cont.method == "commentary": marker_items = [commentary_message(cont.marker_text)] @@ -485,6 +506,13 @@ async def fold_stream( response.status_code, body) log.info("done: %d round(s) | %s | status=incomplete stop=upstream_error", round_no, _fmt_usage(total_usage)) + if diagnostics and request_id: + closed = True + diagnostics.request_failed( + request_id, + reason="continuation_upstream_error", + detail={"round": round_no + 1, "status_code": response.status_code}, + ) yield serialize_event( _synthetic_incomplete( base_response, final_output, @@ -503,6 +531,11 @@ async def fold_stream( log.warning("round %d: upstream EOF with no terminal event", round_no) log.info("done: %d round(s) | %s | status=incomplete stop=upstream_eof", round_no, _fmt_usage(total_usage)) + if diagnostics and request_id: + closed = True + diagnostics.request_finished( + request_id, status="incomplete", stopped_reason="upstream_eof" + ) yield serialize_event( _synthetic_incomplete( base_response, final_output, @@ -521,6 +554,11 @@ async def fold_stream( status = ((terminal or {}).get("response") or {}).get("status", "completed") log.info("done: %d round(s) | %s | status=%s stop=%s", round_no, _fmt_usage(total_usage), status, stopped_reason or "natural") + if diagnostics and request_id: + closed = True + diagnostics.request_finished( + request_id, status=status, stopped_reason=stopped_reason or "natural" + ) yield serialize_event( _reconstruct_terminal( terminal, base_response, final_output, @@ -535,6 +573,9 @@ async def fold_stream( log.warning("upstream error mid-stream (round %d): %r", round_no, exc) log.info("done: %d round(s) | %s | status=incomplete stop=upstream_error", round_no, _fmt_usage(total_usage)) + if diagnostics and request_id: + closed = True + diagnostics.request_failed(request_id, reason="upstream_stream_error", detail=repr(exc)) yield serialize_event( _synthetic_incomplete( base_response, final_output, @@ -543,6 +584,8 @@ async def fold_stream( ) return finally: + if diagnostics and request_id and not closed: + diagnostics.request_finished(request_id, status="closed") try: await response.aclose() except Exception: diff --git a/pyproject.toml b/pyproject.toml index 9ab4732..99553fc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,6 +7,7 @@ dependencies = [ "httpx>=0.27", "starlette>=0.37", "uvicorn>=0.30", + "zstandard>=0.23", ] [dependency-groups] diff --git a/run_usage_portal.py b/run_usage_portal.py new file mode 100644 index 0000000..c8745ba --- /dev/null +++ b/run_usage_portal.py @@ -0,0 +1,16 @@ +#!/usr/bin/env python3 +"""Run the CPA usage self-service portal.""" +from __future__ import annotations + +import uvicorn + +from cpa_usage_portal import create_app, load_config_from_env + + +def main() -> None: + cfg = load_config_from_env() + uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port) + + +if __name__ == "__main__": + main() diff --git a/tests/test_cpa_usage_portal.py b/tests/test_cpa_usage_portal.py new file mode 100644 index 0000000..f6b458e --- /dev/null +++ b/tests/test_cpa_usage_portal.py @@ -0,0 +1,587 @@ +#!/usr/bin/env python3 +"""Offline tests for the CPA usage self-service portal.""" +from __future__ import annotations + +import json +import sqlite3 +import sys +import tempfile +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT)) + +from starlette.testclient import TestClient + +from cpa_usage_portal.app import create_app +from cpa_usage_portal.budget import suggest_equal_budget +from cpa_usage_portal.config import PortalConfig +from cpa_usage_portal.cpamp import range_window +from cpa_usage_portal.key_policy import KeyPolicyState +from cpa_usage_portal.pricing import apply_event_pricing +from cpa_usage_portal.quota_state import QuotaState +from cpa_usage_portal.redaction import redact, safe_event +from cpa_usage_portal.retention import enforce_usage_retention +from cpa_usage_portal.security import ( + hash_preview, + key_policy_hash, + normalize_key_hash, + sha256_hex, + sign_session, + verify_session, +) + + +_RESULTS: list[tuple[str, bool, str]] = [] + + +def check(name: str, cond: bool, detail: str = "") -> None: + _RESULTS.append((name, bool(cond), detail)) + + +class FakeCPAMP: + def __init__(self, expected_hash: str, other_hash: str) -> None: + self.expected_hash = expected_hash + self.other_hash = other_hash + self.seen_hashes: list[str] = [] + self.seen_windows: list[tuple[bool, int]] = [] + + async def health(self): + return {"ok": True} + + async def analytics(self, *, api_key_hash, window, include_events=False, + include_model_stats=False, + event_limit=100, before_ms=None, before_id=None): + self.seen_hashes.append(api_key_hash) + self.seen_windows.append((include_events, window.to_ms - window.from_ms)) + data = { + "summary": { + "total_calls": 2, + "success_calls": 1, + "failure_calls": 1, + "success_rate": 0.5, + "input_tokens": 100, + "output_tokens": 50, + "cached_tokens": 20, + "cache_read_tokens": 0, + "cache_creation_tokens": 0, + "total_tokens": 300, + "reasoning_tokens": 100, + "total_cost": 0, + }, + "timeline": [], + "model_share": [{"model": "gpt-5.5", "calls": 2, "tokens": 300, "cost": 0}], + "model_stats": [ + { + "model": "gpt-5.5", + "calls": 2, + "success_calls": 1, + "failure_calls": 1, + "success_rate": 0.5, + "input_tokens": 100, + "output_tokens": 50, + "cached_tokens": 20, + "cache_read_tokens": 0, + "cache_creation_tokens": 0, + "total_tokens": 300, + "cost": 0, + } + ] if include_model_stats else [], + "api_key_stats": [ + {"api_key_hash": api_key_hash, "calls": 2, "total_tokens": 300}, + {"api_key_hash": self.other_hash, "calls": 99, "total_tokens": 999}, + ], + } + if include_events: + data["events"] = { + "items": [ + { + "event_hash": "evt-a", + "request_id": "req-a", + "timestamp_ms": window.to_ms - 1000, + "api_key_hash": api_key_hash, + "model": "gpt-5.5", + "failed": False, + "input_tokens": 100, + "output_tokens": 50, + "cached_tokens": 20, + "total_tokens": 200, + "reasoning_tokens": 80, + "latency_ms": 1234, + "ttft_ms": 321, + "cost": 0, + "service_tier": "priority", + "reasoning_effort": "high", + }, + { + "event_hash": "evt-b", + "request_id": "req-b", + "timestamp_ms": window.to_ms - 500, + "api_key_hash": self.other_hash, + "model": "gpt-5.5", + "failed": True, + "fail_summary": "Authorization: Bearer should-not-leak", + }, + ], + "has_more": False, + "total_count": 2, + } + return data + + +def write_state(path: Path, raw_key: str, disabled_key: str) -> tuple[str, str, str]: + key_hash = sha256_hex(raw_key) + cpamp_hash = sha256_hex("alice-key") + disabled_hash = sha256_hex(disabled_key) + path.write_text( + json.dumps( + { + "keys": [ + { + "id": "alice-key", + "key_hash": key_policy_hash(key_hash), + "name": "Alice", + "enabled": True, + "rpm": 12, + "models": [ + { + "alias": "gpt-5.5", + "provider": "codex", + "target_model": "gpt-5.5", + "input_price_per_million": 5, + "output_price_per_million": 30, + "cache_read_price_per_million": 0.5, + } + ], + "daily_limit_usd": 5, + "weekly_limit_usd": 30, + "daily_usage_usd": 0.5, + }, + { + "id": "disabled-key", + "key_hash": key_policy_hash(disabled_hash), + "name": "Disabled", + "enabled": False, + "models": [ + { + "alias": "gpt-5.5", + "input_price_per_million": 5, + "output_price_per_million": 30, + "cache_read_price_per_million": 0.5, + } + ], + }, + ] + }, + ensure_ascii=False, + ), + encoding="utf-8", + ) + return key_hash, cpamp_hash, disabled_hash + + +def test_hash_and_session() -> None: + raw = " cpa_test_key " + hex_hash = sha256_hex(raw) + check("sha256 trims raw key", hex_hash == sha256_hex(raw.strip())) + check("normalize strips sha256 prefix", normalize_key_hash(key_policy_hash(hex_hash)) == hex_hash) + check("hash preview hides middle", hash_preview(hex_hash).startswith(hex_hash[:8])) + + token = sign_session({"key_hash": hex_hash, "key_name": "Alice"}, "secret", ttl_seconds=60) + payload = verify_session(token, "secret") + check("session verifies", payload is not None and payload.get("key_hash") == hex_hash) + check("session rejects wrong secret", verify_session(token, "other") is None) + + +def test_key_policy_and_budget(tmp: Path) -> None: + key_hash, cpamp_hash, _ = write_state(tmp / "state.json", "cpa_live", "cpa_disabled") + state = KeyPolicyState.load(tmp / "state.json") + record = state.get(key_hash) + check("key policy finds key", record is not None and record.name == "Alice") + check("key policy validates raw key hash", state.get_by_raw_hash(key_hash) is record) + check("key policy cpamp hash uses policy id", + record is not None and record.cpamp_hash == cpamp_hash) + check("key policy finds cpamp hash", state.get_by_cpamp_hash(cpamp_hash) is record) + check("key policy safe dict hides full hash", + record is not None and key_hash not in json.dumps(record.safe_dict())) + check("key policy keeps clean model aliases", + record is not None and record.models == ["gpt-5.5"], + str(record.models if record else None)) + check("key policy parses per-model prices", + record is not None + and record.model_prices["gpt-5.5"].input_per_million == 5 + and record.model_prices["gpt-5.5"].output_per_million == 30 + and record.model_prices["gpt-5.5"].cache_read_per_million == 0.5, + str(record.model_prices if record else None)) + check("key policy safe dict exposes limits", + record is not None + and record.safe_dict()["limits"]["daily_usd"] == 5 + and record.safe_dict()["limits"]["weekly_usd"] == 30, + str(record.safe_dict() if record else None)) + + suggestion = suggest_equal_budget(state.enabled_keys(), total_daily_usd=10, total_weekly_usd=70) + check("budget enabled count", suggestion.enabled_key_count == 1) + check("budget daily assigned", suggestion.per_key_daily_usd == 10) + check("budget patch uses policy hash", suggestion.patches[0]["key_hash"].startswith("sha256:")) + + bad_state = KeyPolicyState([record.__class__(**{**record.__dict__, "raw": {}, "model_prices": {}})]) + try: + suggest_equal_budget(bad_state.enabled_keys(), total_daily_usd=1) + blocked = False + except ValueError: + blocked = True + check("budget blocks missing prices", blocked) + + +def test_redaction_and_safe_event() -> None: + expected = sha256_hex("cpa_live") + other = sha256_hex("cpa_other") + redacted = redact("Authorization: Bearer abc123 and api_key=secret") + check("redact bearer", "abc123" not in redacted and "[REDACTED]" in redacted, redacted) + safe = safe_event( + { + "api_key_hash": expected, + "event_hash": "evt", + "fail_summary": "access_token=secret", + "failed": True, + }, + expected_hash=expected, + ) + check("safe event accepts own hash", safe is not None) + check("safe event redacts failure", safe is not None and "secret" not in safe.get("failure", "")) + check("safe event has short failure brief", + safe is not None and safe.get("failure_brief") == "access_token=[REDACTED]", + str(safe)) + header_blob = json.dumps({ + "Cf-Cache-Status": "DYNAMIC", + "Set-Cookie": "secret-cookie", + "Strict-Transport-Security": "max-age=31536000", + "X-Codex-Plan-Type": "pro", + }) + clean = safe_event( + { + "api_key_hash": expected, + "failed": False, + "fail_summary": header_blob, + }, + expected_hash=expected, + ) + check("safe event drops success header blob", + clean is not None and clean.get("failure") == "" and clean.get("failure_brief") == "", + str(clean)) + failed = safe_event( + { + "api_key_hash": expected, + "failed": True, + "fail_status_code": 502, + "fail_summary": header_blob, + }, + expected_hash=expected, + ) + check("safe event summarizes failed header blob", + failed is not None and failed.get("failure_brief") == "Upstream response headers omitted; inspect status code and quota fields.", + str(failed)) + long_failure = safe_event( + { + "api_key_hash": expected, + "failed": True, + "fail_summary": "x" * 1000, + }, + expected_hash=expected, + ) + check("safe event bounds failure detail", + long_failure is not None and len(long_failure.get("failure", "")) <= 600, + str(long_failure)) + check("safe event rejects other hash", + safe_event({"api_key_hash": other}, expected_hash=expected) is None) + + +def test_pricing_breakdown(tmp: Path) -> None: + write_state(tmp / "state.json", "cpa_live", "cpa_disabled") + record = KeyPolicyState.load(tmp / "state.json").enabled_keys()[0] + events = apply_event_pricing([ + { + "model": "gpt-5.5", + "input_tokens": 100, + "cached_tokens": 20, + "output_tokens": 50, + "reasoning_tokens": 30, + "total_tokens": 150, + "service_tier": "priority", + "cost": 0, + } + ], record.model_prices) + event = events[0] + breakdown = event.get("cost_breakdown") or {} + costs = breakdown.get("costs") or {} + parts = sum(float(costs.get(name) or 0) for name in ("input", "cached_input", "cache_read", "cache_creation", "output")) + check("pricing breakdown emitted", breakdown.get("price_model") == "gpt-5.5", str(breakdown)) + check("pricing breakdown total matches cost", + abs(float(event.get("cost") or 0) - float(costs.get("total") or 0)) < 1e-12, + str(event)) + check("pricing breakdown parts sum to total", + abs(parts - float(costs.get("total") or 0)) < 1e-12, + str(costs)) + check("pricing breakdown keeps reasoning metric", + (breakdown.get("tokens") or {}).get("reasoning") == 30 + and (breakdown.get("tokens") or {}).get("visible_output_estimate") == 20, + str(breakdown.get("tokens"))) + token_breakdown = breakdown.get("tokens") or {} + check("pricing treats cpamp cached tokens as cache hits", + token_breakdown.get("cpamp_cached_input") == 20 + and token_breakdown.get("fine_grained_cache_read") == 0 + and token_breakdown.get("effective_cache_read_for_hit_rate") == 20 + and token_breakdown.get("cache_semantics") == "cpamp_compatible_cached_tokens", + str(token_breakdown)) + check("pricing charges cached input even when fine-grained read is zero", + float(costs.get("cached_input") or 0) > 0 and float(costs.get("cache_read") or 0) == 0, + str(costs)) + + raw_events = apply_event_pricing([ + { + "model": "gpt-5.5", + "input_tokens": 100, + "cached_tokens": 80, + "cache_tokens": 80, + "cache_read_tokens": 30, + "cache_creation_tokens": 10, + "output_tokens": 0, + "cost": 0, + } + ], record.model_prices) + raw_tokens = (raw_events[0].get("cost_breakdown") or {}).get("tokens") or {} + check("pricing normalizes raw cache bucket like CPAMP", + raw_tokens.get("cpamp_cached_input") == 40 + and raw_tokens.get("fine_grained_cache_read") == 30 + and raw_tokens.get("fine_grained_cache_creation") == 10 + and raw_tokens.get("effective_cache_read_for_hit_rate") == 70 + and raw_tokens.get("cache_semantics") == "raw_cache_tokens_normalized_to_cpamp", + str(raw_tokens)) + + +def test_retention(tmp: Path) -> None: + db = tmp / "usage.sqlite" + conn = sqlite3.connect(db) + conn.execute("create table usage_events (id integer primary key autoincrement, timestamp_ms integer not null)") + conn.execute("insert into usage_events(timestamp_ms) values (?)", (1_000,)) + conn.execute("insert into usage_events(timestamp_ms) values (?)", (10_000_000_000,)) + conn.commit() + conn.close() + result = enforce_usage_retention(db, keep_days=7, batch_size=1, now_ms_value=10_000_000_000) + verify = sqlite3.connect(db) + try: + count = verify.execute("select count(*) from usage_events").fetchone()[0] + finally: + verify.close() + check("retention deletes old rows", result.deleted == 1 and count == 1, str(result)) + + +def test_quota_state(tmp: Path) -> None: + quota = QuotaState(tmp / "quota.sqlite") + limits = quota.get_limits("alice-key") + check("quota state defaults empty", limits.five_hour_usd is None and limits.monthly_usd is None) + updated = quota.set_limits("alice-key", five_hour_usd=1.5, monthly_usd=20, actor="tester") + check("quota state stores local limits", updated.five_hour_usd == 1.5 and updated.monthly_usd == 20) + points = quota.reset("alice-key", window="5h", actor="tester", reset_at_ms=1_800_000_000_000) + check("quota state stores reset watermark", points["5h"] == 1_800_000_000_000, str(points)) + window, reset_at = quota.effective_window("alice-key", "5h", now_ms_value=1_800_000_100_000) + check("quota state applies reset watermark", + reset_at == 1_800_000_000_000 and window.from_ms == 1_800_000_000_000, + str((window, reset_at))) + month = range_window("month", now_ms_value=1_783_108_800_000) + check("range window supports month", month.from_ms < month.to_ms) + + +def test_app_routes(tmp: Path) -> None: + key_hash, cpamp_hash, other_raw_hash = write_state(tmp / "state.json", "cpa_live", "cpa_disabled") + other_hash = sha256_hex("other-policy-id") + cfg = PortalConfig( + key_policy_state_path=str(tmp / "state.json"), + local_state_db_path=str(tmp / "quota.sqlite"), + cpamp_admin_key="cpamp_test", + session_secret="session_secret", + cookie_secure=False, + ) + with TestClient(create_app(cfg)) as client: + fake = FakeCPAMP(cpamp_hash, other_hash) + client.app.state.cpamp = fake + + health = client.get("/healthz") + check("portal healthz ok", health.status_code == 200 and health.json().get("ok") is True) + html = client.get("/") + check("portal dashboard html ok", html.status_code == 200 and "CPA 用量自助页" in html.text) + admin_public = client.get("/admin/") + check("portal admin hidden without proxy header", admin_public.status_code == 404) + admin_html = client.get("/admin/", headers={"x-usage-admin": "1"}) + check("portal admin dashboard html ok", admin_html.status_code == 200 and "CPA 用量管理" in admin_html.text) + check("portal admin supports usage-admin mount", "API_BASE" in admin_html.text and "/usage-admin" in admin_html.text) + check("portal admin has one save all action", + "保存全部" in admin_html.text and "data-save" not in admin_html.text, + admin_html.text[:200]) + check("portal admin defaults all-key request view", + "全部 Key" in admin_html.text and "用户/Key" in admin_html.text, + admin_html.text[:200]) + check("portal admin detail shows useful breakdown", + "Token 组成" in admin_html.text and "费用组成" in admin_html.text and "CPAMP 缓存命中" in admin_html.text, + admin_html.text[:200]) + check("portal admin has no metric crescent", "metric::after" not in admin_html.text) + check("portal dashboard refresh reconnects stream", "startStream({ force: true })" in html.text) + check("portal dashboard revives after background", "visibilitychange" in html.text) + check("portal dashboard shows refresh animation", "is-loading" in html.text and "stream warn" in html.text) + check("portal dashboard shows key limits", "用量限额" in html.text and "日限" in html.text and "周限" in html.text) + check("portal dashboard follows delayed usage updates", + "mergeEvent(JSON.parse(ev.data))" in html.text and "scheduleFollowUpRefreshes" in html.text) + check("portal dashboard applies selected range to events", + "api(`/api/events?range=${encodeURIComponent(range)}&limit=100`)" in html.text + and "当前显示" in html.text) + check("portal dashboard supports 5h and month ranges", + '' in html.text + and '' in html.text) + check("portal dashboard has no metric crescent", "metric::after" not in html.text) + check("portal dashboard explains cache semantics", + "CPAMP 缓存命中" in html.text and "细粒度 Cache Read" in html.text, + html.text[:200]) + + admin_keys = client.get("/admin/api/keys", headers={"x-usage-admin": "1"}) + admin_body = admin_keys.json() + check("portal admin lists quota windows", + admin_keys.status_code == 200 + and {"5h", "24h", "7d", "month"}.issubset(set(admin_body["keys"][0]["usage_windows"].keys())), + str(admin_body)) + limits_update = client.put( + "/admin/api/keys/alice-key/limits", + headers={"x-usage-admin": "1"}, + json={"five_hour_usd": 1.25, "monthly_usd": 20}, + ) + check("portal admin updates local limits", + limits_update.status_code == 200 + and limits_update.json()["me"]["limits"]["five_hour_usd"] == 1.25 + and limits_update.json()["me"]["limits"]["monthly_usd"] == 20, + limits_update.text) + batch_update = client.put( + "/admin/api/keys/limits", + headers={"x-usage-admin": "1"}, + json={"limits": [{"id": "alice-key", "five_hour_usd": "2.5", "monthly_usd": "25"}]}, + ) + check("portal admin batch updates local limits", + batch_update.status_code == 200 + and batch_update.json()["keys"][0]["limits"]["five_hour_usd"] == 2.5 + and batch_update.json()["keys"][0]["limits"]["monthly_usd"] == 25, + batch_update.text) + batch_bad = client.put( + "/admin/api/keys/limits", + headers={"x-usage-admin": "1"}, + json={"limits": [{"id": "alice-key", "five_hour_usd": "not-a-number", "monthly_usd": "25"}]}, + ) + check("portal admin batch rejects invalid limits", + batch_bad.status_code == 400 and "invalid_limit:alice-key" in batch_bad.text, + batch_bad.text) + reset = client.post( + "/admin/api/keys/alice-key/reset", + headers={"x-usage-admin": "1"}, + json={"window": "5h"}, + ) + check("portal admin soft resets window", reset.status_code == 200 and reset.json()["reset_points"]["5h"], reset.text) + fake.seen_hashes.clear() + fake.seen_windows.clear() + + bad = client.post("/api/session", json={"api_key": "nope"}) + check("portal rejects unknown key", bad.status_code == 401) + + login = client.post("/api/session", json={"api_key": "cpa_live"}) + check("portal login ok", login.status_code == 200, login.text) + cookie = login.headers.get("set-cookie", "") + check("portal cookie httponly", "HttpOnly" in cookie, cookie) + check("portal cookie does not contain raw key", "cpa_live" not in cookie, cookie) + + me = client.get("/api/me") + check("portal me ok", me.status_code == 200 and me.json()["me"]["name"] == "Alice", me.text) + check("portal me exposes local limits and reset points", + me.json()["me"]["limits"]["five_hour_usd"] == 2.5 + and me.json()["me"]["limits"]["monthly_usd"] == 25 + and me.json()["me"]["reset_points"]["5h"], + me.text) + usage = client.get("/api/usage?range=24h") + usage_body = usage.json() + check("portal usage ok", usage.status_code == 200 and usage_body["summary"]["total_calls"] == 2) + check("portal usage includes daily and weekly limits", + me.json()["me"]["limits"]["daily_usd"] == 5 and me.json()["me"]["limits"]["weekly_usd"] == 30, + me.text) + check("portal usage recomputes cost from key policy prices", + usage_body["summary"]["total_cost"] > 0 + and usage_body["model_share"][0]["cost"] > 0 + and usage_body["summary"]["cost_source"] == "key_policy", + str(usage_body)) + check("portal usage stat hides cpamp hash", cpamp_hash not in json.dumps(usage_body), str(usage_body)) + check("portal usage stats reject other hash", + len(usage_body["api_key_stats"]) == 1 and usage_body["api_key_stats"][0]["calls"] == 2, + str(usage_body)) + events = client.get("/api/events?range=24h&limit=100") + body = events.json() + check("portal events return selected range", + body.get("range") == "24h" + and any(include_events and delta <= 25 * 60 * 60 * 1000 for include_events, delta in fake.seen_windows), + str(body)) + check("portal filters events to own key", len(body["events"]) == 1, str(body)) + check("portal events recompute cost from key policy prices", + body["events"][0]["cost"] > 0 and body["events"][0]["cost_source"] == "key_policy", + str(body)) + check("portal events include accounting windows", + "accounting" in body["events"][0] + and "24h" in body["events"][0]["accounting"]["included_windows"], + str(body)) + check("portal events include pricing breakdown", + "cost_breakdown" in body["events"][0] + and body["events"][0]["cost_breakdown"]["costs"]["total"] == body["events"][0]["cost"], + str(body["events"][0])) + check("portal events expose cpamp cache hit semantics", + body["events"][0]["cost_breakdown"]["tokens"]["cpamp_cached_input"] == 20 + and body["events"][0]["cost_breakdown"]["tokens"]["effective_cache_read_for_hit_rate"] == 20, + str(body["events"][0]["cost_breakdown"]["tokens"])) + admin_events = client.get("/admin/api/events?key_id=all&range=24h&limit=100", headers={"x-usage-admin": "1"}) + admin_events_body = admin_events.json() + check("portal admin all-key events ok", + admin_events.status_code == 200 + and admin_events_body.get("key_id") == "all" + and admin_events_body["events"][0]["key"]["name"] == "Alice", + str(admin_events_body)) + check("portal admin all-key event hides full hashes", + cpamp_hash not in json.dumps(admin_events_body) and key_hash not in json.dumps(admin_events_body), + str(admin_events_body)) + month_usage = client.get("/api/usage?range=month") + check("portal usage supports month range", + month_usage.status_code == 200 and month_usage.json()["range"] == "month", + month_usage.text) + check("portal never returns full raw api hash", key_hash not in json.dumps(body), str(body)) + check("portal does not use disabled raw hash", other_raw_hash not in json.dumps(body), str(body)) + check("portal cpamp filter used policy-id hash", + fake.seen_hashes and all(h == cpamp_hash for h in fake.seen_hashes)) + + +def main() -> None: + test_hash_and_session() + with tempfile.TemporaryDirectory() as d: + test_key_policy_and_budget(Path(d)) + test_redaction_and_safe_event() + with tempfile.TemporaryDirectory() as d: + test_pricing_breakdown(Path(d)) + with tempfile.TemporaryDirectory() as d: + test_retention(Path(d)) + with tempfile.TemporaryDirectory() as d: + test_quota_state(Path(d)) + with tempfile.TemporaryDirectory() as d: + test_app_routes(Path(d)) + + passed = sum(1 for _, ok, _ in _RESULTS if ok) + for name, ok, detail in _RESULTS: + mark = "PASS" if ok else "FAIL" + line = f"[{mark}] {name}" + if not ok and detail: + line += f" -- {detail}" + print(line) + print(f"\n{passed}/{len(_RESULTS)} checks passed") + sys.exit(0 if passed == len(_RESULTS) else 1) + + +if __name__ == "__main__": + main() diff --git a/tests/test_middleware.py b/tests/test_middleware.py index 216605a..baea754 100644 --- a/tests/test_middleware.py +++ b/tests/test_middleware.py @@ -7,8 +7,10 @@ from __future__ import annotations import asyncio +import hashlib import json import sys +import tempfile from dataclasses import replace from pathlib import Path @@ -16,9 +18,17 @@ FIXTURES = Path(__file__).resolve().parent / "fixtures" sys.path.insert(0, str(ROOT)) +import zstandard as zstd from starlette.datastructures import Headers - -from middleware.app import _make_client, _resolve_upstream_url, _url_is_from_header +from starlette.testclient import TestClient + +from middleware.app import ( + create_app, + _decode_request_body, + _make_client, + _resolve_upstream_url, + _url_is_from_header, +) from middleware.codex import ( continue_call_id, is_truncation_pattern, @@ -29,6 +39,9 @@ ) from middleware.config import load_config from middleware.creds import build_upstream_headers, would_inject_authorization +from middleware.diagnostics import Diagnostics, redact_value +from middleware.engine import summarize_engine_payload +from middleware.key_identity import KeyIdentityResolver from middleware.proxy import fold_stream from middleware.sse import DONE, incremental_sse from middleware.store import IdStore @@ -391,6 +404,7 @@ def test_header_transparency(): ("User-Agent", "codex_cli_rs/1.0"), ("Host", "drop.me"), ("Content-Length", "123"), + ("Content-Encoding", "zstd"), ("Accept-Encoding", "gzip"), ("Responses-API-Base", "https://override/responses"), ("X-Custom", "keep"), @@ -401,10 +415,265 @@ def test_header_transparency(): check("hdr keeps user-agent", low.get("user-agent") == "codex_cli_rs/1.0") check("hdr keeps custom", low.get("x-custom") == "keep") check("hdr keeps authorization", low.get("authorization") == "Bearer agent") - for dropped in ("host", "content-length", "accept-encoding", "responses-api-base"): + for dropped in ( + "host", + "content-length", + "content-encoding", + "accept-encoding", + "responses-api-base", + ): check(f"hdr drops {dropped}", dropped not in low) +def test_zstd_request_body_decode(): + raw = b'{"model":"gpt-5.5","stream":true}' + encoded = zstd.ZstdCompressor().compress(raw) + check("zstd body decodes", _decode_request_body(encoded, "zstd") == raw) + check("identity body unchanged", _decode_request_body(raw, None) == raw) + + +# --- dashboard diagnostics -------------------------------------------------- + + +def test_diagnostics_ring_and_redaction(): + diag = Diagnostics(max_events=2) + diag.record("info", "first", "Authorization: Bearer abc123", + authorization="Bearer abc123", nested={"api_key": "secret"}) + diag.record("info", "second", "ok") + diag.record("warning", "third", "access_token=abc") + recent = diag.recent() + check("diagnostics ring keeps max events", [e["event"] for e in recent] == ["second", "third"], + str([e["event"] for e in recent])) + bearer_redacted = redact_value("Authorization: Bearer abc123") + check("redact bearer text", + "abc123" not in bearer_redacted and "[REDACTED]" in bearer_redacted, + bearer_redacted) + redacted = redact_value({"api_key": "secret", "safe": "value"}) + check("redact sensitive dict key", redacted.get("api_key") == "[REDACTED]") + check("keep safe dict key", redacted.get("safe") == "value") + token_counts = redact_value({"reasoning_tokens": 516, "total_tokens": 1024}) + check("keep token counters", token_counts.get("reasoning_tokens") == 516) + + +def test_diagnostics_request_summaries(): + with tempfile.TemporaryDirectory() as d: + raw = "cpa_live_key" + key_hash = hashlib.sha256(raw.encode("utf-8")).hexdigest() + state_path = Path(d) / "key-policy.json" + state_path.write_text(json.dumps({ + "keys": [{ + "id": "alice-key", + "key_hash": f"sha256:{key_hash}", + "name": "Alice", + "preview": "cpa_...live", + "enabled": True, + }] + }), encoding="utf-8") + identity = KeyIdentityResolver(str(state_path)).identify_authorization(f"Bearer {raw}") + check("key identity resolves bearer safely", + identity.get("known") is True + and identity.get("name") == "Alice" + and raw not in json.dumps(identity), + str(identity)) + unknown = KeyIdentityResolver(str(state_path)).identify_authorization("Bearer other") + check("key identity unknown uses hash preview", + unknown.get("known") is False + and unknown.get("name") == "未识别 Key" + and "other" not in json.dumps(unknown), + str(unknown)) + + clean = Diagnostics(max_events=10, max_requests=5) + rid = clean.request_started( + path="/v1/responses", + model="gpt-5.5", + key_identity={"known": True, "name": "Alice", "preview": "cpa_...live", "source": "test"}, + ) + clean.mark_fold_start(rid, model="gpt-5.5", path="/v1/responses", + upstream_url="http://cpa:8317/v1/responses") + clean.round_decision(rid, round_no=1, reasoning_tokens=140, n=None, + decision="clean", buffered=["message"], truncation_match=False) + clean.request_finished(rid, status="completed", stopped_reason="natural") + summary = clean.recent_requests(limit=1)[0] + check("request summary protected clean", summary.get("protection") == "protected_clean", + str(summary)) + check("request summary keeps reasoning tokens", + summary.get("latest_reasoning_tokens") == 140, str(summary)) + check("request summary clean has no truncation round", + summary.get("first_truncation_round") is None, str(summary)) + check("request summary exposes safe key identity", + (summary.get("key_identity") or {}).get("name") == "Alice" + and "cpa_live_key" not in json.dumps(summary), + str(summary)) + + cont = Diagnostics(max_events=10, max_requests=5) + rid = cont.request_started(path="/v1/responses", model="gpt-5.5") + cont.mark_fold_start(rid, model="gpt-5.5", path="/v1/responses", + upstream_url="http://cpa:8317/v1/responses") + cont.round_decision(rid, round_no=1, reasoning_tokens=516, n=1, + decision="continue", buffered=["message"], truncation_match=True) + cont.continuation_opened(rid, from_round=1, next_round=2, method="commentary") + cont.round_decision(rid, round_no=2, reasoning_tokens=181, n=None, + decision="clean", buffered=["message"], truncation_match=False) + cont.request_finished(rid, status="completed", stopped_reason="natural") + summary = cont.recent_requests(limit=1)[0] + check("request summary auto continued", summary.get("protection") == "auto_continued", + str(summary)) + check("request summary continuation count", summary.get("continuation_count") == 1, + str(summary)) + check("request summary records first truncation round", + summary.get("first_truncation_round") == 1, str(summary)) + check("request summary records first truncation tokens", + summary.get("first_truncation_reasoning_tokens") == 516, str(summary)) + check("request summary latest reasoning can be clean round", + summary.get("latest_reasoning_tokens") == 181, str(summary)) + + risk = Diagnostics(max_events=10, max_requests=5) + rid = risk.request_started(path="/v1/responses", model="gpt-5.5") + risk.mark_fold_start(rid, model="gpt-5.5", path="/v1/responses", + upstream_url="http://cpa:8317/v1/responses") + risk.round_decision(rid, round_no=1, reasoning_tokens=516, n=1, + decision="no_encrypted_content", buffered=["message"], truncation_match=True) + risk.request_finished(rid, status="completed", stopped_reason="no_encrypted_content") + summary = risk.recent_requests(limit=1)[0] + check("request summary risk uncontinued", summary.get("protection") == "risk_uncontinued", + str(summary)) + check("request summary risk decision captured", + summary.get("first_truncation_decision") == "no_encrypted_content", str(summary)) + + passthrough = Diagnostics(max_events=10, max_requests=5) + rid = passthrough.request_started(path="/v1/responses", model="gpt-5.5") + passthrough.mark_passthrough(rid, reason="non-stream", model="gpt-5.5") + passthrough.request_finished(rid, status="passthrough:200") + summary = passthrough.recent_requests(limit=1)[0] + check("request summary passthrough", summary.get("protection") == "passthrough", + str(summary)) + + failed = Diagnostics(max_events=10, max_requests=5) + rid = failed.request_started(path="/v1/responses", model="gpt-5.5") + failed.request_failed(rid, reason="invalid_json_body") + summary = failed.recent_requests(limit=1)[0] + check("request summary failed", summary.get("protection") == "failed", str(summary)) + + retained = Diagnostics(max_events=10, max_requests=2) + for idx in range(3): + rid = retained.request_started(path="/v1/responses", model=f"m{idx}") + retained.request_finished(rid, status="completed") + summaries = retained.recent_requests() + check("request summary retention bounded", len(summaries) == 2, str(summaries)) + check("request summary retention newest", [s["model"] for s in summaries] == ["m1", "m2"], + str(summaries)) + + +async def test_diagnostics_subscriber_broadcast(): + diag = Diagnostics(max_events=5) + queue = diag.subscribe() + diag.record("info", "broadcast", "hello", request_id="req1") + item = await asyncio.wait_for(queue.get(), timeout=1.0) + diag.unsubscribe(queue) + check("diagnostics subscriber receives event", item.get("event") == "broadcast", str(item)) + check("diagnostics subscriber receives fields", + (item.get("fields") or {}).get("request_id") == "req1", str(item)) + + +def test_admin_routes_smoke(): + base = load_config(ROOT / "config.toml") + cfg = replace( + base, + upstream=replace(base.upstream, url="http://127.0.0.1:9/v1/responses"), + ) + with TestClient(create_app(cfg)) as client: + health = client.get("/admin/healthz") + check("admin healthz 200", health.status_code == 200, str(health.status_code)) + check("admin healthz ok", health.json().get("ok") is True, str(health.text)) + + client.app.state.diagnostics.record("info", "manual_event", "hello") + rid = client.app.state.diagnostics.request_started(path="/v1/responses", model="gpt-5.5") + client.app.state.diagnostics.mark_fold_start( + rid, model="gpt-5.5", path="/v1/responses", + upstream_url="http://127.0.0.1:9/v1/responses" + ) + client.app.state.diagnostics.request_finished(rid, status="completed") + logs = client.get("/admin/logs?limit=1") + body = logs.json() + check("admin logs 200", logs.status_code == 200, str(logs.status_code)) + check("admin logs returns recent event", + (body.get("events") or [{}])[-1].get("event") == "request_finished", str(body)) + + requests = client.get("/admin/requests?limit=1") + requests_body = requests.json() + check("admin requests 200", requests.status_code == 200, str(requests.status_code)) + check("admin requests returns summary", + (requests_body.get("requests") or [{}])[-1].get("protection") == "protected_clean", + str(requests_body)) + + status = client.get("/admin/status") + status_body = status.json() + check("admin status 200", status.status_code == 200, str(status.status_code)) + check("admin status has counters", "counters" in status_body, str(status_body)) + check("admin status redacted config host", + (status_body.get("config") or {}).get("upstream_host") == "127.0.0.1:9", + str(status_body.get("config"))) + + html = client.get("/admin/") + check("admin dashboard html 200", html.status_code == 200, str(html.status_code)) + check("admin dashboard contains EventSource", "new EventSource" in html.text) + check("admin dashboard Chinese first screen", "最近请求" in html.text) + check("admin dashboard has trigger round column", "命中轮" in html.text) + check("admin dashboard has latest reasoning column", "末轮思考量" in html.text) + check("admin dashboard has key identity column", "用户/Key" in html.text) + check("admin dashboard has no metric crescent", "metric::after" not in html.text) + check("admin dashboard refresh reconnects stream", "connectStream({ force: true })" in html.text) + check("admin dashboard revives after background", "visibilitychange" in html.text) + check("admin dashboard shows refresh animation", "is-loading" in html.text and "stream warn" in html.text) + check("admin dashboard follows processing requests", + "scheduleRequestFollowUp" in html.text and "setInterval(loadRequests, 5000)" in html.text) + + stream = client.get("/admin/logs/stream?once=1") + check("admin logs stream ready", "event: ready" in stream.text, stream.text[:80]) + check("admin logs stream request event", "event: request" in stream.text, stream.text[:200]) + + engine_health = client.get("/engine/healthz") + check("engine healthz 200", engine_health.status_code == 200, str(engine_health.status_code)) + check("engine healthz mode", engine_health.json().get("mode") == "codexcont-engine", + engine_health.text) + engine_summary = client.post("/engine/v1/responses/analyze", json={ + "model": "gpt-5.5", + "rounds": [ + {"round": 1, "reasoning_tokens": 516, "decision": "continue"}, + {"round": 2, "reasoning_tokens": 181, "decision": "clean"}, + ], + }) + body = engine_summary.json() + check("engine analyze 200", engine_summary.status_code == 200, engine_summary.text) + check("engine analyze auto continued", body.get("protection") == "auto_continued", str(body)) + check("engine analyze first hit", body.get("first_truncation_round") == 1, str(body)) + + +def test_engine_summary_projection(): + clean = summarize_engine_payload({ + "model": "gpt-5.5", + "usage": {"output_tokens_details": {"reasoning_tokens": 140}}, + }) + check("engine summary clean", clean.get("protection") == "protected_clean", str(clean)) + check("engine summary latest tokens", clean.get("latest_reasoning_tokens") == 140, str(clean)) + + risk = summarize_engine_payload({ + "model": "gpt-5.5", + "reasoning_tokens": 516, + "stopped_reason": "no_encrypted_content", + }) + check("engine summary risk", risk.get("protection") == "risk_uncontinued", str(risk)) + check("engine summary truncation n", risk.get("first_truncation_n") == 1, str(risk)) + + failed = summarize_engine_payload({ + "model": "gpt-5.5", + "failure_reason": "Authorization: Bearer secret-token", + }) + check("engine summary failed", failed.get("protection") == "failed", str(failed)) + check("engine summary redacts failure", + "secret-token" not in json.dumps(failed), str(failed)) + + # --- upstream URL resolution via Responses-API-Base header ------------------ @@ -623,6 +892,12 @@ async def _main(): await test_tool_pair_continuation_payload() await test_forward_marker_emits_downstream() test_header_transparency() + test_zstd_request_body_decode() + test_diagnostics_ring_and_redaction() + test_diagnostics_request_summaries() + await test_diagnostics_subscriber_broadcast() + test_admin_routes_smoke() + test_engine_summary_projection() test_upstream_url_resolution() test_auth_safety_guard() test_auth_injection() diff --git a/uv.lock b/uv.lock index f36cfb0..a12d7e4 100644 --- a/uv.lock +++ b/uv.lock @@ -53,6 +53,7 @@ dependencies = [ { name = "httpx" }, { name = "starlette" }, { name = "uvicorn" }, + { name = "zstandard" }, ] [package.metadata] @@ -60,6 +61,7 @@ requires-dist = [ { name = "httpx", specifier = ">=0.27" }, { name = "starlette", specifier = ">=0.37" }, { name = "uvicorn", specifier = ">=0.30" }, + { name = "zstandard", specifier = ">=0.23" }, ] [package.metadata.requires-dev] @@ -145,3 +147,60 @@ sdist = { url = "https://files.pythonhosted.org/packages/c4/1f/fa18009dea8469069 wheels = [ { url = "https://files.pythonhosted.org/packages/88/fa/e1388bbcf24ef3274f45c0c1c7b501fd14971037c1b6ee23610553307497/uvicorn-0.49.0-py3-none-any.whl", hash = "sha256:ba3d14c3ee7e41c6c654c46c9eb489d33213cdd30aa1696eab1374337c13f68f", size = 71376, upload-time = "2026-06-03T22:01:29.037Z" }, ] + +[[package]] +name = "zstandard" +version = "0.25.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/fd/aa/3e0508d5a5dd96529cdc5a97011299056e14c6505b678fd58938792794b1/zstandard-0.25.0.tar.gz", hash = "sha256:7713e1179d162cf5c7906da876ec2ccb9c3a9dcbdffef0cc7f70c3667a205f0b", size = 711513, upload-time = "2025-09-14T22:15:54.002Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/82/fc/f26eb6ef91ae723a03e16eddb198abcfce2bc5a42e224d44cc8b6765e57e/zstandard-0.25.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7b3c3a3ab9daa3eed242d6ecceead93aebbb8f5f84318d82cee643e019c4b73b", size = 795738, upload-time = "2025-09-14T22:16:56.237Z" }, + { url = "https://files.pythonhosted.org/packages/aa/1c/d920d64b22f8dd028a8b90e2d756e431a5d86194caa78e3819c7bf53b4b3/zstandard-0.25.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:913cbd31a400febff93b564a23e17c3ed2d56c064006f54efec210d586171c00", size = 640436, upload-time = "2025-09-14T22:16:57.774Z" }, + { url = "https://files.pythonhosted.org/packages/53/6c/288c3f0bd9fcfe9ca41e2c2fbfd17b2097f6af57b62a81161941f09afa76/zstandard-0.25.0-cp312-cp312-manylinux2010_i686.manylinux2014_i686.manylinux_2_12_i686.manylinux_2_17_i686.whl", hash = "sha256:011d388c76b11a0c165374ce660ce2c8efa8e5d87f34996aa80f9c0816698b64", size = 5343019, upload-time = "2025-09-14T22:16:59.302Z" }, + { url = "https://files.pythonhosted.org/packages/1e/15/efef5a2f204a64bdb5571e6161d49f7ef0fffdbca953a615efbec045f60f/zstandard-0.25.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6dffecc361d079bb48d7caef5d673c88c8988d3d33fb74ab95b7ee6da42652ea", size = 5063012, upload-time = "2025-09-14T22:17:01.156Z" }, + { url = "https://files.pythonhosted.org/packages/b7/37/a6ce629ffdb43959e92e87ebdaeebb5ac81c944b6a75c9c47e300f85abdf/zstandard-0.25.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:7149623bba7fdf7e7f24312953bcf73cae103db8cae49f8154dd1eadc8a29ecb", size = 5394148, upload-time = "2025-09-14T22:17:03.091Z" }, + { url = "https://files.pythonhosted.org/packages/e3/79/2bf870b3abeb5c070fe2d670a5a8d1057a8270f125ef7676d29ea900f496/zstandard-0.25.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:6a573a35693e03cf1d67799fd01b50ff578515a8aeadd4595d2a7fa9f3ec002a", size = 5451652, upload-time = "2025-09-14T22:17:04.979Z" }, + { url = "https://files.pythonhosted.org/packages/53/60/7be26e610767316c028a2cbedb9a3beabdbe33e2182c373f71a1c0b88f36/zstandard-0.25.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5a56ba0db2d244117ed744dfa8f6f5b366e14148e00de44723413b2f3938a902", size = 5546993, upload-time = "2025-09-14T22:17:06.781Z" }, + { url = "https://files.pythonhosted.org/packages/85/c7/3483ad9ff0662623f3648479b0380d2de5510abf00990468c286c6b04017/zstandard-0.25.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:10ef2a79ab8e2974e2075fb984e5b9806c64134810fac21576f0668e7ea19f8f", size = 5046806, upload-time = "2025-09-14T22:17:08.415Z" }, + { url = "https://files.pythonhosted.org/packages/08/b3/206883dd25b8d1591a1caa44b54c2aad84badccf2f1de9e2d60a446f9a25/zstandard-0.25.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:aaf21ba8fb76d102b696781bddaa0954b782536446083ae3fdaa6f16b25a1c4b", size = 5576659, upload-time = "2025-09-14T22:17:10.164Z" }, + { url = "https://files.pythonhosted.org/packages/9d/31/76c0779101453e6c117b0ff22565865c54f48f8bd807df2b00c2c404b8e0/zstandard-0.25.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:1869da9571d5e94a85a5e8d57e4e8807b175c9e4a6294e3b66fa4efb074d90f6", size = 4953933, upload-time = "2025-09-14T22:17:11.857Z" }, + { url = "https://files.pythonhosted.org/packages/18/e1/97680c664a1bf9a247a280a053d98e251424af51f1b196c6d52f117c9720/zstandard-0.25.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:809c5bcb2c67cd0ed81e9229d227d4ca28f82d0f778fc5fea624a9def3963f91", size = 5268008, upload-time = "2025-09-14T22:17:13.627Z" }, + { url = "https://files.pythonhosted.org/packages/1e/73/316e4010de585ac798e154e88fd81bb16afc5c5cb1a72eeb16dd37e8024a/zstandard-0.25.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:f27662e4f7dbf9f9c12391cb37b4c4c3cb90ffbd3b1fb9284dadbbb8935fa708", size = 5433517, upload-time = "2025-09-14T22:17:16.103Z" }, + { url = "https://files.pythonhosted.org/packages/5b/60/dd0f8cfa8129c5a0ce3ea6b7f70be5b33d2618013a161e1ff26c2b39787c/zstandard-0.25.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:99c0c846e6e61718715a3c9437ccc625de26593fea60189567f0118dc9db7512", size = 5814292, upload-time = "2025-09-14T22:17:17.827Z" }, + { url = "https://files.pythonhosted.org/packages/fc/5f/75aafd4b9d11b5407b641b8e41a57864097663699f23e9ad4dbb91dc6bfe/zstandard-0.25.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:474d2596a2dbc241a556e965fb76002c1ce655445e4e3bf38e5477d413165ffa", size = 5360237, upload-time = "2025-09-14T22:17:19.954Z" }, + { url = "https://files.pythonhosted.org/packages/ff/8d/0309daffea4fcac7981021dbf21cdb2e3427a9e76bafbcdbdf5392ff99a4/zstandard-0.25.0-cp312-cp312-win32.whl", hash = "sha256:23ebc8f17a03133b4426bcc04aabd68f8236eb78c3760f12783385171b0fd8bd", size = 436922, upload-time = "2025-09-14T22:17:24.398Z" }, + { url = "https://files.pythonhosted.org/packages/79/3b/fa54d9015f945330510cb5d0b0501e8253c127cca7ebe8ba46a965df18c5/zstandard-0.25.0-cp312-cp312-win_amd64.whl", hash = "sha256:ffef5a74088f1e09947aecf91011136665152e0b4b359c42be3373897fb39b01", size = 506276, upload-time = "2025-09-14T22:17:21.429Z" }, + { url = "https://files.pythonhosted.org/packages/ea/6b/8b51697e5319b1f9ac71087b0af9a40d8a6288ff8025c36486e0c12abcc4/zstandard-0.25.0-cp312-cp312-win_arm64.whl", hash = "sha256:181eb40e0b6a29b3cd2849f825e0fa34397f649170673d385f3598ae17cca2e9", size = 462679, upload-time = "2025-09-14T22:17:23.147Z" }, + { url = "https://files.pythonhosted.org/packages/35/0b/8df9c4ad06af91d39e94fa96cc010a24ac4ef1378d3efab9223cc8593d40/zstandard-0.25.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ec996f12524f88e151c339688c3897194821d7f03081ab35d31d1e12ec975e94", size = 795735, upload-time = "2025-09-14T22:17:26.042Z" }, + { url = "https://files.pythonhosted.org/packages/3f/06/9ae96a3e5dcfd119377ba33d4c42a7d89da1efabd5cb3e366b156c45ff4d/zstandard-0.25.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a1a4ae2dec3993a32247995bdfe367fc3266da832d82f8438c8570f989753de1", size = 640440, upload-time = "2025-09-14T22:17:27.366Z" }, + { url = "https://files.pythonhosted.org/packages/d9/14/933d27204c2bd404229c69f445862454dcc101cd69ef8c6068f15aaec12c/zstandard-0.25.0-cp313-cp313-manylinux2010_i686.manylinux2014_i686.manylinux_2_12_i686.manylinux_2_17_i686.whl", hash = "sha256:e96594a5537722fdfb79951672a2a63aec5ebfb823e7560586f7484819f2a08f", size = 5343070, upload-time = "2025-09-14T22:17:28.896Z" }, + { url = "https://files.pythonhosted.org/packages/6d/db/ddb11011826ed7db9d0e485d13df79b58586bfdec56e5c84a928a9a78c1c/zstandard-0.25.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bfc4e20784722098822e3eee42b8e576b379ed72cca4a7cb856ae733e62192ea", size = 5063001, upload-time = "2025-09-14T22:17:31.044Z" }, + { url = "https://files.pythonhosted.org/packages/db/00/87466ea3f99599d02a5238498b87bf84a6348290c19571051839ca943777/zstandard-0.25.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:457ed498fc58cdc12fc48f7950e02740d4f7ae9493dd4ab2168a47c93c31298e", size = 5394120, upload-time = "2025-09-14T22:17:32.711Z" }, + { url = "https://files.pythonhosted.org/packages/2b/95/fc5531d9c618a679a20ff6c29e2b3ef1d1f4ad66c5e161ae6ff847d102a9/zstandard-0.25.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:fd7a5004eb1980d3cefe26b2685bcb0b17989901a70a1040d1ac86f1d898c551", size = 5451230, upload-time = "2025-09-14T22:17:34.41Z" }, + { url = "https://files.pythonhosted.org/packages/63/4b/e3678b4e776db00f9f7b2fe58e547e8928ef32727d7a1ff01dea010f3f13/zstandard-0.25.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8e735494da3db08694d26480f1493ad2cf86e99bdd53e8e9771b2752a5c0246a", size = 5547173, upload-time = "2025-09-14T22:17:36.084Z" }, + { url = "https://files.pythonhosted.org/packages/4e/d5/ba05ed95c6b8ec30bd468dfeab20589f2cf709b5c940483e31d991f2ca58/zstandard-0.25.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:3a39c94ad7866160a4a46d772e43311a743c316942037671beb264e395bdd611", size = 5046736, upload-time = "2025-09-14T22:17:37.891Z" }, + { url = "https://files.pythonhosted.org/packages/50/d5/870aa06b3a76c73eced65c044b92286a3c4e00554005ff51962deef28e28/zstandard-0.25.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:172de1f06947577d3a3005416977cce6168f2261284c02080e7ad0185faeced3", size = 5576368, upload-time = "2025-09-14T22:17:40.206Z" }, + { url = "https://files.pythonhosted.org/packages/5d/35/398dc2ffc89d304d59bc12f0fdd931b4ce455bddf7038a0a67733a25f550/zstandard-0.25.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:3c83b0188c852a47cd13ef3bf9209fb0a77fa5374958b8c53aaa699398c6bd7b", size = 4954022, upload-time = "2025-09-14T22:17:41.879Z" }, + { url = "https://files.pythonhosted.org/packages/9a/5c/36ba1e5507d56d2213202ec2b05e8541734af5f2ce378c5d1ceaf4d88dc4/zstandard-0.25.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:1673b7199bbe763365b81a4f3252b8e80f44c9e323fc42940dc8843bfeaf9851", size = 5267889, upload-time = "2025-09-14T22:17:43.577Z" }, + { url = "https://files.pythonhosted.org/packages/70/e8/2ec6b6fb7358b2ec0113ae202647ca7c0e9d15b61c005ae5225ad0995df5/zstandard-0.25.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:0be7622c37c183406f3dbf0cba104118eb16a4ea7359eeb5752f0794882fc250", size = 5433952, upload-time = "2025-09-14T22:17:45.271Z" }, + { url = "https://files.pythonhosted.org/packages/7b/01/b5f4d4dbc59ef193e870495c6f1275f5b2928e01ff5a81fecb22a06e22fb/zstandard-0.25.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:5f5e4c2a23ca271c218ac025bd7d635597048b366d6f31f420aaeb715239fc98", size = 5814054, upload-time = "2025-09-14T22:17:47.08Z" }, + { url = "https://files.pythonhosted.org/packages/b2/e5/fbd822d5c6f427cf158316d012c5a12f233473c2f9c5fe5ab1ae5d21f3d8/zstandard-0.25.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4f187a0bb61b35119d1926aee039524d1f93aaf38a9916b8c4b78ac8514a0aaf", size = 5360113, upload-time = "2025-09-14T22:17:48.893Z" }, + { url = "https://files.pythonhosted.org/packages/8e/e0/69a553d2047f9a2c7347caa225bb3a63b6d7704ad74610cb7823baa08ed7/zstandard-0.25.0-cp313-cp313-win32.whl", hash = "sha256:7030defa83eef3e51ff26f0b7bfb229f0204b66fe18e04359ce3474ac33cbc09", size = 436936, upload-time = "2025-09-14T22:17:52.658Z" }, + { url = "https://files.pythonhosted.org/packages/d9/82/b9c06c870f3bd8767c201f1edbdf9e8dc34be5b0fbc5682c4f80fe948475/zstandard-0.25.0-cp313-cp313-win_amd64.whl", hash = "sha256:1f830a0dac88719af0ae43b8b2d6aef487d437036468ef3c2ea59c51f9d55fd5", size = 506232, upload-time = "2025-09-14T22:17:50.402Z" }, + { url = "https://files.pythonhosted.org/packages/d4/57/60c3c01243bb81d381c9916e2a6d9e149ab8627c0c7d7abb2d73384b3c0c/zstandard-0.25.0-cp313-cp313-win_arm64.whl", hash = "sha256:85304a43f4d513f5464ceb938aa02c1e78c2943b29f44a750b48b25ac999a049", size = 462671, upload-time = "2025-09-14T22:17:51.533Z" }, + { url = "https://files.pythonhosted.org/packages/3d/5c/f8923b595b55fe49e30612987ad8bf053aef555c14f05bb659dd5dbe3e8a/zstandard-0.25.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:e29f0cf06974c899b2c188ef7f783607dbef36da4c242eb6c82dcd8b512855e3", size = 795887, upload-time = "2025-09-14T22:17:54.198Z" }, + { url = "https://files.pythonhosted.org/packages/8d/09/d0a2a14fc3439c5f874042dca72a79c70a532090b7ba0003be73fee37ae2/zstandard-0.25.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:05df5136bc5a011f33cd25bc9f506e7426c0c9b3f9954f056831ce68f3b6689f", size = 640658, upload-time = "2025-09-14T22:17:55.423Z" }, + { url = "https://files.pythonhosted.org/packages/5d/7c/8b6b71b1ddd517f68ffb55e10834388d4f793c49c6b83effaaa05785b0b4/zstandard-0.25.0-cp314-cp314-manylinux2010_i686.manylinux_2_12_i686.manylinux_2_28_i686.whl", hash = "sha256:f604efd28f239cc21b3adb53eb061e2a205dc164be408e553b41ba2ffe0ca15c", size = 5379849, upload-time = "2025-09-14T22:17:57.372Z" }, + { url = "https://files.pythonhosted.org/packages/a4/86/a48e56320d0a17189ab7a42645387334fba2200e904ee47fc5a26c1fd8ca/zstandard-0.25.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:223415140608d0f0da010499eaa8ccdb9af210a543fac54bce15babbcfc78439", size = 5058095, upload-time = "2025-09-14T22:17:59.498Z" }, + { url = "https://files.pythonhosted.org/packages/f8/ad/eb659984ee2c0a779f9d06dbfe45e2dc39d99ff40a319895df2d3d9a48e5/zstandard-0.25.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2e54296a283f3ab5a26fc9b8b5d4978ea0532f37b231644f367aa588930aa043", size = 5551751, upload-time = "2025-09-14T22:18:01.618Z" }, + { url = "https://files.pythonhosted.org/packages/61/b3/b637faea43677eb7bd42ab204dfb7053bd5c4582bfe6b1baefa80ac0c47b/zstandard-0.25.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ca54090275939dc8ec5dea2d2afb400e0f83444b2fc24e07df7fdef677110859", size = 6364818, upload-time = "2025-09-14T22:18:03.769Z" }, + { url = "https://files.pythonhosted.org/packages/31/dc/cc50210e11e465c975462439a492516a73300ab8caa8f5e0902544fd748b/zstandard-0.25.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e09bb6252b6476d8d56100e8147b803befa9a12cea144bbe629dd508800d1ad0", size = 5560402, upload-time = "2025-09-14T22:18:05.954Z" }, + { url = "https://files.pythonhosted.org/packages/c9/ae/56523ae9c142f0c08efd5e868a6da613ae76614eca1305259c3bf6a0ed43/zstandard-0.25.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a9ec8c642d1ec73287ae3e726792dd86c96f5681eb8df274a757bf62b750eae7", size = 4955108, upload-time = "2025-09-14T22:18:07.68Z" }, + { url = "https://files.pythonhosted.org/packages/98/cf/c899f2d6df0840d5e384cf4c4121458c72802e8bda19691f3b16619f51e9/zstandard-0.25.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:a4089a10e598eae6393756b036e0f419e8c1d60f44a831520f9af41c14216cf2", size = 5269248, upload-time = "2025-09-14T22:18:09.753Z" }, + { url = "https://files.pythonhosted.org/packages/1b/c0/59e912a531d91e1c192d3085fc0f6fb2852753c301a812d856d857ea03c6/zstandard-0.25.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:f67e8f1a324a900e75b5e28ffb152bcac9fbed1cc7b43f99cd90f395c4375344", size = 5430330, upload-time = "2025-09-14T22:18:11.966Z" }, + { url = "https://files.pythonhosted.org/packages/a0/1d/7e31db1240de2df22a58e2ea9a93fc6e38cc29353e660c0272b6735d6669/zstandard-0.25.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:9654dbc012d8b06fc3d19cc825af3f7bf8ae242226df5f83936cb39f5fdc846c", size = 5811123, upload-time = "2025-09-14T22:18:13.907Z" }, + { url = "https://files.pythonhosted.org/packages/f6/49/fac46df5ad353d50535e118d6983069df68ca5908d4d65b8c466150a4ff1/zstandard-0.25.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:4203ce3b31aec23012d3a4cf4a2ed64d12fea5269c49aed5e4c3611b938e4088", size = 5359591, upload-time = "2025-09-14T22:18:16.465Z" }, + { url = "https://files.pythonhosted.org/packages/c2/38/f249a2050ad1eea0bb364046153942e34abba95dd5520af199aed86fbb49/zstandard-0.25.0-cp314-cp314-win32.whl", hash = "sha256:da469dc041701583e34de852d8634703550348d5822e66a0c827d39b05365b12", size = 444513, upload-time = "2025-09-14T22:18:20.61Z" }, + { url = "https://files.pythonhosted.org/packages/3a/43/241f9615bcf8ba8903b3f0432da069e857fc4fd1783bd26183db53c4804b/zstandard-0.25.0-cp314-cp314-win_amd64.whl", hash = "sha256:c19bcdd826e95671065f8692b5a4aa95c52dc7a02a4c5a0cac46deb879a017a2", size = 516118, upload-time = "2025-09-14T22:18:17.849Z" }, + { url = "https://files.pythonhosted.org/packages/f0/ef/da163ce2450ed4febf6467d77ccb4cd52c4c30ab45624bad26ca0a27260c/zstandard-0.25.0-cp314-cp314-win_arm64.whl", hash = "sha256:d7541afd73985c630bafcd6338d2518ae96060075f9463d7dc14cfb33514383d", size = 476940, upload-time = "2025-09-14T22:18:19.088Z" }, +]