Theming engine + BO-10 (one DB pool, non-blocking audit)

AGENTS.md

@@ -116,10 +116,28 @@ Copilot SDK sandboxes.
 - Tests in tests/unit/ and tests/integration/ -- pytest with asyncio
 - CI/CD via GitHub Actions: deploy.yml (push-to-deploy on main), pr-check.yml (lint+test on PRs)
 - Deploy target: Hostinger KVM 4 VPS (Ubuntu 24.04 + Docker)
-- **Control Plane UI**: All frontend work MUST follow `workbench/control_plane/DESIGN_SYSTEM.md`.
-  Use shared components (`Tabs`, `FilterPills`, etc.) from `src/components/` — never inline
-  ad-hoc tab bars, filter pills, or page headers. Use semantic Tailwind color tokens
-  (`bg-primary`, `text-foreground`, `border-border`) — never arbitrary hex values.
+- **Control Plane UI is THEMED. Read `workbench/control_plane/DESIGN_SYSTEM.md`
+  before writing any of it.** Settings → Appearance switches the whole org between
+  RapidTool, Fluent, Material and Graphite, which disagree about palette, corner
+  radius, icon pack, glass/glow and control behaviour (Material buttons are pills,
+  Graphite's labels are uppercase). Three rules, all machine-checked by
+  `src/lib/theme/conformance.test.ts`:
+  1. **Never write a colour.** Use `bg-primary`, `text-foreground`,
+     `border-border`, `var(--success)` — not `#0ea5e9`, `hsl(…)` or `bg-[#1a1b1e]`.
+     Text on a coloured fill takes the `-foreground` partner, never `text-white`.
+  2. **Never import `lucide-react`.** Use `<Icon name="Plus" />`; Lucide names are
+     the vocabulary, the theme picks the pack.
+  3. **Never hand-roll a control.** Use `Button`/`Input`/`Badge` from
+     `src/components/ui/` — a theme's state layer, focus ring and label transform
+     are not expressible in a class string, which is why the primitives exist.
+  Also use the shared `Tabs`, `FilterPills` and page-header patterns from
+  `src/components/` rather than inlining ad-hoc versions.
+- **Apps that run in the sandbox** (Custom Apps, generative UI, React artifacts)
+  inherit nothing from the shell — they get the `--cc-*` contract instead
+  (`src/lib/theme/app-tokens.ts`, documented in
+  `apps/agents/agent-app-builder/instructions.md`). Style with those tokens and
+  the app follows the org's theme for life; write one hex value and that part of
+  it leaves the design system permanently.
 - Agent-generated artefacts (images, reports, PDFs) MUST be written to
   `inputs/`, `outputs/`, or `agent-data/` within the agent workspace so the
   Control Plane file browser and inline chat cards can discover them.  These

FOUNDATION_BUILDOUT_CHECKLIST.md

@@ -34,11 +34,12 @@ added**:
 |---|---|---|---|
 | 1 | ~~**`main` has no branch protection**~~ **CLOSED 2026‑08‑03** | §BO‑17 / `work_plan.md` WS‑5 | Was 404 / rulesets `[]` for months. **Now enabled:** PRs required (`required_approving_review_count: 0` — a sole maintainer must still be able to land work), `enforce_admins: true` (without it the protection is decorative for the only person who pushes), force‑pushes and deletions **blocked**. ⚠️ **`required_status_checks` is deliberately `null`** — `pr-check.yml` carries `paths-ignore: ["**.md", "ai-company-brain/**"]`, so a docs‑only PR runs **no** checks at all; requiring those contexts would leave every docs PR permanently unmergeable. To require them, first give `pr-check` an always‑runs sentinel job, then add that job as the only required context. |
 | 2 | **No *scheduled* backup; no restore ever exercised** | **§BO‑23** (below) | ◐ 2026‑08‑03: `backup_db.sh` + `restore_db.sh` + runbook shipped, and `apply_migrations.sh` now fails closed without a pre‑migration dump. **But nothing schedules it** (the systemd units need a `deploy/` write) and **no restore has been run**. Measured recovery position meanwhile: Hostinger VM images only, weekly, **2 retained, newest 5 days old**, ~58 min restore, whole‑machine granularity. **OWNER‑GATE** — install the timer, then run one restore. |
-| 3 | **DB engine sprawl** | §BO‑10 | **12 `create_async_engine` call sites across 10 modules** (+ one sync engine), 8 of them undisposed process‑lifetime singletons. One arrived per app. **This is the only item whose cost compounds per app** — fix the seam before the next app, not after. |
+| 3 | ~~**DB engine sprawl**~~ **CLOSED 2026‑08‑06** | §BO‑10 | Was **12 `create_async_engine` call sites across 10 modules** (+ one sync engine), 8 of them undisposed process‑lifetime singletons, one arriving per app — the only item whose cost compounded per app. **Now one engine and one pool for every async caller**, in `acb_common/db.py` (not the gateway: `acb_auth.access` runs in the gateway process and cannot import it). `acb_audit.record()` is non-blocking on the loop, drained at shutdown. A new engine now fails `tests/unit/test_db_engine_seam.py`. Remaining by design: `acb_graph`'s **sync** engine and `email_ingestion`'s per‑run engines. |
 
 Nothing else on this list needs to be closed first. Items 1 and 2 are risk
-containment the owner must action; item 3 is the one an agent should fix before
-the next app opens engine number 13.
+containment the owner must action; item 3 was the one an agent should fix before
+the next app opened engine number 13 — **closed 2026‑08‑06, and the seam is now
+guarded by a test rather than by a note in this file.**
 
 ---
 
@@ -177,8 +178,17 @@ was stale — corrected 2026-08-02 to match §BO‑20) were anonymous‑reachabl
 - **Dependencies:** Alembic; a one‑time baseline of the current schema (`schema.generated.sql` exists as a start).
 - **Approach:** Adopt Alembic (autogenerate baselined against `schema.generated.sql`), run it in `lifespan`/entrypoint, keep the raw files as historical. Add a CI check for unique numeric prefixes until then.
 
-### BO‑10 — Consolidate DB access to one engine/pool *(P2)* ◑
-- **Done (Session 2, 2026‑07‑13):** **every** engine now bounds the CONNECT phase so a slow/unreachable DB can't hang callers — `settings.db_connect_timeout` (default 10s) on `acb_graph.get_engine()` (`ccccdc8`, live in prod), the two gateway asyncpg engines (`1684e1a`), and the four `email_ingestion` async engines (`1ff6c0d`, local, unpushed) via `connect_args={"timeout": …}`. This makes `acb_audit.record()`'s "never block the caller" guarantee real against a hung connect. Test: `tests/unit/test_db_connect_timeout.py`.
+### BO‑10 — Consolidate DB access to one engine/pool *(P2)* ✅ **CLOSED 2026‑08‑06**
+- **Closed (2026‑08‑06).** Every async caller now resolves to ONE engine and ONE pool, and `acb_audit.record()` no longer blocks the loop. What actually shipped, and the two places it departs from the "Approach" line written below in July:
+  - **The seam lives in `packages/acb_common/acb_common/db.py`, not in `acb_graph`.** `acb_graph` was the wrong home twice over: the gateway does not depend on it (nominating it would drag pgvector/AGE into a process that needs neither), and its own engine is **sync**. `acb_common` is the one package every service and every `acb_*` library already imports. `gateway/db.py` remains as a re-export because that is the import path the route packages use.
+  - **`acb_auth.access` is why the seam had to leave the gateway.** It resolves a member's permissions from Postgres *on the request path*, runs inside the gateway process, and cannot import `gateway`. While the seam lived in `gateway/db.py` the gateway had two pools no matter how many route packages were converted — so "one engine" was unreachable by converting routes alone. Its engine had also never carried the connect-phase or `idle in transaction` bounds added after the 2026‑08‑06 outage; it inherits both now.
+  - **Converted:** the six remaining route packages (`admin`, `apps`, `email`, `notes`, `whatsapp`, `workflows`) joining `tasks` and `crm`, plus `acb_auth.access`. Each kept its historical `get_db` / `_get_db` / `_get_session_factory` name as a re-export, so ~50 call sites and every `monkeypatch.setattr(<sibling>, "_get_db", …)` in the test suite are untouched. Verified live: nine consumers, one engine object, real queries.
+  - **Pool ceiling: 30** (`settings.db_pool_size` 10 + `db_max_overflow` 20, both now tunable), unchanged from the pre-consolidation seam. Deliberately *not* raised to the old sum: the twelve pools summed to ~165 connections from one process against a stock `max_connections` of 100 that Langfuse, LiteLLM and the ingestion services also draw from — a budget that could not be spent, only exceeded.
+  - **`dispose()` in the lifespan was considered and rejected**, contradicting the July approach line. The pool's lifetime *is* the process's; a dispose seam is a way to close connections other in-flight handlers are still using. BO‑9's "nothing disposes anything" observation is correct about the fact and wrong about the remedy for this engine.
+  - **`acb_audit.record()`** keeps its sync signature (25-odd call sites, most of them sync) and dispatches the write to `asyncio.to_thread` **only when called from a running event loop**; sync callers still write inline, which is what they expect. `acb_audit.drain()` is awaited last in the gateway lifespan, so shutdown cannot cancel an in-flight row — without it, non-blocking would have been a regression against the old behaviour, where the write completed before the handler returned.
+  - **Ratchet:** `tests/unit/test_db_engine_seam.py` fails the build on a new `create_async_engine` call site (AST-parsed, not grepped — every one of these modules mentions the name in prose saying it does *not* call it) and separately fails when an allowlist entry stops creating an engine, so the allowlist cannot rot into blanket permission. Plus `tests/unit/test_audit_non_blocking.py` (loop not blocked, write genuinely off-thread, sync callers still inline, drain waits, drain is bounded).
+- **Still open, deliberately:** `packages/acb_graph/acb_graph/db.py:32`'s **sync** `create_engine`. It serves a different (sync) caller set — including `acb_audit`'s own write — and folding it in is an `acb_graph` rewrite, not this ticket. The four `email_ingestion` engines also stay: separate process, per-run lifetime, disposed when the run ends. Both are recorded in the test's allowlist with those reasons.
+- **Done earlier (Session 2, 2026‑07‑13):** **every** engine now bounds the CONNECT phase so a slow/unreachable DB can't hang callers — `settings.db_connect_timeout` (default 10s) on `acb_graph.get_engine()` (`ccccdc8`, live in prod), the two gateway asyncpg engines (`1684e1a`), and the four `email_ingestion` async engines (`1ff6c0d`, local, unpushed) via `connect_args={"timeout": …}`. This makes `acb_audit.record()`'s "never block the caller" guarantee real against a hung connect. Test: `tests/unit/test_db_connect_timeout.py`.
 - **Missing — the "three+" above was written in July and is now materially wrong; re‑measured 2026‑08‑03.** It is **12 `create_async_engine(...)` call sites across 10 modules**, plus a 13th **sync** `create_engine` in `acb_graph/db.py:32`:
   | Module | Sites | Shape |
   |---|---|---|
@@ -194,8 +204,8 @@ was stale — corrected 2026-08-02 to match §BO‑20) were anonymous‑reachabl
   | `email_ingestion/scheduler.py:142, 527, 560` | 3 | per‑call, disposed at `:424`/`:540`/`:590` |
   | `packages/acb_graph/acb_graph/db.py:32` | (1 sync) | `create_engine`, a different flavour again |
   **The eight cached singletons are never disposed** — repo‑wide, the only `engine.dispose()` calls are the four `email_ingestion` per‑call engines cleaning up after themselves, and nothing in the gateway lifespan disposes anything (BO‑9). Also still open: sync `acb_audit.record()` blocks the async loop (H11) — connect_timeout bounds the hang but the call is still synchronous.
+  *(The table above is the 2026‑08‑03 measurement, kept as the record of what was found. Every async row in it is closed as of 2026‑08‑06 — see the top of this section for what replaced them and why the July approach line was not followed literally.)*
 - **Why it moved up the list:** the count grew by *one engine per app* — `notes`, `whatsapp`, `workflows` and `apps` all arrived with their own. This is the only foundation item whose cost **compounds per app**, so it is the one to fix before the next app rather than after (see `work_plan.md` §2's "Can we go app by app?" block, exception 3).
-- **Approach:** Provide a single configured async engine in `acb_graph` (sized pool), funnel all callers through it, dispose it in the gateway lifespan, and make `acb_audit.record()` async (or always call via `to_thread`).
 
 ### BO‑11 — Decide `acb_schemas`: wire in or delete *(P2)* ✅
 - **Done:** deleted the package (0 production importers, drifted from the ORM — H10). Removed its 7 `pyproject` dependency declarations + `tool.uv.sources` entry, the smoke‑test import, and the stale "wire/API surface" comment in `acb_graph/models.py`; re‑locked. Bonus: this exposed a latent under‑declared dependency — `orchestrator/triage/schema.py` uses pydantic `EmailStr` (needs `email‑validator`) but only got it transitively via `acb_schemas`; now declared explicitly as `pydantic[email]` on the orchestrator.
@@ -1674,7 +1684,7 @@ non‑blocking style backlog.
 
 1. **P0 hardening sprint (do first):** **BO‑23 (backup/restore — scripts + runbook are AGENT‑SAFE; it is P0 because it is the only unrecoverable failure mode here)**, BO‑8 (rotate+purge secrets), BO‑2 (auth enforcement — ✅ since), BO‑1 (Action Broker), BO‑3 (mutation governance). These close the Critical trust‑boundary and governance gaps that everything else sits on.
 2. **P1 sprint:** BO‑7 (sandbox), BO‑5 (observability+cost), BO‑6 (migrations), BO‑12/BO‑14 (runtime + permission model), BO‑15 (LLM config SoT — **tier→model half only**), BO‑17/BO‑18 (gates), **BO‑20 (event‑bus consumer + job queue)**. *(BO‑19 closed 2026‑08‑03.)*
-3. **P2/P3:** BO‑9, **BO‑10 (promoted in practice — it is the one item that compounds per app; see the verdict block at the top)**, BO‑11, BO‑13, BO‑16, **BO‑21 (memory activation)**.
+3. **P2/P3:** BO‑9, ~~BO‑10~~ **(✅ closed 2026‑08‑06 — one engine/pool + non-blocking audit; it was the one item that compounded per app)**, BO‑11, BO‑13, BO‑16, **BO‑21 (memory activation)**.
 
 **Competitive‑informed items** (proven reference implementations from Hermes Agent / OpenClaw — full mapping in `ai-company-brain/specs/competitive_hardening_2026-07.md`): CH‑1→BO‑7/BO‑14, CH‑2→BO‑1, CH‑3→BO‑20, CH‑4→WBS 3.3, CH‑5→BO‑12, CH‑6→BO‑21, CH‑7→Phase‑5 Annealer, CH‑8→BO‑5. These do not change the sequencing above — they attach a "what good looks like" reference to items we already have, plus the two new items (BO‑20/BO‑21) the comparison surfaced.