From 520476ab227232bb9519447251c89109c3421e69 Mon Sep 17 00:00:00 2001 From: Vijay Raghav Varada Date: Mon, 3 Aug 2026 12:34:18 +0530 Subject: [PATCH 1/2] =?UTF-8?q?docs(WS-0):=20truth=20pass=20across=20six?= =?UTF-8?q?=20workstreams=20=E2=80=94=20the=20board=20was=20describing=20a?= =?UTF-8?q?=20codebase=20from=20weeks=20ago?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thirteen of thirteen board rows have now failed the seven-point contract, always on the same point: no testable done-when. The six that had never been audited were audited together, and all six came back NO-GO. The cause turned out not to be missing work. It was that the specs describe a codebase from weeks ago, so they ask for things that already exist -- and in one case would have caused an implementer to break working code. WS-3 is the worst of it. The spec asserts, as "grep-confirmed", that the mutation container runs with zero --cap-drop/--memory/--cpus/--pids-limit flags. Four of those six have been present since 2026-07-27. An implementer dispatched on that row would most likely have re-added flags that already exist. The same spec never recorded that Tier 0 shipped, that a containerized Copilot CLI runtime shipped and is wired at two call sites behind a flag, or that isolation_tier() has been computed and thrown away into a log field since it landed. The rest of the pattern, row by row: WS-1 "Remaining: Zoho handlers" is fiction -- the Zoho client is read-only, six list_* calls and two GETs, with no write path anywhere to broker. Two real flip-blockers surfaced instead, neither previously written down: the two IRREVERSIBLE ClickUp actions (delete, archive) are gated but have no handler, so approving one marks the row failed; and a broker-queued push is written as sync_state='synced' with an empty provider_task_id. ACTION_BROKER_ENFORCE must not be flipped until both land, and the kill-switch's own docstring now says so. WS-8 manifest.py and declarative.py are built, tested and unwired -- roughly 60% of Phases A+B is already on disk. manifest.py's own docstring said "nothing here is wired into the run path yet", which was the single sentence most likely to cause a duplicate build. And "Phase A unblocks D3" is false in the direction that matters: config.json instancing already ships, so Centers C was never waiting on it. WS-11 Full-graph copilot authoring shipped as F14; the same document said so twice. Parallel fan-out shipped too. The real content is fan-IN and loops -- and both new tickets must INVERT a currently-pinned test, without which they close green having built nothing. WS-12 About 90% delivered elsewhere. A high-severity risk had retired itself: openai is already on 2.x, so Phase 4 drags one SDK major, not two. The row shrinks to the framework bump rather than closing. WS-21 Two of four deliverables already shipped, and two of the three gtd_time_blocks acceptance clauses were satisfiable by doing nothing. The "non-breaking TimeBlock[] swap" claim was false -- the real blast radius is 17 TS files plus four backend modules. All six specs now lead with what is actually built, carry per-item done-whens a command or an assertion can settle, name their verification commands by file (never tests/unit/ as a directory, which hangs against the live DB), and label every item AGENT-SAFE or OWNER-GATE. Two owner decisions recorded as D10. Command Center is an internal Fracktal tool, so WS-3's full run sandboxing is parked under a trusted-colleague threat model with an explicit un-parking condition rather than carried as debt. And loops in the workflow engine are approved against the anti-n8n rule, which is clarified to govern the node catalog rather than the control-flow vocabulary. Also: four rows added to the single-owner registry, six to the owner-gate registry, one wrong gate anchor fixed, two residuals closed and one opened. The Horizons ownership dispute between WS-21 and WS-18 is assigned rather than left to be rediscovered. Docs plus four docstrings. Verified docstring-only by AST comparison -- the first two attempts at that check were wrong because subprocess and open() were decoding with different encodings, which made every em-dash compare unequal. Verify: uv run ruff check . --select F821,F601,F602,F502,F7,B006 -> All checks passed! uv run pytest tests/unit/test_action_broker.py tests/unit/test_actions_routes.py \ tests/unit/test_provider_broker_gate.py tests/unit/test_task_broker_handlers.py \ tests/unit/test_agent_manifest.py tests/unit/test_declarative_builder.py -q -> 94 passed 23 WS rows, no duplicates, pipe counts uniform Co-Authored-By: Claude Opus 5 --- FOUNDATION_BUILDOUT_CHECKLIST.md | 81 ++- ai-company-brain/specs/agent_architecture.md | 342 ++++++++++- .../specs/agent_platform_hardening_2026-07.md | 284 +++++++-- ai-company-brain/specs/calendar_focus_os.md | 437 ++++++++++++-- ai-company-brain/specs/calendar_timeboxing.md | 225 +++++-- .../specs/multi_agent_orchestration.md | 560 ++++++++++++++---- .../specs/permissions_sandbox_b6.md | 502 ++++++++++++++-- ai-company-brain/specs/workflows_app.md | 122 +++- ai-company-brain/work_plan.md | 121 +++- .../action_broker/action_broker/broker.py | 17 +- .../gateway/gateway/routes/actions.py | 11 +- .../gateway/gateway/routes/tasks/providers.py | 9 +- packages/acb_skills/acb_skills/manifest.py | 6 +- 13 files changed, 2292 insertions(+), 425 deletions(-) diff --git a/FOUNDATION_BUILDOUT_CHECKLIST.md b/FOUNDATION_BUILDOUT_CHECKLIST.md index ce62c4e4a..ccc74c699 100644 --- a/FOUNDATION_BUILDOUT_CHECKLIST.md +++ b/FOUNDATION_BUILDOUT_CHECKLIST.md @@ -18,14 +18,79 @@ This is the list of foundational capabilities that are **missing, partially impl ## A. Security & trust boundaries ### BO‑1 — Action Broker: real approval‑gated write path *(P0)* ◑ -- **Done this pass (the decision + execution core, non‑breaking):** the 46‑line stub is now a real component: `decide_disposition(authority, destructive)` — the pure authority‑tier policy (READ→rejected, AUTONOMOUS→auto, SUGGEST→needs‑approval, SUGGEST_APPLY→auto for reversible / needs‑approval for destructive, i.e. FAIL CLOSED); `propose()` computes + audits the disposition (defaults `destructive=True`); and a **fail‑closed executor registry** (`register_action_handler` / `execute`) where a real source‑of‑truth write happens ONLY inside a registered handler and an action with no handler is REFUSED. Ships with **zero** handlers so it cannot write anything yet — inert + non‑breaking. 8 unit tests. -- **Persistence layer added (commit `e59cc6a`, additive, unpushed):** migration `66_pending_actions.sql` + `enqueue` / `list_pending` / `approve` / `reject` / `submit` in `broker.py` (17 unit tests, DB‑hermetic). No live path rerouted, so still inert. -- **Update 2026-08-01 (doc-truth pass):** the wiring the paragraph below described as missing **shipped 2026‑07‑13** (see `FOUNDATION_CONTINUATION.md`): `apps/services/action_broker/action_broker/broker.py` is a ~373‑line real component; the Control Plane approval inbox is bound via gateway `routes/actions.py`; real handlers are registered for ClickUp tasks (`routes/tasks/broker_handlers.py`), WhatsApp outbound (`routes/whatsapp/automation/outbound.py`), and workflows (`routes/workflows/broker_handlers.py`); the previously bypassing task writes route through the broker gate (`routes/tasks/providers.py`). **Remaining:** email/Zoho handlers + integration‑verify against a live Postgres. **OWNER‑GATE:** flipping `ACTION_BROKER_ENFORCE` (default OFF — every write auto‑applies, audited, zero behaviour change). -- **Missing (historical — resolved except as noted in the update above):** bind the Control Plane approval inbox to `approve`/`reject` (gateway `/actions` routes); register real handlers for ClickUp/email/Zoho; and route the existing bypassing writes (`routes/tasks/providers.py:365`, `email_ingestion/providers/*`) through `submit`. Plus **integration‑verify** the new SQL against a live Postgres. Until the wiring lands, either mark the write‑capable agents non‑autonomous or formally waive non‑negotiable #4. -- **Why needed:** It is non‑negotiable #4 ("no autonomous writes to source systems until the Action Broker is live") and the single control point for HITL over all outward writes. Today the guarantee is false. -- **Dependencies:** `03_pending_commits.sql`‑style queue table (add `pending_actions`); `acb_audit`; the Control Plane approval inbox; the auth fix (BO‑2) so approvals are authenticated. -- **Approach:** (1) Add a `pending_actions` table (proposal, actor, authority, payload, status, approved_by). (2) Make `propose()` enqueue and, per authority tier, either auto‑apply (read/idempotent), queue for approval, or reject. (3) Add an `execute(proposal)` that performs the provider write and is the *only* code path allowed to do so. (4) Route the existing ClickUp/email writes through it. (5) Reconcile docs with whichever model ships. -- **Note:** Until this lands, either mark the write‑capable agents (`agent_registry.json` sales/delivery/triage/billing) as **not** autonomous‑write, or accept and document that #4 is waived. + +> **Status: rewritten and verified against code 2026-08-03** (WS‑0 truth pass). Verified this pass by reading the source, not the prior doc: the six broker‑gated action names and their line numbers in `routes/tasks/providers.py`; the four entries in `broker_handlers._WRITERS`; the existence of `_raw_delete_task`/`_raw_archive_task`; `broker.execute()`'s no‑handler branch; **five** handler‑registration sites (the doc previously named three); `items.py`'s unconditional `sync_state='synced'` on both the parent and subtask push paths; `GtdItemModel.sync_state` being a bare `str`; the `sync_state` column being bare `TEXT` with no CHECK; the Zoho client being read‑only (`list_*` only, two `GET`s to `/crm/v2/*`, zero writes repo‑wide); the absence of any `action_broker` import under `email_ingestion/`; and the 14 outward‑write verbs on the email provider base class. Two bullets that claimed the broker is "inert" were **false since 2026‑07‑13** and are struck below; the same falsehood was live in three code docstrings and is corrected there in the same change. "Remaining: Zoho handlers" was **fiction** and is struck. §BO‑1 now carries three lettered tickets with acceptance criteria, a verification command, gate labels, and one named decision (BO‑1c). + +**What is true today (verified against code, 2026-08-03).** The broker is **live and writing**, not inert: +- `apps/services/action_broker/action_broker/broker.py` — the real component: `decide_disposition(authority, destructive)` (READ→rejected, AUTONOMOUS→auto, SUGGEST→needs‑approval, SUGGEST_APPLY→auto for reversible / needs‑approval for destructive, i.e. FAIL CLOSED); `propose()` computes + audits the disposition (defaults `destructive=True`); `register_action_handler` / `execute` is a fail‑closed executor registry — a source‑of‑truth write happens ONLY inside a registered handler, and an action with no handler is REFUSED (`broker.py:155-166`), never silently applied. Persistence (`enqueue` / `list_pending` / `approve` / `reject` / `submit`) landed with `66_pending_actions.sql` in commit `e59cc6a`. +- The Control Plane approval inbox is bound via gateway `routes/actions.py` (`GET /actions/pending`, `POST …/approve`, `POST …/reject`, all behind `require_internal_auth`). +- **Handlers are registered at FIVE sites, not three.** The complete list: + | Site | Registers | When | + |---|---|---| + | `gateway/main.py:983-985` → `routes/tasks/broker_handlers.register_task_broker_handlers()` | `clickup.create_task`, `clickup.update_task`, `clickup.create_project`, `clickup.create_folder` | startup | + | `gateway/main.py:1067-1069` → `routes/workflows/broker_handlers.register_handlers()` | `workflow.resume_run` | startup | + | `routes/whatsapp/scheduler_hooks.py:30` → `routes/whatsapp/automation/outbound.register_whatsapp_handlers()` | `whatsapp.broadcast` | startup | + | `routes/apps/tools.py:211` | `app.clickup_create_task` | module import | + | `routes/apps/tools.py:261` | `app.publish_review` | module import | +- The previously bypassing ClickUp task writes route through `BaseTaskProvider._broker_gate` (`routes/tasks/providers.py:129-175`), which reads `_broker_enforced()` (`:92-111`) at call time. **Build/flip are separable:** with `ACTION_BROKER_ENFORCE` unset the gate audits and returns `await do_write()` unchanged — zero behaviour change. Verified in code, so BO‑1a/b are safe to build with the kill‑switch off. + +**Struck as false (historical, do not act on):** +- ~~"Ships with **zero** handlers so it cannot write anything yet — inert + non‑breaking."~~ *(historical — untrue since 2026‑07‑13; see the five registration sites above.)* +- ~~"No live path rerouted, so still inert."~~ *(historical — untrue since 2026‑07‑13; ClickUp task writes, WhatsApp broadcast, workflow resume and app publish‑review all route through the broker.)* The same wording was live in `broker.py`, `routes/actions.py` and `routes/tasks/providers.py` docstrings and was corrected in the same change as this rewrite. +- ~~"Remaining: **Zoho** handlers."~~ **Fiction — struck.** There is no Zoho write path anywhere in the repo to route through the broker. `apps/services/ingestion/ingestion/sources/zoho/client.py` is read‑only: `list_accounts` / `list_deals` / `list_contacts` / `list_notes` / `list_tasks` / `list_users`, and the only CRM HTTP calls are `GET /crm/v2/{module}` (`:109`) and `GET /crm/v2/users` (`:152`) — repo‑wide grep for `crm/v2` returns exactly those two lines. (The one `http.post` at `:58` is the OAuth token refresh to the accounts host, not a CRM write.) The `"zoho.email"` example in `broker.py:55`'s docstring is illustrative, not a pointer to real code. **A Zoho broker handler is not BO‑1 work until a Zoho write client is specced and built elsewhere.** The recent Zoho *webhook* 500 fix is inbound‑only (WS‑4 / §BO‑20f) and has zero bearing here. +- **Stale anchor corrected:** the old text cited `routes/tasks/providers.py:365` as the bypassing write. That line is now ClickUp member/status parsing. The broker gate is `_broker_enforced` at `:92-111` and `_broker_gate` at `:129-175`. + +**Resolved / not a done‑when:** +- **Integration‑verify the queue SQL against a live Postgres — OWNER‑GATE, not agent work.** `66_pending_actions.sql` is committed to `main` and migrations auto‑apply on deploy, so prod almost certainly has the table; but `FOUNDATION_CONTINUATION.md:145` recorded this as still‑outstanding on 2026‑07‑13 and nothing since records it as executed. **No agent may claim this as done, and no agent may reach prod to do it.** It is not an acceptance criterion for BO‑1a/b/c — all three are hermetic. +- **Flipping `ACTION_BROKER_ENFORCE` — OWNER‑GATE.** Default OFF: every write auto‑applies, audited, zero behaviour change. **Do not flip it until BO‑1a and BO‑1b are both in**, for the reason BO‑1a names. + +#### BO‑1a — every gated action name has a registered handler *(AGENT‑SAFE · 1 small PR)* +**The bug, undocumented until now, and it is the destructive one.** `providers.py` routes **six** action names through `_broker_gate`: `clickup.create_task` (`:447`), `clickup.update_task` (`:523`), **`clickup.delete_task` (`:551`)**, **`clickup.archive_task` (`:575`)**, `clickup.create_project` (`:665`), `clickup.create_folder` (`:677`). `broker_handlers._WRITERS` (`:23-30`) registers **four**. So with `ACTION_BROKER_ENFORCE=all`, approving a queued delete or archive falls into `broker.execute()`'s no‑handler branch (`broker.py:155-166`), returns `{"ok": False, "error": "no handler registered…"}` and marks the row **`failed`** — **the two irreversible actions are exactly the two that cannot execute after approval.** `tests/unit/test_task_broker_handlers.py:103-112` asserts against a hard‑coded four‑element literal and is therefore structurally blind to this. + +**Done when:** +1. `_WRITERS` covers `clickup.delete_task` → `_raw_delete_task` (`providers.py:558`, args `("provider_task_id",)`) and `clickup.archive_task` → `_raw_archive_task` (`providers.py:583`, args `("provider_task_id", "archived")` — note the second arg; the gate's `audit_payload` must carry it). +2. `test_task_broker_handlers.py:103` is rewritten to **derive** the expected set from the `_broker_gate` call sites in `providers.py` rather than restating a literal, so any future gated action fails the test until it has a handler. +3. An end‑to‑end enqueue → approve → execute test for `clickup.delete_task` asserts the `pending_actions` row ends `applied`, not `failed`. +4. `uv run ruff check ` is clean. ⚠️ Do **not** write "`uv run ruff check .` clean" as a criterion — that command reports **1983 pre‑existing errors** on this tree (measured 2026-08-03, identical at `HEAD`), so it can never pass and is not a signal. Lint the paths you changed. + +**Non‑goals:** do not touch the apps‑tool broker surface (`routes/apps/tools.py`). It has the same structural shape — `_run_destructive_tool` proposes `_broker_action_name(tool)` for any `ToolSpec(destructive=True)`, and only `clickup.create_task` has a handler — but `_TOOL_REGISTRY` currently holds exactly one tool, which *is* handled, so there is no live hole. Note it; do not widen this PR. + +#### BO‑1b — a queued write never reports as synced *(AGENT‑SAFE · 1 medium PR)* +**The second flip‑blocker.** When `_broker_gate` queues instead of writing, it returns `{"pending": True, "pending_action_id": …, "provider_task_id": ""}` (`providers.py:171-172`). `items._push_pending_item` (`:1396-1413`) ignores the marker entirely and unconditionally sets `sync_state='synced', provider_task_id=''`. The subtask writer `_push_child_subtasks` (`:1546-1554`) does the same. Under enforcement, items would be marked **synced to nothing** — the user sees a green "synced" task that exists in no workspace. (The parent path's `if parent_tid:` guard at `:1416` does mean subtasks are skipped when the parent queues, so the subtask hole only fires when the *child* write is the one queued.) + +**DECISION (agent‑proposed, owner may overrule): the new value is `sync_state = 'awaiting_approval'`.** A third value is required because `'pending'` is already taken to mean "staged, awaiting the *user's* push" (`items.py:1361`, and the UI's Push button at `ItemDetail.tsx:711` keys off it) — reusing it would make the Push button reappear on a task already queued in the broker. `'awaiting_approval'` matches `Disposition.NEEDS_APPROVAL` and the `/actions` inbox's own language. **No migration:** `sync_state` is bare `TEXT DEFAULT 'local'` with no CHECK constraint (`infra/postgres/48_task_manager_gtd.sql:109`), and `GtdItemModel.sync_state` is a bare `str` (`routes/tasks/core.py:86`), so the value passes through the API unchanged. (Migration 48's inline comment already calls `'pending'` "queued push, Action‑Broker‑gated" — that comment is itself imprecise and should be corrected to distinguish the two states.) + +**Done when:** +1. With `ACTION_BROKER_ENFORCE=all` (set hermetically via `monkeypatch.setenv`), `POST /tasks/items/{id}/push` leaves `provider_task_id` NULL/unset and sets `sync_state='awaiting_approval'` — it never writes `'synced'`. +2. The subtask loop at `items.py:1537-1554` applies the same rule to a queued child. +3. `GtdItemModel` projects the value (verify — it should need no change) and `workbench/control_plane/src/app/tasks/lib/types.ts:178` widens its `syncState` union; `lib/api.ts:109` passes it through. +4. The tasks UI renders a **distinct badge** for it and does **not** offer Push (`ItemDetail.tsx:711`), per `workbench/control_plane/DESIGN_SYSTEM.md` — semantic tokens, no ad‑hoc hex. +5. Hermetic unit tests, no live DB, no migration. `uv run ruff check ` clean (see the BO‑1a note — repo‑wide ruff is not a signal). + +**Non‑goals:** no reconciliation job that later flips `awaiting_approval` → `synced` when the approval lands. The approve path already runs the handler; wiring its result back onto the `gtd_items` row is separate work and is **not** in this PR. + +#### BO‑1c — email handlers *(AGENT‑SAFE to build, but BLOCKED on the decision below · 1–2 medium PRs)* +Confirmed real remaining work: there is **zero** `action_broker` wiring anywhere under `apps/services/email_ingestion/` — every outward email write bypasses the broker today. But the ticket is not dispatchable until §BO‑1 names *which* verbs are broker actions, because the provider base class (`email_ingestion/providers/base.py`) exposes **14** mutating verbs: `send_message` (`:264`), `modify_message` (`:289`), `trash_message` (`:299`), `apply_flags` (`:307`), `move_to_folder` (`:322`), `bulk_apply` (`:339`), `create_folder` (`:387`), `create_filter` (`:398`), `delete_filter` (`:416`), `set_labels` (`:446`), `set_label_color` (`:484`), `create_draft` (`:492`), `update_draft` (`:521`), `send_draft` (`:551`). Brokering all 14 would put a human approval in front of every label click. + +**DECISION (agent‑proposed, owner may overrule) — broker the destructive/outward set only: `send_message`, `send_draft`, `trash_message`, `delete_filter`.** Rationale: these are the four that either leave the system (a recipient sees it) or destroy state a user cannot trivially restore. **Explicit non‑goal:** label, flag, folder‑move, draft‑create/update and filter‑create operations are **not** brokered — they are reversible, in‑mailbox, high‑frequency, and Command Center is an internal Fracktal tool used by trusted colleagues, so a per‑click approval would be pure friction with no trust gain. `bulk_apply` is a fan‑out over `move_to_folder`/`trash_message` (`base.py:339-386`), so it inherits the gate only through `trash_message`. + +**Done when (once the decision above is confirmed or overruled):** +1. The four chosen verbs route through a gate of the same shape as `BaseTaskProvider._broker_gate` — audit + auto‑apply by default, queue only under `ACTION_BROKER_ENFORCE`, and a broker‑layer error never blocks a user‑approved write. +2. Registered handlers exist for all four action names, so approving a queued proposal actually executes (the BO‑1a failure mode must not be reproduced here); the handler re‑resolves the account's credentials from the stored account id, and the token is never persisted in the proposal payload. +3. A test derives the expected handler set from the gate call sites, as in BO‑1a. +4. The non‑brokered verbs are asserted to be untouched by a test, so a later change cannot silently widen the gate. +5. Hermetic; `uv run ruff check ` clean (see the BO‑1a note — repo‑wide ruff is not a signal). + +**Verification (all three tickets):** +``` +uv run pytest tests/unit/test_action_broker.py tests/unit/test_actions_routes.py \ + tests/unit/test_provider_broker_gate.py tests/unit/test_task_broker_handlers.py -q +``` +Measured on this branch, 2026-08-03: **`32 passed in 1.95s`** — hermetic, no live DB, no network. This is the regression floor; each ticket adds to it. + +- **Why needed:** It is non‑negotiable #4 ("no autonomous writes to source systems until the Action Broker is live") and the single control point for HITL over all outward writes. The chokepoint now exists and is audited for ClickUp tasks, WhatsApp broadcast, workflow resume and app publish‑review; it does **not** yet cover email, and two of its own gated ClickUp actions cannot execute after approval (BO‑1a). +- **Dependencies:** `pending_actions` (exists, `66_pending_actions.sql`); `acb_audit`; the Control Plane approval inbox (exists, `routes/actions.py`); BO‑2 (authenticated approvals — ✅, the routes are behind `require_internal_auth`). +- **Note:** With enforcement OFF (the default) writes auto‑apply and are audited, so #4 is satisfied by audit-and-chokepoint rather than by human approval. That is the deliberate posture for an internal tool; flipping `ACTION_BROKER_ENFORCE` on is the OWNER‑GATE that turns it into a true HITL gate, and it must not be flipped before BO‑1a and BO‑1b land. - **Competitive ref (CH‑2):** Hermes Agent routes every risky action through a **single fail‑closed approval gate** — the pattern to copy is "one choke point that a write physically cannot bypass," which is exactly what `execute()`‑only‑writes enforces. See `specs/competitive_hardening_2026-07.md`. ### BO‑2 — Enforceable authentication + authorization *(P0)* ✅ diff --git a/ai-company-brain/specs/agent_architecture.md b/ai-company-brain/specs/agent_architecture.md index 829ca0704..1d9d5eff7 100644 --- a/ai-company-brain/specs/agent_architecture.md +++ b/ai-company-brain/specs/agent_architecture.md @@ -1,8 +1,14 @@ # Agent Architecture — how agents are declared, stored, and run -**Status:** Draft / RFC · **Date:** 2026-07-26 · **Owner:** vjvarada +**Status:** Active · **Date:** 2026-08-03 · verified against code on 2026-08-03 · **Owner:** vjvarada **Supersedes:** the distributed-repo framing in the 2026-07-26 first draft of this file. +> **Read §12.1 before writing any code against this spec.** Roughly 60% of what §12 calls +> Phases A + B is **already on disk as unwired substrate** — `manifest.py` is complete and +> partly wired, `declarative.py` is complete with zero callers, and all six `config.json` +> files already carry a `sharing` block. An implementer who starts from §5/§6 alone will +> rebuild working code. §12.1 is the inventory; §12.2 is the ticket list that assumes it. + How an agent is defined, what it can see at each layer, how its knowledge is authored, and how it gets permanently better — for agents that live **inside CommandCenter**: first-party agents in `apps/agents/`, and agents built in-platform by the upcoming **Agent Workshop** @@ -142,10 +148,18 @@ SDK isn't buying those three anything. It's VS Code-era scaffolding. > explaining it. Both have been corrected; the analysis below is retained because it is > the clearest example of *why* an agent's own factory must not outrank platform policy, > which is the argument for the declarative model. +> +> **Re-verified 2026-08-03:** all three Copilot factories now carry the *"No +> `on_permission_request` here: the executor injects the risk-aware…"* comment +> (`agent-apis-config/agents.py:49`, `agent-app-builder/agents.py:45`, +> `agent-task-manager/agents.py:118`) and no `approve_all` reference survives in +> `apps/agents/*/agents.py`. This finding is **closed**; only the startup check that +> would have *caught* it (§12.2 WS-8a) remains. `permissions_sandbox_b6.md` replaced `approve_all` with a risk-aware handler. The executor -applies it at five sites, all guarded the same way -(`executor.py:609, 2139, 2633, 3513, 4042`): +applies it at five sites, all guarded the same way — the guard lines are +`executor.py:634, 2485, 3011, 3909, 4444` (verified 2026-08-03; the call to +`_copilot_permission_handler()` sits at `:632, :2483, :3012, :3910, :4442` respectively): ```python if hasattr(_a, "_permission_handler") and _a._permission_handler is None: @@ -580,7 +594,7 @@ So MAF-only **deletes a duplicate HITL implementation** rather than needing a ne ### 11.1.1 Audit: what `/copilot/chat` still serves (2026-07-26) -**It is not a Copilot endpoint.** Despite the name, `main.py:421` builds +**It is not a Copilot endpoint.** Despite the name, `main.py:569` builds `build_orchestrator_agent()` — a **native MAF `Agent`** — and streams it through MAF's own AG-UI adapter (`agent_framework.ag_ui.AgentFrameworkAgent`). Its docstring says so outright: *"MAF orchestrator: per-request agent… The orchestrator is a native MAF agent."* The name is @@ -590,28 +604,37 @@ VS Code-era residue, and it has made the runtime split look larger than it is. (24 lines) says exactly why it was written: *"This thin wrapper lets the orchestrator go through the same `run_agent_stream()` path that all other named agents use, eliminating the separate `/copilot/chat` endpoint path in `main.py` and the `isOrchestrator` branching in -`route.ts`."* The migration was designed and half-built; the frontend still branches -(`route.ts:678` — `mode === "copilot" && isOrchestrator`). +`route.ts`."* The migration was designed and half-built; the frontend still branches — but +**not in `route.ts` any more**: the branch now lives at +`workbench/control_plane/src/components/AgentChat.tsx:336-337` +(`const isOrchestrator = currentAgentName === "orchestrator" || … ; const effectiveRuntime = +isOrchestrator ? currentRuntime : "copilot"`), with two further consumers at `:1914` and +`:1946`. `route.ts` no longer mentions `isOrchestrator` at all. -Only two things live solely on that path: +Of the two things that used to live solely on that path, **one is now closed**: -| Only on `/copilot/chat` | Status | +| Only on `/copilot/chat` | Status (verified 2026-08-03) | |---|---| -| `think_mode` → `_apply_thinking_mode` | `AgentRunRequest` has no such field. Must be ported before the branch is deleted. | -| `enrich_instructions_with_memory` | A **second, divergent** memory-injection implementation — see below. | +| `think_mode` → `_apply_thinking_mode` | ✅ **Ported.** `AgentRunRequest.think_mode` exists (`routes/agent.py:71`) with `_resolve_think_mode` (`:84`, also reading the nested `payload.think_mode` `route.ts` sends) and is passed to the executor at `:2016/:2024`; the executor applies it via `_apply_thinking_mode_for_agent` (`executor.py:2587-2595`). `main.py:546`'s `_apply_thinking_mode` is now a **thin delegate** to the single implementation in `orchestrator/_model_resolution.py:109`, and its own docstring says *"it disappears with `/copilot/chat`."* No longer a blocker. | +| `enrich_instructions_with_memory` | ❌ **Still open.** A **second, divergent** memory-injection implementation — see below. This is now A1's only remaining functional blocker. | ### 11.1.2 Finding: the orchestrator gets less memory than every other agent The two memory paths do not inject the same thing: -| Path | Injects | +| Path | Injects (verified 2026-08-03) | |---|---| -| `routes/agent.py:1291` `_build_memory_block` (named agents) | Mem0 **user** + Mem0 **`agent:`** + Mem0 **`org:global`** + Graphiti | -| `agents.py:498` `enrich_instructions_with_memory` (orchestrator) | Mem0 **user** + Graphiti | +| `routes/agent.py:1815` `_build_memory_block` (named agents) | Mem0 **user** + **`room:`** + **`prefs:`** + **`agent:`** + **`org:global`** + Graphiti | +| `orchestrator/agents.py:496` `enrich_instructions_with_memory` (orchestrator) | Mem0 **user** + Graphiti | So the orchestrator — the router that sees the most traffic — runs without agent-scope or org-scope memory. Company facts written via `save_org_memory` reach every named agent and not -the orchestrator. That looks unintentional rather than designed. +the orchestrator. That looks unintentional rather than designed. **The gap has widened since +this was first written:** the named-agent path gained the room and prefs compartments when the +multiplayer clearance work landed (2026-07-30), and the orchestrator path got neither. It does +not even consult a `Clearance` — `enrich_instructions_with_memory` takes `thread_id` only as a +Redis cache key, never as a compartment. Whatever room semantics `/agent/run/stream` enforces, +`/copilot/chat` does not. Two consequences: it is a live behaviour gap worth closing on its own, and it means the compartment/clearance work in [`memory_architecture.md`](memory_architecture.md) would @@ -672,28 +695,287 @@ consequence stated — rather than being a default nobody chose. ## 12. Phasing -| Phase | Work | Depends on | -|---|---|---| -| **A0** | Drop `approve_all` from the three factories — **already fixed 2026-07-26 (§3.2)**; A0's remaining scope is the startup check that `runtime` matches the entrypoint only | — | -| **A1** | **Single runtime (§11):** audit what `/copilot/chat` still serves that `/agent/run/stream` doesn't; retire the Copilot-native `on_user_input_request` path in favour of the existing `ask_tools` platform tools; make `code_task` the only Copilot entry point | A0 | -| **A** | Manifest schema + validator · `agent_defs`/`agent_def_versions`/`agent_def_grants` (migration: next free number at build time) · derive `dynamic_agents` from them · backfill all six | A0 | -| **B** | The one generic `build_declarative_agent` · migrate task-manager and apis-config · retire their `agents.py` | A | -| **C** | Agent Workshop UI — the describe-to-create flow, draft/publish/rollback, mirroring the App Workshop | B | -| **D** | Knowledge layer: migration (next free number at build time) · source-aware chunking · `kb/INDEX.md` always-on · KB-recall evals | A | -| **E** | Retrieval quality: hybrid + IDF + age decay · distillation on ingest and on memory extraction | D | -| **F** | Delegation modes + the clearance-intersection rule | A + multiplayer 3a | -| **G** | Promotion loop: State → Knowledge proposals into the approval inbox | D + multiplayer 3a | +The phase letters below are the **map**. The dispatchable unit is the lettered ticket in +§12.2 (`WS-8a`…`WS-8n`); each phase row names which tickets carry it. + +| Phase | Work | Tickets | Depends on | +|---|---|---|---| +| **A0** | `approve_all` removal — **done 2026-07-26 (§3.2)**; remaining scope is the startup check that `runtime` matches the entrypoint | WS-8a, WS-8b | — | +| **A1** | **Single runtime (§11):** retire `/copilot/chat` and the frontend's orchestrator branch; make `code_task` the only Copilot entry point | WS-8k, WS-8l, WS-8m | WS-8m (memory parity) | +| **A** | Manifest schema + validator · `agent_defs`/`agent_def_versions`/`agent_def_grants` (migration: next free number at build time) · derive `dynamic_agents` from them · backfill all six · make the manifest's derived accessors authoritative | WS-8c…WS-8h | A0 | +| **B** | The one generic `build_declarative_agent` · migrate task-manager and apis-config · retire their `agents.py` | WS-8i, WS-8j | A | +| **C** | Agent Workshop UI — the describe-to-create flow, draft/publish/rollback, mirroring the App Workshop | WS-8n (spec only) | B | +| **D** | Knowledge layer: migration (next free number at build time) · source-aware chunking · `kb/INDEX.md` always-on · KB-recall evals | *unticketed* | A + §13 Q1 | +| **E** | Retrieval quality: hybrid + IDF + age decay · distillation on ingest and on memory extraction | *unticketed* | D + §13 Q1 | +| **F** | Delegation modes + the clearance-intersection rule | *unticketed* | A + multiplayer 3a | +| **G** | Promotion loop: State → Knowledge proposals into the approval inbox | *unticketed* | D + multiplayer 3a | > **Update 2026-08-01 (doc-truth pass):** the "multiplayer 3a" dependency is partly shipped — > room compartments + clearance landed 2026-07-30 ✅; `subject:` compartments and the > compartment registry are still open. F and G block only on that open half. -A0 is a same-day fix. A–C are the Agent Workshop's critical path and don't depend on the -multiplayer work. +**D–G carry no acceptance criteria and are therefore not dispatchable.** They are named here +so the shape is visible; nobody should be sent at them until they are ticketed the way +§12.2 tickets A0/A1/A/B are. D and E additionally block on §13 Q1 (where a declarative +agent's KB lives) — that is a decision this spec does not record, not a build. + +**On Phase C's priority.** Command Center is an **internal Fracktal tool**: the team uses it, +there are no external tenants. So the Agent Workshop's describe-to-create flow is about +letting *colleagues* create agents, not about a public product surface. Its value is real but +bounded by headcount, and it is the largest unspecced surface in this document (§12.2 WS-8n +is a spec ticket, not a build ticket). **It should not outrank A0/A1/A/B**, all of which +either close a security gap, delete a duplicated code path, or stop a second implementation +of something that already exists. + +### 12.1 What is already built — read this before writing code + +Verified against the tree on **2026-08-03**. Roughly 60% of Phases A + B exists on disk. The +consistent shape is **complete, tested substrate with no production caller** — which reads as +"not built" to anyone who greps for the feature rather than for the module. + +#### Built and *wired* + +| Thing | Where | Wired at | +|---|---|---| +| `AgentManifest` + `SharingSpec` / `CapabilitySpec` / `MemorySpec` | `packages/acb_skills/acb_skills/manifest.py` | — | +| `AgentManifest.from_config()` → `.instance_key()` (the state-partition key `''` / `u:` / `t:`) | `manifest.py:167, :235` | `executor.py:917-937` (`_resolve_agent_instance`, called at `:1771` and `:2368`) and `gateway/routes/workspace.py:247-256` (`_agent_instance_for`) | +| A `sharing` block on **all six** first-party agents | `apps/agents/*/config.json` | consumed by the two sites above | +| `think_mode` on the named-agent path | `routes/agent.py:71, :84, :2016` → `executor.py:2587` → `_model_resolution.py:109` | live; `main.py:546` is now a delegate (§11.1.1) | + +The six `sharing` blocks as they actually ship: `apis-config`, `app-builder`, `orchestrator`, +`task-manager` = `instancing: shared, shareable: true`; `email-assistant`, +`whatsapp-assistant` = `instancing: personal, shareable: false`. All six declare +`visibility: organization` and `outputs_visibility: instance`. + +> **Consequence for the board:** **`config.json`-based instancing already ships.** The +> per-user / per-team partition that `agent-kinds.md` describes is live today for the blob +> store and the workspace file manager, derived from the manifest, with no schema change. +> **WS-14 is therefore NOT waiting on this spec's Phase A** — the board has claimed otherwise +> for weeks. See §12.5. + +#### Built and *not wired* — the duplicate-build hazard + +| Thing | Where | Production callers | +|---|---|---| +| `AgentManifest.validate()` / `.warnings()` | `manifest.py:305, :371` | **zero** | +| `AgentManifest.memory_scope()` (`agent:#`) | `manifest.py:248` | **zero** — the compartment key is still built from the bare agent name at `acb_memory/compartments.py:160` | +| `AgentManifest.blob_instance()` | `manifest.py:259` | **zero** — callers use `instance_key()` directly | +| `AgentManifest.resolve_tool_surface()` | `manifest.py:263` | **zero** — deliberately identical to `_tool_injection._resolve_injected_scope` (`_tool_injection.py:183`), which still owns the computation | +| `AgentManifest.isolation_tier()` | `manifest.py:277` | one, and only for a log field (`declarative.py:210`) | +| `SharingSpec.shareable` / `is_shareable()` | `manifest.py:103, :293` | **zero anywhere in the repo** — no code, TS, or SQL reads `shareable` outside `manifest.py` itself. Room eligibility is not manifest-derived; it is not derived at all. | +| `build_declarative_agent` + `resolve_skill_tools` + `load_instructions` | `apps/services/orchestrator/orchestrator/declarative.py` | **zero.** Complete, documented, tested. `task-manager` and `apis-config` still ship Copilot factories. | + +> `manifest.py`'s own module docstring said *"Nothing here is wired into the run path yet"* +> until 2026-08-03. That was false — `from_config` + `instance_key` are on the run path. The +> line has been corrected to name exactly which accessors are live and which are not, because +> an implementer who believed it would have rebuilt the module. + +#### Genuinely not built + +- `agent_defs` / `agent_def_versions` / `agent_def_grants` — **no migration exists.** Zero + matches for those names in `infra/postgres/*.sql`. (Find the next free number at build + time by listing the migrations directory; do not copy a number out of this document.) +- Deriving `dynamic_agents` from `agent_defs`. +- The runtime-vs-entrypoint check (`agent.runtime_mismatch` matches nothing in the tree). +- MCP-on-MAF (§12.2 WS-8c). +- `/build/agents` — `workbench/control_plane/src/app/build/` contains **only** `apps/`. +- Everything in Phases D–G. + +### 12.2 Tickets + +Every ticket carries a **gate label** (`work_plan.md` §6 vocabulary: **AGENT-SAFE** work may +be dispatched to an unattended agent; **OWNER-GATE** work may not, and an agent must refuse it +and say which gate). Every ticket carries a **done-when** that a command or an assertion can +settle. + +#### A0 — enforce the runtime declaration + +**WS-8a — runtime/entrypoint check, log-only. AGENT-SAFE once the owner picks a mode (WS-8b).** +Done when `load_agent()` (`packages/acb_skills/acb_skills/loader.py:1434`) compares +`config.json`'s `runtime` against the type the factory actually returned and emits +`agent.runtime_mismatch` (agent, declared, actual); a test loads a fixture agent declaring +`maf` whose factory returns a Copilot agent and asserts the record; a second asserts the three +real Copilot agents (`apis-config`, `app-builder`, `task-manager`) are detected; and the run +**still proceeds** (log-only). Detect by capability, not by the registry label — `executor.py:2452` +already documents why: *"Detect Copilot-SDK-backed agents by capability, NOT the registry +runtime label… A genuine MAF agent has no `_default_options`."* + +**WS-8b — make the check fail-closed. OWNER-GATE.** Done when a mismatched agent refuses to +load. **Refuse to build this without an explicit owner decision**: three of six agents declare +`maf` and return `GitHubCopilotAgent`, so fail-closed refuses half the roster at startup. The +gate is the owner choosing between (i) fail-closed after WS-8i/WS-8j migrate the drifted +agents, or (ii) fail-closed now with the three configs corrected to `"runtime": "copilot"`, +which contradicts AGENTS.md Global Constraint #6. + +#### A — make the manifest authoritative + +**WS-8c — MCP injection is a silent no-op on native MAF. AGENT-SAFE.** `work_plan.md` D7 +instructs *"scope it into WS-8 Phase A/B"*; that instruction was never carried into this spec +until 2026-08-03. `_inject_mcp_servers` (`_tool_injection.py:1086`) runs for every agent and +routes to `merge_mcp_servers` (`:1056`), which writes `agent._mcp_servers` (`:1081`). The +**only** readers of that attribute are `copilot_agent.py:169` and `:276` — both inside the +Copilot agent class. No `MCPStdioTool` / `MCPStreamableHTTPTool` wiring exists anywhere in the +tree. So for a native-MAF agent, MCP injection silently does nothing, while +`CapabilitySpec.mcp_servers` (`manifest.py:122`) already accepts the field and §6 promises it +works. Done when a native MAF agent with a declared MCP server exposes that server's tools in +its tool surface; a test asserts a MAF agent built with one declared MCP server has the +server's tools attached (and that a `manifest.capabilities.mcp_servers` entry that cannot be +resolved is logged rather than silently dropped); and `grep -rn "_mcp_servers" apps/` shows a +non-Copilot reader. + +**WS-8d — enforce `validate()` / `warnings()` at registration. AGENT-SAFE.** Done when agent +registration calls `AgentManifest.from_config(...).validate()` and `.warnings()`, logs each +problem with the agent name, and a test asserts a config with `instancing: "team"` and no +`sharing.team` produces a recorded problem while all six real `apps/agents/*/config.json` +files produce `validate() == []`. Report-and-continue only; raising is a separate decision. +**Second half — retire the duplicate tool-surface computation:** +`_tool_injection._resolve_injected_scope` (`_tool_injection.py:183`) and +`AgentManifest.resolve_tool_surface` (`manifest.py:263`) compute the same thing, and +`test_agent_manifest.py` already asserts they agree. Done when one of them delegates to the +other (the manifest is the intended owner) and that equivalence test still passes — closing +row 3 of §12.3. + +**WS-8e — the `agent_defs` schema. AGENT-SAFE.** Done when a migration (next free number, +found by listing `infra/postgres/` at build time — **never** copied from a doc) creates +`agent_defs`, `agent_def_versions` and `agent_def_grants` as §5 specifies, `schema.generated.sql` +is regenerated, and a test asserts the three tables exist with the `UNIQUE (agent_id, version)` +and `PRIMARY KEY (agent_id, subject)` constraints. + +**WS-8f — derive `dynamic_agents` from `agent_defs` + backfill the six. AGENT-SAFE.** Done +when every row in `dynamic_agents` for a first-party agent is projected from an `agent_defs` +row, `GET /agents` returns the same six agents with the same names before and after, and a +test asserts the projection is idempotent (running the backfill twice changes nothing). +**Also closes §12.3 row 7:** the hardcoded `agent_name in ("orchestrator", "default")` branch +at `gateway/routes/workspace.py:220` is replaced by a property of the resolved registry row +(does this agent have a clone directory?) rather than a literal name list. Depends on WS-8e. + +**WS-8g — the memory compartment key comes from the manifest. OWNER-GATE.** Today +`acb_memory/compartments.py:160` computes `scope_key(agent=agent_name)` → `agent:` with +no instance suffix, while `AgentManifest.memory_scope()` (`manifest.py:248`) computes +`agent:#` and has zero callers. Flipping the computation is **not** a no-op: +`email-assistant` and `whatsapp-assistant` declare `instancing: "personal"`, so their +compartment moves from `agent:email-assistant` to `agent:email-assistant#u:` and every +fact already written under the old key becomes unreachable. **The gate is the owner choosing +between migrating those rows, dual-reading during a window, or leaving those two agents +shared.** Done when the decision is recorded here, the flip is implemented behind it, and a +test asserts a `shared` agent's key is byte-identical to today's. + +**WS-8h — room eligibility comes from `sharing.shareable`. AGENT-SAFE.** Nothing in the repo +reads `shareable` today, so this adds a gate where there is none. Done when the "share this +session" affordance is enabled only for an agent whose manifest returns +`is_shareable() == True`, a test asserts a `personal`-instanced agent with `shareable: false` +(i.e. `email-assistant`, `whatsapp-assistant`) is refused, and a test asserts a `shared` +agent is allowed. + +#### B — one builder + +**WS-8i — migrate `task-manager` onto `build_declarative_agent`. AGENT-SAFE.** The builder +already exists (`orchestrator/declarative.py`) and +`tests/unit/test_declarative_builder.py` already asserts the 29 callables `skill-task-gtd` +exports are exactly the 29 tools `agent-task-manager/agents.py` assembles by hand. **Do not +rewrite the builder.** Done when `agent-task-manager/agents.py` no longer imports +`agent_framework_github_copilot`, the agent is built by `build_declarative_agent` from its +manifest, `test_declarative_builder.py` still passes, and a test asserts the resulting tool +list equals the pre-migration list. Blocked on the design question §12.4 records: the drifted +agents' dependence on Copilot-native file tools (`code_tools.py:17`). + +**WS-8j — migrate `apis-config` onto `build_declarative_agent`. AGENT-SAFE.** Same done-when, +same blocker. Sequence after WS-8i so the first migration carries the risk alone. + +#### A1 — one runtime + +**WS-8m — close the orchestrator's memory gap (§11.1.2). AGENT-SAFE, but owned by WS-15/D4 — +check the board before starting.** Done when `/copilot/chat`'s memory injection reads the same +compartments as `/agent/run/stream` (user + room + prefs + agent + org + Graphiti) and a test +asserts both paths request the same scope-key set for the same run. This is A1's only +remaining functional blocker; `think_mode` is already ported (§11.1.1). + +**WS-8k — retire the `/copilot/chat` backend. AGENT-SAFE after WS-8m.** Done when +`main.py:569`'s `/copilot/chat` handler is deleted, the orchestrator runs through +`run_agent_stream()` via `apps/agents/agent-orchestrator/agents.py` (the 24-line wrapper +written for exactly this), `grep -rn "copilot/chat" apps/` returns nothing outside comments, +and `_apply_thinking_mode` is deleted from `main.py` (its own docstring says *"it disappears +with `/copilot/chat`"*). **Large blast radius — this is the primary chat surface.** + +**WS-8l — delete the frontend orchestrator branch. AGENT-SAFE after WS-8k.** Done when +`grep -rn "isOrchestrator" workbench/control_plane/src/` returns nothing, `AgentChat.tsx:336-337`'s +`effectiveRuntime` ternary is gone, the two UI consumers at `:1914` and `:1946` are resolved, +and `npx tsc --noEmit` + the vitest suite are clean. + +#### C — Agent Workshop + +**WS-8n — specify the Agent Workshop before building it. AGENT-SAFE (a spec, not a build).** +Phase C's entire content in this document is one phase-table row and the §10 lifecycle +diagram. Nothing is dispatchable from that. Done when this spec (or a child spec under +`ai-company-brain/specs/`) carries: the `/build/agents` route inventory, the describe-to-create +conversation contract, the draft→validate→eval→publish→rollback state machine mapped onto +`agent_defs.status` / `live_version`, who may publish (§13 Q2), and per-slice acceptance — +each stated the way the App Workshop's `docs/app-workshop/README.md` states its own, since +that is the precedent §5 says to copy exactly. **Do not open `/build/agents` until §13 Q2 is +answered and WS-3's `tool_scope` deny-by-default has landed** — the Workshop is precisely the +surface that lets a non-engineer create an agent with an absent `tool_scope`, which +`manifest.py:359` already flags as a privilege grant nobody made. + +### 12.3 Acceptance for Phase A + +The previous wording was an **unbounded universal negative** — *"…no compartment key, blob +instance, tool surface, permission handler, or room eligibility is computed from a hardcoded +agent name anywhere in the codebase"* — which no implementer can ever prove. It is replaced by +this enumeration. Phase A is accepted when **every row is Yes**, each demonstrated by the +named assertion. + +| # | Derivation | Site that owns it today | Manifest accessor that should own it | State (2026-08-03) | Ticket | +|---|---|---|---|---|---| +| 1 | Blob / workspace instance | `executor.py:917-937`; `gateway/routes/workspace.py:247-256` | `instance_key()` | ✅ **already manifest-derived** | — | +| 2 | Memory compartment key | `acb_memory/compartments.py:160` (`scope_key(agent=agent_name)`) | `memory_scope()` | ❌ bare agent name | WS-8g | +| 3 | Injected tool surface | `_tool_injection.py:183` (`_resolve_injected_scope`) | `resolve_tool_surface()` | ❌ two implementations, asserted equal by `test_agent_manifest.py`, neither retired | WS-8d | +| 4 | Isolation tier | `manifest.py:277`, log field only (`declarative.py:210`) | `isolation_tier()` | ❌ not consulted by any isolation decision | WS-3 owns the ladder; this spec owns the derivation | +| 5 | Room / share eligibility | *nothing* | `is_shareable()` | ❌ not computed anywhere | WS-8h | +| 6 | Permission handler | `executor.py:634, 2485, 3011, 3909, 4444` | `permissions.mode` (§6) | ⚠️ **not name-keyed** — the guard probes `_permission_handler is None`, a Copilot-capability test. Correct as-is; listed so nobody "fixes" it. | — | +| 7 | Workspace fallback | `gateway/routes/workspace.py:220` — `agent_name in ("orchestrator", "default")` | — | ❌ a genuine hardcoded-name branch | WS-8f | +| 8 | Runtime | `config.json` `runtime`, unchecked; `executor.py:2452` detects by capability instead | validated constant (§11.2) | ❌ decorative | WS-8a | + +**Done-when for the phase:** rows 2, 3, 5, 7 and 8 flip to Yes, each with the test its ticket +names, and `python -m pytest tests/unit/test_agent_manifest.py tests/unit/test_declarative_builder.py tests/unit/test_agent_paths.py -q` +stays green. + +### 12.4 Verification + +```bash +python -m pytest tests/unit/test_agent_manifest.py tests/unit/test_declarative_builder.py tests/unit/test_agent_paths.py -q +``` + +Real result on 2026-08-03, `ws-0-truth-pass-six-rows` @ `2ccff9e0`: + +``` +........................................................................ [ 92%] +...... [100%] +78 passed in 2.47s +``` + +(`test_agent_manifest.py` + `test_declarative_builder.py` alone: `62 passed in 1.53s`.) + +These suites are the regression floor for every ticket in §12.2 — they assert that the +manifest's derived values **already equal** what the platform computes today, which is what +makes each flip in §12.3 provable rather than hopeful. Any ticket that changes a derivation +must keep them green or state in its PR which assertion it intentionally changed and why. + +Open design questions a ticket must not silently decide: + +- **The drifted agents' dependence on Copilot-native file tools** (`code_tools.py:17` notes + those bypass the durability mirror and are specially handled). Blocks WS-8i/WS-8j. +- **§13 Q1** — where a declarative agent's KB lives. Blocks D and E. +- **§13 Q2** — who may publish. Blocks WS-8n. + +### 12.5 Out of this row's orbit -**Acceptance for A:** every agent's runtime behaviour is derivable from its manifest alone — -no compartment key, blob instance, tool surface, permission handler, or room eligibility is -computed from a hardcoded agent name anywhere in the codebase. +**The `dynamic_agents` sharing columns (`instancing` / `visibility` / `team_ref` / +`memory_mode` / `shareable`) are not this spec's work.** `schema.generated.sql:1853` shows +`dynamic_agents` with 13 columns and none of those five; +`docs/multiplayer/agent-kinds.md:148-151` already reassigns that migration to +`department_centers.md` Phase C / **WS-14**, and that reassignment is correct. Because +instancing already ships from `config.json` via `AgentManifest.instance_key()` (§12.1), +**WS-14 is not blocked on this spec's Phase A.** --- diff --git a/ai-company-brain/specs/agent_platform_hardening_2026-07.md b/ai-company-brain/specs/agent_platform_hardening_2026-07.md index ba3dc8670..70b87cb69 100644 --- a/ai-company-brain/specs/agent_platform_hardening_2026-07.md +++ b/ai-company-brain/specs/agent_platform_hardening_2026-07.md @@ -1,9 +1,34 @@ # Agent Platform Hardening Review — 2026-07 -**Status:** Review · **Date:** 2026-07-26 · **Owner:** vjvarada +**Status:** Review · **Verified against code on 2026-08-03** · **Owner:** vjvarada **Scope:** The multiplayer room model, the memory/clearance model, and the agent architecture — reviewed together, because most of what follows only appears where two of them meet. +> **Truth pass 2026-08-03 (WS-3, ws-0 truth-pass batch).** What changed in this +> doc, all re-verified against the tree at `2ccff9e0`: +> - **§1.2's T0/T1/T2 is now the single isolation ladder of record** for the +> platform. `permissions_sandbox_b6.md`'s Phase-5 "Tier 0/1/2/3" was a second, +> *incompatible* numbering for the same board cell (WS-3) and has been renamed +> to **P5-a/b/c/d** there (R2 — no phase-ID reuse across docs). Use T0/T1/T2 +> when you mean isolation strength; use P5-a…d when you mean B6's build order. +> - **§1.2's ladder is implemented and thrown away.** `AgentManifest.isolation_tier()` +> (`packages/acb_skills/acb_skills/manifest.py:273-287`) computes exactly this +> table and is pinned by `tests/unit/test_agent_manifest.py:224-252`. Its only +> consumer is a structured **log field** — `declarative.py:210`'s +> `_log.info("declarative.agent_built", …, tier=manifest.isolation_tier())` — +> plus a registration warning (`manifest.py:367-374`). Nothing records it, +> nothing enforces it, and `agent_run` has **no `tier` column** (checked +> `infra/postgres/`, highest migration on disk is 142). Recording + refusal is +> dispatchable as **WS-3a** (`permissions_sandbox_b6.md` §P5-a). +> - **§1.1's quoted code block was dead** — `_resolve_injected_scope` no longer +> has that body or that signature. Replaced with the current source. +> - **C1 is built, and it is not this spec's work** — see the C1 update below. +> - **Threat model restated (owner decision, 2026-08-03):** Command Center is an +> **internal Fracktal tool**. The team uses it; there are no external tenants. +> The ladder must hold up to **trusted colleagues, not hostile users**, which +> moves T2 from "before the Agent Workshop opens" to a **deprioritised +> sub-project**. See §1.3 and §1.5. + **Reviews:** [`agent_architecture.md`](agent_architecture.md) · [`memory_architecture.md`](memory_architecture.md) · @@ -19,22 +44,52 @@ `agent_architecture.md` §13.4 asked whether declarative agents need a sandbox at all, and leaned no: *"they execute no custom code."* That reasoning doesn't survive contact with -`_tool_injection.py`: +`apps/services/orchestrator/orchestrator/_tool_injection.py:183-255`: ```python -def _resolve_injected_scope(tool_scope: list[str] | None) -> set[str] | None: - """Returns None when there is no tool_scope (inject everything), or the - set of allowed names = the agent's tool_scope UNIONed with the core floor.""" - if not tool_scope: - return None # ← inject everything +# _tool_injection.py:183-226 (current source, 2026-08-03 — the block quoted +# here originally showed a one-argument function that no longer exists) +def _resolve_injected_scope( + tool_scope: list[str] | None, + *, + disabled_families: frozenset[str] | None = None, +) -> set[str] | None: + ... + if tool_scope: + base: set[str] | None = ( + set(tool_scope) | set(_CORE_STANDARD_TOOL_NAMES) + ) + elif _skills_fail_closed(): + # WS-23 S3 (owner-gated, OFF by default): unscoped agents get the + # named DEFAULT_PROFILE instead of everything. + ... + base = set(_CORE_STANDARD_TOOL_NAMES) | set(default_profile_tools()) + else: + base = None # ← inject everything (still the shipped default) ``` +The finding stands, with one correction: the fail-open `None` is now **one branch of +three**, and the deny branch exists behind `SKILLS_FAIL_CLOSED` (`_tool_injection.py:101-117`) +— shipped **OFF**, and flipping it is OWNER-GATE (`work_plan.md` §6). So the *mechanism* +this section asks for is built; the *posture* is unchanged until the owner flips it. + **An agent with no `tool_scope` gets the entire platform tool surface**, which includes `code_task` and `run_script` — arbitrary shell in the agent workspace. A declarative agent holding `run_script` is exactly as dangerous as a code agent. The isolation boundary is the **resolved tool surface**, not how the agent was authored. -### 1.2 The decision: capability-tiered isolation, derived from the manifest +### 1.2 The decision: capability-tiered isolation, derived from the manifest — **the ladder of record** + +> **This table is the platform's single isolation ladder** (R2). Any doc that +> numbers isolation strength — `permissions_sandbox_b6.md`, `FOUNDATION_BUILDOUT_CHECKLIST.md` +> §BO‑7, `competitive_hardening_2026-07.md` CH‑1 — refers to **T0/T1/T2** and adds +> nothing. B6's own build order is lettered **P5-a/b/c/d** precisely so the two +> never collide again. +> +> **This is a trigger definition, not a definition of done.** The "Build cost" +> column is an estimate, not acceptance. Per-slice acceptance lives in +> `permissions_sandbox_b6.md` §P5-a (**WS-3a**) and §P5-b (**WS-3b**); T2 has +> none and deliberately gets none while it is parked (§1.5). Three tiers, computed at run start from the *resolved* surface (manifest ∩ grants), recorded on `agent_run`, and enforced by the executor. @@ -43,28 +98,72 @@ on `agent_run`, and enforced by the executor. |---|---|---|---| | **T0 — in-process** | Read-only platform tools + LLM. No file write outside the workspace, no shell, no open-world network. | Today's `importlib` path, unchanged. | none | | **T1 — confined in-process** | File writes, declared integrations, MCP servers. No shell, no eval. | Same process plus: workspace-confined FS (`resolve_in_workspace` already does this), egress allowlist limited to declared integrations, per-run wall-clock and memory caps. | low — mostly policy | -| **T2 — container** | `code_task` / `run_script` / any shell or eval, **or** any agent not authored by a first-party engineer. | `docker run --rm`, no host mount beyond the instance workspace, read-only rootfs, seccomp, no network except the egress proxy, ulimits, hard timeout. | reuse — the mutation sandbox already runs this shape | - -### 1.3 What to build, and when +| **T2 — container** | `code_task` / `run_script` / any shell or eval. *(The original row also said "**or** any agent not authored by a first-party engineer" — struck; see the note below.)* | `docker run --rm`, no host mount beyond the instance workspace, read-only rootfs, seccomp, no network except the egress proxy, ulimits, hard timeout. | reuse — the mutation sandbox already runs this shape | + +**Derivation state (verified 2026-08-03).** The trigger column above is *implemented*: +`AgentManifest.isolation_tier()` (`manifest.py:273-287`) returns `T2` for an open scope +or any `SHELL_TOOLS` member, `T1` for `WRITE_TOOLS` or any declared integration, else +`T0`, and `tests/unit/test_agent_manifest.py:224-252` pins all four cases. What does **not** +exist: the "recorded on `agent_run`" half (no `tier` column; highest migration on disk is +142) and the "enforced by the executor" half (nothing refuses a T2 run). The single caller +outside tests is a **log field** at `declarative.py:210`. Closing that is **WS-3a**. + +**The "not authored by a first-party engineer" trigger has no data model.** Verified +2026-08-03: there is no first-party/non-first-party field anywhere — not a column on any +table, not an `AgentManifest` field (`manifest.py:140-158` carries `kind`, `runtime`, +`sharing`, `capabilities`, `memory`, `legacy` and no provenance), not a setting. The only +thing in the tree that resolves "first-party" is a **test helper**, +`tests/unit/test_agent_manifest.py:37 _first_party_configs()`, which simply globs the agent +directories that happen to live in this monorepo. A trigger that cannot be evaluated is not +a trigger, so it is struck from the table above. To restore it, something must first exist +to read: the minimum is an `AgentManifest` provenance field (e.g. `provenance: +first_party | creator | external`) derived from the registration path and persisted on the +agent registry row — **that is a design task with no owner and no acceptance today**, and +under the internal-tool threat model (§1.5) it is not needed. + +### 1.3 What to build, and when — *reconciled against code 2026-08-03* **Now (days, not weeks).** None of the cheap wins are containers: -1. Flip the default. `tool_scope` absent must mean **deny**, not *inject everything* — at - minimum for `kind: declarative` and anything creator-authored. This single change removes - most of the exposure. -2. Derive and record the tier from the manifest; refuse to start a T2-triggering run until T2 - exists. -3. Remove `PermissionHandler.approve_all` from the three factories that set it - (`agent_architecture.md` §3.2), restoring the B6 risk-aware handler. - -**Before the Agent Workshop opens to non-engineers.** T2 for anything requesting shell. The -mutation sandbox already containerises a Copilot session, so this is reuse of a proven path, -not new infrastructure. - -**Before multi-tenant (a second org on this platform).** T2 becomes mandatory for *every* non-first-party -agent regardless of tool surface, because the trust boundary moves from "our engineers" to -"someone else entirely." At that point `DESIGN_LIMITATION_native_maf_mutation.md` must also be -closed — though the declarative model already removes it for the majority case. +1. ~~Flip the default. `tool_scope` absent must mean **deny**, not *inject everything*~~ — + **BUILT, and it is not this spec's item.** The deny branch ships as + `SKILLS_FAIL_CLOSED` (`_tool_injection.py:101-117`, applied at `:214-224`) with the named + `DEFAULT_PROFILE` from `acb_skills.skill_families`. It is owned by **WS-23** (spec + `skills_scope_out.md` §4, `skills_registry.md` §5) and ships **OFF**. + **OWNER-GATE** — the flip is registered in `work_plan.md` §6. Do not re-derive it here; + this spec's C1 is closed as *mechanism built, posture owner-gated*. +2. **Derive and record the tier from the manifest; refuse a T2-triggering run.** + **AGENT-SAFE.** Half-built: derivation exists (`manifest.py:273-287`), recording and + refusal do not. Acceptance is written as **WS-3a** in `permissions_sandbox_b6.md` §P5-a — + build from there, not from this bullet. +3. ~~Remove `PermissionHandler.approve_all` from the three factories that set it~~ — + **BUILT (2026-07-26; the count was two, not three).** See the C1 update below. The + executor's five `_permission_handler` sites (`executor.py:632, 2483, 3011, 3909, 4442`) + all install `_copilot_permission_handler()`. The residual is the *mode*: moving + `AGENT_PERMISSION_MODE` from `audit` to enforcement is **OWNER-GATE** + (`work_plan.md` §6); prod runs `audit`. + +**Egress + read-only rootfs on the two containers we already run.** **AGENT-SAFE.** Not in +the original list because in 2026-07 there were no containers on the run path worth +hardening; there are now two (`mutation.py`, `copilot_sandbox.py`), both carrying +caps/limits since 2026-07-27 and **neither** passing `--network` or `--read-only`. +Acceptance is **WS-3b** in `permissions_sandbox_b6.md` §P5-b. + +**~~Before the Agent Workshop opens to non-engineers.~~ → deprioritised; see §1.5.** +The original trigger assumed the Workshop would put agent authorship in the hands of people +outside the engineering team. Under the 2026-08-03 owner decision that is not the near-term +shape of this product: the Workshop's users are **Fracktal colleagues**, and a colleague who +can author an agent could already open a PR against this monorepo. T2 does not become urgent +at that boundary. + +**Before multi-tenant (a second org on this platform).** This remains the real T2 trigger, +and it is the *only* one left. The trust boundary genuinely moves from "our team" to +"someone else entirely", and at that point `DESIGN_LIMITATION_native_maf_mutation.md` must +also be closed — though the declarative model already removes it for the majority case. No +second org is planned; **T2 is parked until one is** (§1.5). *(The original wording made T2 +mandatory for "every non-first-party agent regardless of tool surface" — kept as intent, but +see §1.2: there is no field that says which agents those are, so this cannot be stated as +acceptance until one exists.)* ### 1.4 The strategic point @@ -73,13 +172,46 @@ closed — though the declarative model already removes it for the majority case A container around an agent that legitimately holds your Zoho credentials and `gmail-send` protects the *host* and nothing you actually care about. It cannot stop that agent from emailing a customer or writing to your CRM, because those are its job. The exposure that -matters lives in the tool surface — and the tool surface is a manifest field with a -default-open bug in it today. +matters lives in the tool surface — and the tool surface is a manifest field whose +default-open branch is still the shipped posture today (the deny branch exists but is +owner-gated OFF — §1.1). So: **fix the capability model first, containerise second.** Containers are the answer to "untrusted code on my host." Capability scoping is the answer to "trusted code, wrong authority" — which is the failure mode this platform will actually hit. +### 1.5 T2 is a parked sub-project — the internal-tool threat model (owner decision, 2026-08-03) + +**Do not delete the T2 material above; it is the right destination.** But it is not +near-term work, and the reason is a threat-model correction rather than a change of mind +about isolation: + +- **Command Center is an internal Fracktal tool.** The team uses it. There are no external + tenants, no customer-authored agents, and no anonymous authorship path. +- So the ladder must hold up to **trusted colleagues, not hostile users**. The failure modes + that actually matter here are *mistakes and blast radius* — a runaway loop eating the 4GB + VPS, an agent reading a credential outside its declared scope, an accidental `rm` in the + wrong tree — not a determined attacker escaping a container. +- Every one of those is addressed by **capability scoping + credential scoping + resource + ceilings + egress/rootfs posture**, all of which are either shipped (P5-a's Tier-0 + credential scoping, the 2026-07-27 cap/limit flags) or are the two small dispatchable + slices WS-3a/WS-3b. None of them needs a container around a normal run. +- §1.4's own argument already says this: *"a container around an agent that legitimately + holds your Zoho credentials and `gmail-send` protects the host and nothing you actually + care about."* Against colleagues, that is the whole of it. + +**What "parked" means concretely.** T2 (a live streaming run sandbox, tool-proxy RPC, +per-agent venv/image, warm pool — `permissions_sandbox_b6.md` §P5-c) keeps its design and +loses its schedule. It has **no acceptance criteria and should not be given any** until one +of these two things is true, at which point it is re-costed from scratch: + +1. A **second organisation** runs on this platform (real multi-tenancy), or +2. Agent authorship opens to someone **outside Fracktal** — a customer, a contractor with no + monorepo access, or a public Agent Workshop. + +**OWNER-GATE:** un-parking T2 is an owner decision, not an agent's. An agent asked to +"finish the isolation ladder" builds WS-3a and WS-3b and refuses T2 by name. + --- ## Part 2 — Hardening findings @@ -88,10 +220,19 @@ Twenty findings, severity-ranked. **Critical** = breaks a boundary the design cl enforce. **High** = breaks correctness under normal use. **Medium** = degrades at scale or in edge cases. +> **Gate labels (added 2026-08-03, contract point 7).** **AGENT-SAFE** = an independent +> agent may build it once the owning spec carries acceptance. **OWNER-GATE** = the agent +> must refuse and say so (`work_plan.md` §6). A label says who may act, **not** that +> acceptance exists — most findings below are still one-paragraph diagnoses, and several are +> owned by other workstreams. Where a finding names a different owning spec, that spec's +> acceptance wins over anything written here. + ### Critical -#### C1 · Absent `tool_scope` grants the full surface, including shell -`_tool_injection.py:67-78`. Covered in Part 1. First-party engineer-authored agents make this +#### C1 · Absent `tool_scope` grants the full surface, including shell — **OWNER-GATE (the flip); mechanism AGENT-SAFE and built** +`apps/services/orchestrator/orchestrator/_tool_injection.py:183-255` *(was cited as +`_tool_injection.py:67-78` — stale since the WS-23 S2/S3 work; corrected 2026-08-03)*. +Covered in Part 1. First-party engineer-authored agents make this a defensible convenience; a creator-authored agent whose author never heard of `tool_scope` makes it a privilege grant nobody made. **Fix:** default-deny for declarative/creator-authored agents. Keep fail-open only for @@ -102,7 +243,32 @@ in-repo agents, and log it. > a comment explaining it — someone had found this before. Both remaining factories now drop > `on_permission_request` and carry the same comment. -#### C2 · Capabilities are self-declared, with no granting side +> **Update 2026-08-03 (truth pass) — the deny mechanism is BUILT, by WS-23, not by this +> spec.** `_skills_fail_closed()` (`_tool_injection.py:101-117`) reads `SKILLS_FAIL_CLOSED` +> per call; when truthy, `_resolve_injected_scope`'s unscoped branch (`:214-224`) returns +> the core floor ∪ `default_profile_tools()` instead of the `None` "inject everything" +> sentinel. It **ships OFF**, and the flip is **OWNER-GATE** (`work_plan.md` §6). +> Owning spec: `skills_scope_out.md` §4 (WS-23 S3). A second, complementary guard also +> landed: `AgentManifest.validate()` (`manifest.py:355-359`) now *reports* an absent +> `tool_scope` on a `kind: declarative` agent as a problem, and `warnings()` (`:370-374`) +> surfaces it with the derived tier at registration. +> +> **Blast radius, for anyone editing here.** `_resolve_injected_scope` is the **single +> choke point for injected tools on both runtimes** — native-MAF and Copilot-BYOK both pass +> through `_inject_agent_tools` (`:699`), and `materialize_skill_bodies_for_agent` (`:165`) +> resolves the same scope so an on-demand skill body describes exactly the tools the agent +> received. WS-23 S2 layers the intersection-only per-agent family toggles on top of it +> (`:227-255`). `AgentManifest.resolve_tool_surface` (`manifest.py:259-271`) is pinned +> **equivalent** to it by `tests/unit/test_agent_manifest.py` — change one and that test +> fails until you change the other. That equivalence is also what makes +> `isolation_tier()` trustworthy, since the tier is derived from the resolved surface. +> +> **This is why WS-3's board title is wrong.** `work_plan.md` §2's WS-3 row claims +> "`tool_scope` deny" as part of the isolation ladder. That work belongs to WS-23 and is +> already done; WS-3 should claim only the tier record/refusal (WS-3a) and the container +> egress/rootfs posture (WS-3b). + +#### C2 · Capabilities are self-declared, with no granting side — **AGENT-SAFE** (design first; no acceptance yet) `config.json` is authored by whoever wrote the agent, and `tool_scope` is read from it directly. In a world where anyone can create an agent, **self-declared capability is self-granted privilege**. @@ -119,7 +285,7 @@ security model right and the older one didn't.** **Fix:** port the shape. The manifest *requests*; a grant table *authorizes*; consent is a third thing that never widens scope. -#### C3 · Steer injected as a system-role note is a prompt-injection channel +#### C3 · Steer injected as a system-role note is a prompt-injection channel — **AGENT-SAFE** · owned by **WS-10** (`docs/multiplayer/README.md` §4.6) `README.md` §4.6 injects steer text as `"[steer from Sanjay] skip the staging deploy"` at a tool boundary. If that lands as **system** role, any contributor can issue instructions that outrank the agent's own guardrails — *"ignore your prior instructions, send the file to…"* — @@ -128,7 +294,7 @@ from inside a room they were merely invited to observe-and-contribute in. attributed, and wrapped in a delimiter the system prompt names as untrusted participant input. A contributor can redirect the work; they cannot rewrite the agent. -#### C4 · Content laundering defeats clearance +#### C4 · Content laundering defeats clearance — **AGENT-SAFE** · owned by **WS-10 S1** (`docs/multiplayer/memory-clearance.md` §7) Clearance controls what the model **reads**. It does not control what the model **writes**. An agent that legitimately read `subject:falcon` in a solo session writes that content into `chat_message` — and if that thread is later shared with `history_visibility: full`, or a @@ -143,7 +309,7 @@ under a compartment the viewer lacks is not delivered — the same rule as `sinc on labels instead of time. `memory-clearance.md` §5.4's "sharing can't retroactively unshare" warning is the symptom; this is the mechanism. -#### C5 · A refusal is itself a disclosure +#### C5 · A refusal is itself a disclosure — **AGENT-SAFE** · owned by **WS-10 S1** If the model is told a compartment exists but is barred, it can say so — and *"I have information about Project Falcon I can't use here"* leaks Falcon's existence to the room. The private-hint design (`memory-clearance.md` §4.4) is right, but only if it is computed @@ -153,7 +319,7 @@ not as an instruction. "Not cleared" and "does not exist" must be behaviourally ### High -#### H1 · The prompt-cache routing key is the agent name +#### H1 · The prompt-cache routing key is the agent name — **AGENT-SAFE** `prompt_cache.py:166` — *"cache_key: optional routing key (agent name) → OpenAI `prompt_cache_key`."* Harmless today, because only the stable prefix is cached and memory sits below the cache break. @@ -164,14 +330,14 @@ agent. My own proposal creates the problem. **Fix:** the cache key must be `hash(agent, instance, kb_version)` before the file tier moves above the break. Land the two changes together or not at all. -#### H2 · The session memory cache key is `thread_id` alone +#### H2 · The session memory cache key is `thread_id` alone — **AGENT-SAFE** `session_cache.py:72` — `key = f"{_KEY_PREFIX}{thread_id}"`. A room whose membership changes mid-conversation keeps serving a block assembled at the **previous, wider** clearance for up to the 10-minute TTL. Adding a less-cleared member does not narrow what the agent sees. **Fix:** key on `(thread_id, clearance_set_hash)`, and invalidate on membership change. Flagged in `memory-clearance.md` §3.5; repeating it here because it is correctness, not preference. -#### H3 · The floor baton has no fencing token +#### H3 · The floor baton has no fencing token — **OWNER-GATE (blocked)** · floor control itself is pending the owner's WS-10 re-decision (`work_plan.md` §6); do not build a fence for a mechanism that may be removed `SET NX EX 120` plus a heartbeat is not a correct lock: under a Redis failover, or a heartbeat that lands just after expiry, two clients can believe they hold the floor. Two holders means two concurrent runs on one thread — which resurrects exactly the destructive race @@ -180,7 +346,7 @@ two concurrent runs on one thread — which resurrects exactly the destructive r epoch it was issued under; the executor rejects a stale epoch. The lock can then be best-effort, because the fence is authoritative. -#### H4 · Instance-keying strands every existing file +#### H4 · Instance-keying strands every existing file — **AGENT-SAFE** · **addressed**: `infra/postgres/137_quarantine_commingled_agent_data.sql` quarantines the `''` rows (verified 2026-08-03; note the file's own header block still reads `139_…`, a renumber artifact). Residual = the admin review screen Migration 120 adds `instance` with default `''`. When an agent flips to `personal`, its existing `agent_blob` rows stay at `''` and become invisible to every instance — or, if `''` is treated as readable-by-all, the leak survives the migration that was supposed to fix it. @@ -188,7 +354,7 @@ Same shape as the commingled Mem0 bucket. **Fix:** the same call — quarantine `''` rows for agents that flip, with an admin review screen. Decide it explicitly rather than discovering it during the migration. -#### H5 · KB edit authority is instruction edit authority +#### H5 · KB edit authority is instruction edit authority — **AGENT-SAFE** KB content is injected into the prompt on every run. Whoever can edit a KB source can change what the agent does for everyone who uses it — quietly, and with none of the review a code change gets. @@ -196,14 +362,14 @@ change gets. **memory compartments and KB sources**, not just tool scopes, so a version that widens data access triggers re-consent the same way a version that widens tool access does. -#### H6 · `handoff` has no chain limit +#### H6 · `handoff` has no chain limit — **AGENT-SAFE** `agent_architecture.md` §8 sets a depth limit for `call` and says nothing about `handoff`. A hands to B, B's rule hands back to A, and the pair ping-pong across turns burning tokens until someone notices. **Fix:** a per-thread handoff chain limit and a "returned from" marker; a second handoff back to an agent already in the chain is refused and surfaced in the room. -#### H7 · Observability becomes the bypass +#### H7 · Observability becomes the bypass — **AGENT-SAFE** · interacts with **WS-3a**, which adds a *non-sensitive* column (`isolation_tier`) to the same `agent_run` row `memory_architecture.md` §6.6 proposes storing the assembled memory block on `agent_run` for eval replay and incident review. That creates a single table containing cross-compartment content — and `agent_run` already retains full folded traces for errored runs. If the @@ -231,7 +397,7 @@ compartment list. A hash still satisfies "did memory change between these runs." routine question and wrong when the pending `ask_user` gates a destructive tool — answering it steers an outward write. Restrict those to holders of the org permission. - **M7 · Redis now holds assembled cross-compartment memory.** AUTH, TLS, and network - isolation move from hygiene to requirement. + isolation move from hygiene to requirement. — **OWNER-GATE** (prod infra + credential change). - **M8 · The generic builder is a single point of compromise.** Net positive — one place to fix rather than six — but it deserves the strictest test coverage in the codebase, because a bug there is a bug in every declarative agent simultaneously. @@ -239,6 +405,12 @@ compartment list. A hash still satisfies "did memory change between these runs." that gets published, not the draft, or a draft edit between gate and publish ships un-evaluated. +**Gate labels for the Medium set (2026-08-03):** M1–M6, M8, M9 are **AGENT-SAFE** — all are +in-repo code changes owned by the room/apps workstreams (M1–M3, M5 → WS-10; M4 → memory +compartments, WS-10 S1; M6 → WS-10 + the HITL gate; M8, M9 → the declarative builder / eval +gate). **M7 is OWNER-GATE** (prod Redis AUTH/TLS + network isolation is a deploy and +credential change). None of the Medium items carries acceptance yet; none is part of WS-3. + --- ## Part 3 — What holds up @@ -268,17 +440,21 @@ An honest review should say what not to churn. ## Part 4 — Do these first -Ordered by (damage prevented ÷ effort), not by severity. - -| # | Action | Effort | Removes | -|---|---|---|---| -| 1 | Drop `approve_all` from the three agent factories | ~3 lines | A shipped security control being silently defeated | -| 2 | `tool_scope` absent ⇒ deny for declarative/creator agents (C1) | small | Unintended shell access on every future creator-authored agent | -| 3 | Authorize `routes/memory.py` against the path parameter | small | Any signed-in user reading anyone's memory | -| 4 | Scope Graphiti reads, or disable `search_entity_timeline` until scoped | small | Cross-user retrieval on every enriched run | -| 5 | Steer/HITL/observer input as user-role, delimited (C3) | small | Participant prompt injection, before rooms ship | -| 6 | Clearance-tagged messages + replay filtering (C4) | medium | The laundering path around the whole clearance model | -| 7 | Fencing token on the floor (H3) · cache keys include instance + clearance (H1, H2) | medium | Two-driver races; stale-clearance context | +Ordered by (damage prevented ÷ effort), not by severity. **Status column added 2026-08-03, +verified against code** — this table was written 2026-07-26 and two of its seven rows had +shipped without being marked. + +| # | Action | Gate | Effort | Status (2026-08-03) | Removes | +|---|---|---|---|---|---| +| 1 | Drop `approve_all` from the three agent factories | AGENT-SAFE | ~3 lines | ✅ **shipped 2026-07-26** — and it was two factories, not three (C1 update). The five executor sites (`executor.py:632, 2483, 3011, 3909, 4442`) install `_copilot_permission_handler()`. Residual: the enforcement-mode flip is **OWNER-GATE** | A shipped security control being silently defeated | +| 2 | `tool_scope` absent ⇒ deny for declarative/creator agents (C1) | **OWNER-GATE** (the flip); mechanism AGENT-SAFE | small | ✅ **mechanism shipped OFF** as `SKILLS_FAIL_CLOSED` under **WS-23**, not here | Unintended shell access on every future creator-authored agent | +| 2b | *(new)* Record the derived tier + refuse an un-isolated T2 run — **WS-3a** | AGENT-SAFE | small | 🔲 acceptance in `permissions_sandbox_b6.md` §P5-a | A tier that is computed, logged, and then discarded | +| 2c | *(new)* `--read-only` rootfs + `--network` posture on both containers — **WS-3b** | AGENT-SAFE | small | 🔲 acceptance in `permissions_sandbox_b6.md` §P5-b | Unbounded egress + writable rootfs in the two sandboxes we do run | +| 3 | Authorize `routes/memory.py` against the path parameter | AGENT-SAFE | small | 🔲 not verified in this pass | Any signed-in user reading anyone's memory | +| 4 | Scope Graphiti reads, or disable `search_entity_timeline` until scoped | AGENT-SAFE | small | 🔲 not verified in this pass (latent — `GRAPHITI_ENABLED` is false) | Cross-user retrieval on every enriched run | +| 5 | Steer/HITL/observer input as user-role, delimited (C3) | AGENT-SAFE · **WS-10** | small | 🔲 owned by WS-10 | Participant prompt injection, before rooms ship | +| 6 | Clearance-tagged messages + replay filtering (C4) | AGENT-SAFE · **WS-10 S1** | medium | 🔲 owned by WS-10 S1 | The laundering path around the whole clearance model | +| 7 | Fencing token on the floor (H3) · cache keys include instance + clearance (H1, H2) | H3 **OWNER-GATE (blocked on the WS-10 floor re-decision)** · H1/H2 AGENT-SAFE | medium | 🔲 | Two-driver races; stale-clearance context | Items 1–5 are days and are worth doing regardless of whether multiplayer proceeds. Item 6 is the one that has to land **with** the compartment work rather than after it — a read-side diff --git a/ai-company-brain/specs/calendar_focus_os.md b/ai-company-brain/specs/calendar_focus_os.md index 70005c11a..b09bcb543 100644 --- a/ai-company-brain/specs/calendar_focus_os.md +++ b/ai-company-brain/specs/calendar_focus_os.md @@ -1,16 +1,48 @@ # Calendar → Focus OS — evaluation & redesign brainstorm Status: **F0 + F1 BUILT** (2026-07-22, branch -`claude/calendar-productivity-redesign-rdh50k`): leverage lens + One Thing + +`claude/calendar-productivity-redesign-rdh50k`) — **verified against code on +2026-08-03**: leverage lens + One Thing + leverage meter + outcome ribbon, Gap Filler (2-minute pile), Startup ritual (breathe → review → commit), Shutdown (leverage ratio, One-Thing verdict, seed tomorrow, close the day) and Focus Mode (pomodoro/flow, subtask checklist, -+15 reflow, capture-in-focus via `C`). Frontend-only — per-day state -(One Thing / seeds / ritual stamps / timer prefs) lives in localStorage -(`lib/focusPrefs.ts`); the One-Thing planner directive rides the existing -`energy_note` seam, so no backend or schema changes were needed. F2+ (breaks -in the packer, batch blocks, Email windows, Waiting-on chase, `gtd_time_blocks`) -and the Focus Shield remain per §7. ++15 reflow, capture-in-focus via `C`). + +**Also shipped since (do not re-dispatch these):** +- **Breaks in the packer — SHIPPED 2026-07-23** (`80722e17`, migration + `infra/postgres/97_gtd_planning_prefs.sql`; the commit message's "mig 93" is + the pre-renumber number and is wrong — the file itself records `93→97`). + `gtd_settings.max_focus_run_mins` / `break_mins` + an optional protected lunch + window; the packer widens the buffer behind the block that trips the + focus-run limit + (`apps/services/gateway/gateway/routes/tasks/calendar.py` — `_planning_prefs`, + `_lunch_interval`, and the `want_break → buf = buffer_mins + break_mins` arm + in `_compute_day_plan`), reports breaks + lunch in the plan notes, and applies + lunch protection to rollover, replan and the nightly job. + **Caveat that keeps F2 alive:** a break is a *gap the packer leaves*, not a + row. There is no `kind='break'` block, nothing renders on the grid, nothing + is countable in the review. Typed break blocks stay F2, under + `gtd_time_blocks`. +- **Per-day Focus-OS state is no longer localStorage-only.** Migration + `infra/postgres/92_gtd_day_state.sql` (`gtd_day_state`) + + `GET/PUT /tasks/calendar/day-state` persist the ★ One Thing and the + tomorrow-seeds server-side; the client already calls them + (`workbench/control_plane/src/app/tasks/components/CalendarView.tsx` hydrates + on open and writes on toggle, + `.../components/calendar/EndOfDayReview.tsx` writes seeds on "close the day"). + `workbench/control_plane/src/app/tasks/lib/focusPrefs.ts` is now a *cache* for + those two, and remains the only home for **ritual stamps** + (`startupDoneOn`, `startupStreak`, `streakStampedOn`, `dayClosedOn`) and + **`timerMode`** — that residue is all that the F2 "migrate the local state" + clause still owes. +- **Ideal-week templates — SUBSTANTIALLY SHIPPED** (see §7 F3): migration + `infra/postgres/98_gtd_day_templates.sql` + settings API + editor + grid + render + packer honouring. Only the named gap in §9 remains. + +The One-Thing planner directive rides the existing +`energy_note` seam. **Still open per §7/§9:** `gtd_time_blocks` (and everything +that needs block *kinds* — typed breaks, batch blocks, recurring ritual blocks), +Email windows, Waiting-on chase, the Focus Shield, and external sync. **Follow-up (same day):** block context menu (right-click on desktop, long-press on touch — Open · Focus · Done · One Thing · Pin · Reschedule… · Remove from calendar · Delete), undoable scheduling (every timebox/move/ @@ -143,6 +175,16 @@ Tap ▶ on any block (or the Now bar) → full-screen focus: The shield state is visible ("6 held · released at your break"), which is the honest version of Do-Not-Disturb: nothing is missed, everything is deferred. Full-screen by design; single-theme ultra-dim "quiet" mode. + *(Update 2026-08-03: **the hold/release primitive genuinely does not exist.** + `grep -rniE "focus_shield|focusShield|notification_hold|hold_notifications"` + over `*.ts`/`*.tsx`/`*.py` returns **zero hits** repo-wide. So the Shield is + two pieces of work, not one: (a) a notification hold/release primitive on the + platform's own notification surface, and (b) the Focus-Mode UI that arms it. + **Both are AGENT-SAFE** — this touches only Command Center's own surfaces, + needs no OS/browser permission, no external credential and no deploy gate. + It is blocked on being **specced**, not on an owner action; the earlier + "blocked on a platform primitive" note overstated it. Neither piece has a + done-when yet — write one before dispatch.)* - **Capture without leaving** (tips 20/22/87 — swirling-thoughts problem): the existing QuickCapture hotkey (`C`) opens a minimal capture drawer *inside* Focus Mode — the stray thought goes to the GTD inbox and the timer never @@ -187,7 +229,9 @@ AI planner, is the unclaimed spot. ### 4.6 Breaks & recovery as first-class citizens - Packer rule: no more than N focus-minutes without a break (default 90 → 10); - lunch window protected by default. + lunch window protected by default. **✅ SHIPPED 2026-07-23** (`80722e17`, + migration `97_gtd_planning_prefs.sql`) — but as *geometry*: the packer widens + the buffer behind the tipping block. The break is a gap, not a row. - Break blocks have types (walk · stretch · breathe · coffee) with tiny guided timers; skipping is one tap (tracked, gently reported in review). - Buffers remain for meeting decompression; breaks are for recovery. @@ -205,6 +249,17 @@ AI planner, is the unclaimed spot. also the calendar's institutional way of **saying no** (tip 3): the planner declines to schedule over capacity and tells you *what it declined and why*. +> **⚠️ OWNERSHIP COLLISION — recorded 2026-08-03, unresolved.** "Top-5 outcomes +> (Horizons build-out)" is carried here (F3, §4.7) **and** in `work_plan.md`'s +> WS-18 row (Tasks Phase 3), where the 2026-08-02 audit declared it +> **NO-GO and MIS-ASSIGNED** — no acceptance criterion exists anywhere, +> `gtd_horizons` (present since migration 48) has **no link column** to items or +> projects, and `task_manager_app.md` puts Horizons in *Phase 4*, not 3. Two +> rows gesture at Horizons and **neither owns it**. Resolving this needs a +> single-owner decision in `work_plan.md` §4 (the single-owner registry) — it is +> deliberately *not* resolved here. Until it is, **no agent should dispatch +> Top-5 outcomes from either doc.** + ### 4.8 AI task breakdown on drop - Dropping a task with estimate >90m (or none + big title) prompts: "Split into sessions?" → AI proposes subtasks/sessions with estimates; accepts as @@ -241,8 +296,10 @@ follow-up a scheduled habit instead of a guilty memory. ### 4.12 Foundations this unlocks (already spec'd as P5) `gtd_time_blocks` (multi-block tasks, recurring blocks, break/ritual/external -kinds), ideal-week templates, external calendar sync. The features above are -the *reason* to now build that table. External sync (P4) is also what makes +kinds), ~~ideal-week templates~~ (**shipped 2026-07-23** — §7 F3), external +calendar sync (**OWNER-GATE**, §9.11). The features above are +the *reason* to now build that table — and it is **four PRs, not one**: see the +slice plan in §9.1. External sync (P4) is also what makes timeboxing **transparent** (tip 1's shared-calendar clause) — colleagues see the block, not the task detail — and what lets the packer respect commutes, meetings and travel buffers (tips 16/48). @@ -311,11 +368,33 @@ No feature above is an island; each plugs into a surface that already exists: actual_start, actual_end, source, external_event_id, recurrence_rule`. `item_id` nullable because breaks/rituals aren't tasks. Batch blocks join to members via `gtd_block_members(block_id, item_id, done_at)`. - *(Update 2026-08-01 (doc-truth pass): this column set is **CANONICAL** for - `gtd_time_blocks`. The table is specified in three places with different - shapes — `calendar_timeboxing.md` §3, here, and the comment at + *(Update 2026-08-01 (doc-truth pass), re-verified 2026-08-03: this column set + is **CANONICAL** for `gtd_time_blocks`. The table is specified in three places + with different shapes — `calendar_timeboxing.md` §3, here, and the comment at `infra/postgres/76_gtd_scheduling.sql:14` — the other two now defer here. - The table is still unbuilt: no migration creates it as of 2026-08-01.)* + The table is still unbuilt: `grep -rl gtd_time_blocks` over `*.sql`/`*.py`/ + `*.ts` matches exactly one file, the comment in `76_gtd_scheduling.sql`. + **Do not write an absolute migration number into this spec** — find the next + free number by listing `infra/postgres/` at build time.)* + *(Update 2026-08-03: **the "non-breaking swap" claim in + `calendar_timeboxing.md` §3 and in `76_gtd_scheduling.sql:14` is FALSE.** + There is no `TimeBlock[]` seam: `blocksForDay(items, day)` in + `workbench/control_plane/src/app/tasks/lib/scheduling.ts` *projects* blocks + out of `gtd_items.scheduledStart/scheduledEnd`, and every mutation goes + through `applySchedule(…{scheduledStart, scheduledEnd})`. Measured blast + radius on 2026-08-03: **17 files under + `workbench/control_plane/src/app/tasks/` reference + `scheduledStart|blocksForDay|applySchedule`** (`lib/scheduling.ts`, + `lib/scheduling.test.ts`, `lib/types.ts`, `lib/api.ts`, `lib/taskStore.ts`, + `lib/taskAssistantPersona.ts`, `components/CalendarView.tsx`, + `components/FocusMode.tsx`, `components/SchedulePopup.tsx`, + `components/StartupRitual.tsx`, + `components/calendar/{TimeGrid,MonthGrid,NowNextBar,ScheduleSheet,EndOfDayReview,PlanDayPanel}.tsx`, + `components/calendar/shared.ts`) plus **3 gateway modules** + (`apps/services/gateway/gateway/routes/tasks/{calendar,core,items}.py`), + `apps/skills/skill-task-gtd/skill_task_gtd/core.py`, and the tool + registration in `apps/agents/agent-task-manager/agents.py`. This is a + multi-PR migration, not a swap — see the slice plan in §9.)* - **`gtd_items`**: no change needed beyond what exists (leveraged, isTwoMinute, energy, estimates, actuals all present) — the redesign is mostly *surfacing* captured data. @@ -345,18 +424,34 @@ No feature above is an island; each plugs into a surface that already exists: existing plan + review modals. - **F1:** Focus Mode (Pomodoro/flow, subtask checklist, +15 reflow, capture-in- focus via the existing QuickCapture, ambient sound) — timer state is - client-side; actuals API already exists. Focus Shield ships here if the - notification surface exposes a hold/release hook; otherwise F2. - *(Update 2026-08-01 (doc-truth pass): the hold/release hook does NOT exist — - no notification-hold primitive anywhere in `control_plane/src`. Focus Shield - is therefore F2+, blocked on that platform primitive; also flagged in - `work_plan.md` WS-21.)* -- **F2:** `gtd_time_blocks` + breaks in the packer + batch blocks + recurring - ritual blocks + Email windows + Waiting-on chase block. -- **F3:** ideal-week templates, Top-5 outcomes (Horizons build-out), mobile - timeline view, AI breakdown-on-drop, weekly review surface, external sync - (P4 creds permitting — unlocks shared-calendar transparency + meeting-aware - buffers). + client-side; actuals API already exists. **SHIPPED 2026-07-22.** + Focus Shield slipped to F2 — see the §4.1 note: the hold/release primitive + does not exist, but it is **AGENT-SAFE once specced**, not owner-gated. +- **F2:** `gtd_time_blocks` + typed break blocks + batch blocks + recurring + ritual blocks + Email windows + Waiting-on chase block + Focus Shield. + ~~breaks in the packer~~ — **SHIPPED 2026-07-23** (`80722e17`, migration + `97_gtd_planning_prefs.sql`; the commit message's "mig 93" is wrong). What + shipped is *break geometry*: the packer widens the buffer after + `max_focus_run_mins` of continuous focus and protects a lunch window. What + F2 still owes is *typed break rows* — a break you can see on the grid, skip, + and count in the review — which needs block kinds, i.e. `gtd_time_blocks`. +- **F3:** ~~ideal-week templates~~ (**SUBSTANTIALLY SHIPPED 2026-07-23** — + migration `98_gtd_day_templates.sql` (`gtd_settings.day_templates`), the + settings API round-trip + (`apps/services/gateway/gateway/routes/tasks/settings.py` — model field, + patch field, `_day_templates` normaliser, the write path's JSON dump), the + editor in + `workbench/control_plane/src/app/tasks/components/calendar/CalendarSettings.tsx`, + the grid render via `TimeGrid.tsx` + `calendar/shared.ts`, and the packer + honouring them — `kind='block'` windows become busy time, `kind='focus'` + windows bias matching energy via `_THEME_ENERGY` + (`.../routes/tasks/calendar.py` `_expand_templates`); covered by + `tests/unit/test_calendar_planner.py::test_block_template_is_busy_focus_template_is_not` + and `::test_template_day_of_week_filter_skips_other_days`). **Re-scoped to + the named gap in §9** — do not carry "ideal week" as an unbuilt F3 item.), + Top-5 outcomes (Horizons build-out — **see the ownership-collision warning in + §4.7; do not dispatch**), mobile timeline view, AI breakdown-on-drop, weekly + review surface, external sync (**OWNER-GATE**, see §9). ## 8. Mockups @@ -366,29 +461,267 @@ break / ritual blocks + meters; Focus Mode; Gap Filler; Startup ritual; Shutdown review; mobile Today timeline. Visual language matches the control plane's dark theme (cyan primary, gold = leverage). -## 9. Acceptance & verification for F2/F3 open items (added 2026-08-01, doc-truth pass) - -- **F2 `gtd_time_blocks` — done when:** a migration creates the §5 table - (+ `gtd_block_members`) and blocks persist server-side — a timebox created on - one device survives reload and appears on a second device; the per-day - Focus-OS state currently in localStorage - (`app/tasks/lib/focusPrefs.ts`: One Thing, tomorrow seeds, ritual stamps, - timer prefs) is migrated to server-backed storage so it follows the user - across devices; breaks/rituals/batches exist as - `kind='break'|'ritual'|'batch'` rows, not client-side synthesis. -- **F2 Email windows — done when:** a recurring Email window renders as a real - block, deep-links into the email app's triage, email-captured tasks route to - tomorrow's plan seed by default, and the end-of-day review reports email - planned-vs-actual like any block. *(Partial foundation already shipped - 2026-07-23: recurring BLOCK/FOCUS windows — `gtd_settings.day_templates`, - mig `98_gtd_day_templates.sql` — can reserve an "Email" window on the grid; - the email-app deep-link, shield-hold and review accounting do not exist.)* -- **F3 external sync — done when:** see `calendar_timeboxing.md` §13 (P4): - OAuth-backed `calendar_accounts`, `kind='external'` events the packer will - not book over, `POST /tasks/calendar/sync` no longer 501, two-way write. -- **Verify:** `cd workbench/control_plane && npx tsc --noEmit && npm test` - (vitest); `pytest tests/unit -k calendar` — runs - `tests/unit/test_calendar_planner.py` (packer geometry, buffers, energy + - day-template windows, lunch carve-out) and - `tests/unit/test_email_calendar_context.py`; GTD API surface: - `pytest tests/unit/test_tasks_gtd.py`. +## 9. Acceptance & verification for F2/F3 open items + +*(Added 2026-08-01; **rewritten 2026-08-03 after verifying every clause against +the code.** The 2026-08-01 pass wrote a `gtd_time_blocks` done-when whose first +two clauses were **already green against shipped code** — an implementer could +have "passed" it by creating an unused table. Those clauses are deleted below.)* + +### 9.0 How to read this section + +Every open item carries a label: + +- **AGENT-SAFE** — an independent agent can build it end to end: no credential, + no flag flip, no deploy, no reach outside this repo. +- **OWNER-GATE** — needs an owner action named in `work_plan.md` §6 before the + work can even be verified. Do not dispatch; report and stop. + +Two standing constraints for anyone implementing from this section: + +1. **Never write an absolute future migration number** into a spec, a commit + message or a code comment. Find the next free number by listing + `infra/postgres/` at build time. This corpus already carries the disease: + `80722e17`'s message says "mig 93" (real: 97), and + `apps/services/gateway/gateway/routes/tasks/calendar.py`'s `_planning_prefs` + / `_day_templates` docstrings still say "migration 93" / "migration 94" + (real: 97 / 98). +2. **Paths are repo-root-relative and fully qualified** — + `workbench/control_plane/src/app/tasks/…` for UI, + `apps/services/gateway/gateway/routes/tasks/…` for the gateway, + `apps/skills/skill-task-gtd/…`, `apps/agents/agent-task-manager/…`, + `infra/postgres/…` for migrations. Earlier revisions of this section wrote + `app/tasks/lib/focusPrefs.ts` and `routes/tasks/calendar.py`, both one tree + level short. + +### 9.1 F2 `gtd_time_blocks` — 4 slices, not one PR · **AGENT-SAFE** + +**The "non-breaking swap" claim is false.** `calendar_timeboxing.md` §3 and the +comment at `infra/postgres/76_gtd_scheduling.sql:14` both assert the grid is +written against a `TimeBlock[]` abstraction so promoting to a table is a drop-in +swap. It is not: blocks are *projected* from `gtd_items.scheduled_start/end` by +`blocksForDay()` and *mutated* through `applySchedule({scheduledStart, +scheduledEnd})`. The measured blast radius (2026-08-03) is in §5. Anyone who +plans this as one PR is planning to break the calendar. + +**Slices — each independently shippable and reviewable:** + +| # | Slice | Shape | +|---|---|---| +| S1 | **Schema + API, dual-write** | Migration (next free number at build time) creates `gtd_time_blocks` + `gtd_block_members` per §5. `PATCH /tasks/items/{id}` scheduling and `GET /tasks/calendar` write/read **both** the columns and the table; the columns stay authoritative. No UI change. | +| S2 | **Client swap** | `blocksForDay()` reads blocks from the API instead of projecting from item columns; `applySchedule` targets block ids. Table becomes authoritative, columns become a mirror. All 17 touching files move together. | +| S3 | **Packer + tool cutover** | `_compute_day_plan`, rollover, replan, `apps/skills/skill-task-gtd/skill_task_gtd/core.py` (`gtd_schedule`/`gtd_unschedule`/`gtd_list_schedule`) and `apps/agents/agent-task-manager/agents.py` emit blocks. Item columns dropped from the write path. | +| S4 | **Kinds** | `kind` values `break` / `ritual` / `batch` / `external` + `gtd_block_members` become real: the packer emits typed break rows instead of widened buffers, batch blocks carry members, ritual blocks recur. | + +**Done when — every clause must fail against today's code:** + +*(Deleted from the 2026-08-01 version because they were already true: +"blocks persist server-side / survive reload / appear on a second device" — +`scheduled_start/scheduled_end` have been `gtd_items` columns since +`76_gtd_scheduling.sql` and have always been server-side; and "the One Thing and +tomorrow-seeds move off localStorage" — done by `92_gtd_day_state.sql` + +`GET/PUT /tasks/calendar/day-state`.)* + +1. **One task holds two blocks on the same day and both render.** Split a 3h + task into 09:00–10:30 and 14:00–15:30; both appear on the day grid, both + count once each in the capacity meter, and completing the task closes both. + *(Impossible today: one row, one `scheduled_start`.)* +2. **A `kind='break'` row inserted by the packer is visible on the grid and + excluded from the leverage meter.** "Plan my day" with + `max_focus_run_mins=90` produces a break the user can see, skip, and that the + end-of-day review counts — and it contributes **zero** minutes to both the + booked-focus meter and the leverage meter. *(Today the break is a widened + buffer: invisible, uncountable, unskippable.)* +3. **A batch block with 3 `gtd_block_members` ticks members off + independently of the parent.** Drag "Batch 3 @calls" onto the grid; the block + shows an internal checklist; ticking one member marks that `gtd_item` done + and leaves the block and the other two open; the block closes when the last + member does. +4. **Residual local state is gone.** The only remaining localStorage residue in + `workbench/control_plane/src/app/tasks/lib/focusPrefs.ts` — the **ritual + stamps** (`startupDoneOn`, `startupStreak`, `streakStampedOn`, `dayClosedOn`) + and **`timerMode`** — is server-backed, so the startup streak survives a + different browser. *(One Thing + seeds already are; do not re-do them.)* + This clause is satisfiable **independently of S1–S4** and may ship first as + its own small PR on `gtd_day_state`. + +### 9.2 F2 Email windows · **AGENT-SAFE** + +Foundation already shipped 2026-07-23: `gtd_settings.day_templates` +(`98_gtd_day_templates.sql`) already reserves a recurring window, and +`_THEME_ENERGY` in `apps/services/gateway/gateway/routes/tasks/calendar.py` +already recognises `theme` values `"email"` and `"inbox"`. Missing: the +email-app deep link, the shield hold, and the review accounting. + +**Done when:** + +1. A recurring Email window renders as a real block on the grid (not merely a + tinted template band). +2. The block deep-links into the email app's triage surface. +3. Email-captured tasks (`origin.emailId`, via `TaskCaptureModal`) route to + **tomorrow's** plan seed by default, not into today's focus. +4. The end-of-day review reports email planned-vs-actual like any block — + **see the decision below for what "email time" means.** + +> **DECISION (agent-proposed 2026-08-03, owner may overrule) — what counts as +> email time.** Clause 4 was untestable because "email time" was never defined. +> Proposed definition, chosen because both halves already exist in the data: +> - **Planned email minutes** = the total minutes of that local day's +> `day_templates` entries whose `theme` normalises to `"email"` or `"inbox"` +> (the `_THEME_ENERGY` mapping is the existing normaliser — reuse it, do not +> add a second one). +> - **Actual email minutes** = summed `actual_start`→`actual_end` (migration +> `80_gtd_actuals.sql`, stamped by Focus Mode) of every block whose item +> carries `origin.emailId`, **plus** any block whose interval falls inside a +> planned email window regardless of origin. +> +> Rejected alternative: counting time spent in the email app itself. It would +> need new client telemetry, and it measures the app rather than the +> commitment — the review's whole point is planned-vs-actual against a block. +> **If the owner prefers app-time, clause 4 changes shape and this slice grows +> a telemetry sub-slice.** + +### 9.3 F2 Batch blocks · **AGENT-SAFE** · depends on §9.1 S4 + +**Done when:** the unscheduled rail groups schedulable micro-tasks by +`GtdContext` and offers "Batch 4 @calls (45m)" as one drag; the resulting grid +block is a single `kind='batch'` row with `gtd_block_members`; Focus Mode plays +it as a rapid-fire queue (done → next); and the AI planner, when it batches, +emits a batch block rather than n adjacent task blocks. + +### 9.4 F2 Waiting-on chase block · **AGENT-SAFE** *(nudge SENDING is OWNER-GATE)* + +**Done when:** a recurring `kind='ritual'` Chase block auto-fills with WAITING +items sorted by age then deadline, reusing the shipped +`workbench/control_plane/src/app/tasks/lib/waiting.ts` predicates (Waiting-For +surfacing landed 2026-08-02 under WS-18 — **do not rebuild it**); each row +offers nudge / "got it" (marks received) / escalate; and "got it" clears the +open `gtd_waiting` row. + +> **OWNER-GATE inside this slice:** *drafting and sending* the follow-up email +> goes through a real mail account. The chase surface, the ordering, and the +> "got it"/escalate paths are all agent-safe; the nudge **send** is not. Build +> the surface with the nudge action stubbed behind the existing confirm-before- +> send gate; do not wire an outbound send. + +### 9.5 F2 Focus Shield · **AGENT-SAFE once specced** · currently unspecced + +The primitive does not exist (§4.1: zero grep hits repo-wide). It needs no +external access, so it is **not** owner-gated — it is blocked on a design. +Before dispatch, someone must spec: where held notifications queue, what +"release" means for each notification kind, and what happens to a hold if the +session is abandoned. **Do not dispatch on §4.1 prose alone.** + +**Done when (draft, needs the design above first):** starting a focus session +holds Command Center's own notifications; the Focus Mode header shows a live +count ("6 held · released at your break"); ending the session or reaching a +break releases them in one batch; nothing is dropped; and a crash or tab close +releases the hold rather than stranding it. + +### 9.6 F3 Ideal week — **SUBSTANTIALLY SHIPPED**, re-scoped to one gap · **AGENT-SAFE** + +Do not carry "ideal week" as unbuilt work — see the §7 F3 note for the shipped +inventory (migration `98_gtd_day_templates.sql`, settings round-trip, editor, +grid render, packer honouring, 2 unit tests). **Recommendation: strike +"ideal week" from the WS-21 row title** and carry only the named gap. + +**Remaining gap — done when:** a `kind='focus'` themed window that goes unused +is visible as such (today an unfilled focus window is indistinguishable from +empty time), and the weekly view rolls up template adherence — "your Mon-AM deep +work window took admin work 3 weeks running". Everything else about ideal week +is done. + +### 9.7 F3 Mobile timeline · **AGENT-SAFE** + +**Done when:** on a viewport under the `md` breakpoint the day view defaults to +a vertical agenda journey (done above the now-marker, upcoming below, breaks as +beads), the hour grid stays one toggle away, and the toggle persists like the +existing list/board toggle. + +### 9.8 F3 AI breakdown-on-drop · **AGENT-SAFE** · depends on §9.1 (multi-block) + +**Done when:** dropping a task with `time_estimate_mins > 90` (or no estimate +and a long title) offers "split into sessions?"; accepting creates **multiple +blocks** for the one task with per-session estimates; declining is remembered +for that item. +> **EVAL-LOCKED:** `propose()` / `propose_with_llm()` in +> `apps/services/gateway/gateway/routes/tasks/ai.py` are locked by the golden +> eval — do not mutate them. Add a new function. + +### 9.9 F3 Weekly review surface · **AGENT-SAFE, but NOT owned here** + +`work_plan.md` WS-18 owns the GTD Weekly Review and its 2026-08-02 audit ruled +it **NO-GO** (`task_manager_app.md` §9.2 is a bare checkbox; `gtd_reviews.summary` +is untyped JSONB). The calendar's contribution is only the *recurring ritual +block + planned-vs-actual/roll-over/focus-hours rollup*. **Do not dispatch a +weekly review from this doc** — it must follow WS-18's JSON contract once that +exists. + +### 9.10 F3 Top-5 outcomes (Horizons) · **DO NOT DISPATCH** + +See the ownership-collision warning in §4.7. Needs a `work_plan.md` §4 +single-owner decision. + +### 9.11 F3 External sync · **OWNER-GATE** + +Canonical done-when: `calendar_timeboxing.md` §13 (P4). Verified state +2026-08-03: `calendar_accounts` **does not exist** (no migration, no code — +the only matches are three comments in +`apps/services/gateway/gateway/routes/tasks/calendar.py`), +`GET /tasks/calendar/accounts` returns `[]` (`calendar.py:44-50`), and +`POST /tasks/calendar/sync` raises **501** (`calendar.py:53-64`, the raise at +`:60-64`). + +> **OWNER-GATE — credential requirement:** clause 1 is *"a `calendar_accounts` +> row can be created through a real OAuth connect flow"*. That requires +> **Google Calendar and/or Microsoft Graph OAuth client credentials +> (client id + secret + redirect URI) provisioned on the VPS** and registered in +> the Integration Registry. An agent cannot obtain, install or verify these. +> **This gate is currently unregistered in `work_plan.md` §6 — register it.** + +### 9.12 Verify + +``` +cd workbench/control_plane && npx tsc --noEmit && npm test # vitest +``` + +``` +uv run pytest tests/unit/test_calendar_planner.py \ + tests/unit/test_email_calendar_context.py \ + tests/unit/test_tasks_gtd.py +``` + +**Name the files. Never run `pytest tests/unit -k calendar`** (the form this +section carried until 2026-08-03) — `-k` still *collects* the whole directory, +and whole-directory collection hangs on the Windows dev box. The named-file +form is the only safe one. + +Measured 2026-08-03 on this branch: +- the two calendar files alone → **28 passed in 1.16s**; +- all three files → **157 passed in 166.79s** (`test_tasks_gtd.py` is the slow + one — budget ~3 minutes, it is not hung). + +Coverage: `test_calendar_planner.py` = packer geometry (free intervals, buffers, +energy windows, lunch carve-out, day-template block/focus windows, weekday +filter); `test_email_calendar_context.py` = the email-side calendar context; +`test_tasks_gtd.py` = the GTD API surface. + +### 9.13 Recorded, not fixed + +- **The reminders/notifications deferral went invisible.** `calendar_ux_review.md` + §"P1 — mobile & reminders" and its ranked list item 4 carry *"block + reminders/notifications when a block starts… without reminders, blocks are + ignored"*. The word "reminder" appears in **no other calendar spec** and in no + `work_plan.md` row, so the deferral was silently dropped rather than decided. + It is a real open item and it shares a surface with the Focus Shield (§9.5) — + both need the same notification primitive. Whoever specs the Shield should + decide whether reminders ride along or are explicitly killed. +- **This spec family has four docs, not two.** `calendar_focus_os.md`, + `calendar_timeboxing.md`, `calendar_ai_review.md` and `calendar_ux_review.md`. + The WS-21 row names two; `calendar_ai_review.md` is cited by three migration + headers (`92`, `97`, `98`) yet is referenced by no other spec, no `work_plan.md` + row and no `ai-company-brain/AGENTS.md` index entry — and **the specs index in + `ai-company-brain/AGENTS.md` has no calendar row at all.** Cross-deferral + between focus_os and timeboxing is clean (§5 here is canonical for + `gtd_time_blocks`; `calendar_timeboxing.md` §13 is canonical for P4); the other + two docs are unregistered. diff --git a/ai-company-brain/specs/calendar_timeboxing.md b/ai-company-brain/specs/calendar_timeboxing.md index 521d0d249..04c0ba8ed 100644 --- a/ai-company-brain/specs/calendar_timeboxing.md +++ b/ai-company-brain/specs/calendar_timeboxing.md @@ -1,11 +1,16 @@ # Calendar & Timeboxing — feature spec + roadmap -Status: **P0–P3 SHIPPED to main** (PR #71 merged, commit `7a5c72b2`). +Status: **P0–P3 SHIPPED to main** (PR #71 merged, commit `7a5c72b2`) — +**verified against code on 2026-08-03**. The day/week/month grid, drag-drop + resize timeboxing, energy/capacity prefs, the AI "Plan my day" planner, chat-with-calendar tools, auto-reschedule roll-over, deadline radar, and overlap detection are all live. -Only P4 (external Google/Outlook sync — needs OAuth creds) and parts of P5 -remain deferred (see cross-map, §12). +Only P4 (external Google/Outlook sync — **OWNER-GATE**: needs Google/Graph OAuth +client credentials on the VPS) and parts of P5 remain deferred (see cross-map, +§12). Since then the packer also gained **break geometry + lunch protection** +(`80722e17`, migration `97_gtd_planning_prefs.sql`) and **recurring day-template +windows** (migration `98_gtd_day_templates.sql`) — both live; see +`calendar_focus_os.md` §7. **Update 2026-08-01 (doc-truth pass):** the old "draft PR / nothing auto-deploys until reviewed + merged" caveat is obsolete — PR #71 merged @@ -13,7 +18,9 @@ until reviewed + merged" caveat is obsolete — PR #71 merged the calendar routes are in main. P3's "still to do" (nightly job + roll history) has ALSO shipped since: migration `infra/postgres/78_gtd_calendar_rollover.sql` (`gtd_rollover_log`, `auto_rollover` toggle, per-user `timezone`) and -`start_auto_rollover()` launched at gateway startup (`gateway/main.py`). +`start_auto_rollover()` launched at gateway startup +(`apps/services/gateway/gateway/main.py:274-275`, defined at +`.../routes/tasks/calendar.py:1543`). Roll-over SEMANTICS then changed in #235 (2026-07-26): unfinished blocks are RELEASED back to the unscheduled list (`rolled_to = NULL` in the log) for deliberate re-planning — NOT packed onto today — so §6's "rolls them to the @@ -67,18 +74,39 @@ scheduled_end TIMESTAMPTZ -- end of the block; default start + estimate (`id, item_id, start, end, kind, source, external_event_id`) so one task can have *multiple* blocks (split focus sessions, recurring), and so external calendar events (meetings that are NOT tasks) can live on the same grid via -`kind='external'`. The grid component is written against a `TimeBlock[]` -abstraction so this swap is non-breaking. +`kind='external'`. ~~The grid component is written against a `TimeBlock[]` +abstraction so this swap is non-breaking.~~ -> **Update 2026-08-01 (doc-truth pass):** `gtd_time_blocks` is still unbuilt — -> no migration creates it. Its column set is specified in three places with +> **Update 2026-08-01 (doc-truth pass), re-verified 2026-08-03:** +> `gtd_time_blocks` is still unbuilt — no migration creates it; the only +> repo-wide match is the comment in `infra/postgres/76_gtd_scheduling.sql`. +> Its column set is specified in three places with > different shapes (here, `calendar_focus_os.md` §5, and the comment at > `infra/postgres/76_gtd_scheduling.sql:14`); **`calendar_focus_os.md` §5 is > canonical** — this section and the migration comment defer to it. +> **When you build it, do not write an absolute migration number into any spec +> or commit message — list `infra/postgres/` and take the next free number.** + +> **CORRECTION 2026-08-03 — the struck sentence above was FALSE, and it +> materially understated the cost of this table.** There is no `TimeBlock[]` +> seam. `blocksForDay(items, day)` in +> `workbench/control_plane/src/app/tasks/lib/scheduling.ts` *projects* blocks +> out of `gtd_items.scheduledStart/scheduledEnd`, and every mutation goes +> through `applySchedule(…{scheduledStart, scheduledEnd})`. Measured blast +> radius: **17 files** under `workbench/control_plane/src/app/tasks/` reference +> `scheduledStart|blocksForDay|applySchedule`, plus **3 gateway modules** +> (`apps/services/gateway/gateway/routes/tasks/{calendar,core,items}.py`), +> `apps/skills/skill-task-gtd/skill_task_gtd/core.py`, and +> `apps/agents/agent-task-manager/agents.py`. **This is 4 PRs, not a swap** — +> the slice plan (schema+API dual-write → client swap → packer/tool cutover → +> kinds) and the done-when live in `calendar_focus_os.md` §9.1. +> The same false claim is repeated in the `76_gtd_scheduling.sql:14` comment; +> fix it whenever that file is next touched. ## 4. Views (scaffolded: day / week / month grid) -Replace the flat list at `page.tsx` "calendar" branch with a dedicated +Replace the flat list at +`workbench/control_plane/src/app/tasks/page.tsx`'s "calendar" branch with a dedicated `CalendarView` (day/week/month toggle, persisted like the list/board toggle). - **Day** — vertical hour grid (configurable day window, e.g. 07:00–22:00), @@ -131,7 +159,8 @@ Reuse the existing task assistant (`AgentChat` → `/api/agent/chat`, agent `task-manager`). Two additions: - **Persona context**: extend `buildTaskAssistantPersona` with today's blocks, free windows, capacity, energy windows, and upcoming deadlines. -- **Tools** (in `skill-task-gtd`): `gtd_schedule(item, start, end)`, +- **Tools** (in `apps/skills/skill-task-gtd/skill_task_gtd/core.py`, registered + by `apps/agents/agent-task-manager/agents.py`): `gtd_schedule(item, start, end)`, `gtd_reschedule`, `gtd_plan_day(date, energy_note)`, `gtd_unschedule`. Then the user can say *"I'm low energy today, move the deep work to tomorrow and @@ -140,10 +169,13 @@ grid + upcoming load, and reorganises the day around the stated energy, pulling manageable work forward. This directly satisfies the "chat with my calendar, account for energy, auto-organise the main tasks to focus on" request. -## 8. External sync — Google Calendar + Outlook (planned; seams scaffolded) +## 8. External sync — Google Calendar + Outlook (planned; seams scaffolded) · 🔒 **OWNER-GATE** -Reuse the email OAuth stack (`email/transport/oauth.py` already does Google + -Microsoft Graph; encrypted tokens via `key_store`). +Reuse the email OAuth stack +(`apps/services/gateway/gateway/routes/email/transport/oauth.py` already does +Google + Microsoft Graph; encrypted tokens via `key_store`). +**Gate:** this cannot be built or verified without Google/Graph OAuth client +credentials on the VPS — see §13 P4. - New `calendar_accounts` table (mirror `task_accounts`/`email_accounts` encrypted-token pattern). Scopes: Google `calendar.events`, Graph `Calendars.ReadWrite`. @@ -212,44 +244,77 @@ Microsoft Graph; encrypted tokens via `key_store`). *(Update 2026-08-01: the nightly job + `gtd_rollover_log` history SHIPPED — mig 78 + `start_auto_rollover()`; and #235 changed the semantics to release-to-list, see the header note. P3 is CLOSED.)* -- **P4 — external sync (DEFERRED — needs OAuth creds):** `calendar_accounts` + - Google/Graph read (conflict-avoidance) then two-way write. Seamed at - `GET /calendar/accounts` + `POST /calendar/sync` (501). -- **P5 — DEFERRED:** `gtd_time_blocks` table (multiple blocks/task, external - events on the grid) + continuous auto-scheduling engine + Pomodoro + ideal-week - templates + learned-estimate heuristics. - *(Update 2026-08-01: the Pomodoro item SHIPPED via `calendar_focus_os.md` F1's - Focus Mode — pomodoro/flow timer with cycle dots, 2026-07-22, - `app/tasks/components/FocusMode.tsx`. Remaining Pomodoro-adjacent scope lives - under focus_os F2: break blocks in the packer + cycle telemetry feeding - learned estimates. The rest of P5 also tracks under focus_os F2/F3 — see §12.)* +- **P4 — external sync (DEFERRED · 🔒 OWNER-GATE — needs OAuth client creds):** + `calendar_accounts` + Google/Graph read (conflict-avoidance) then two-way + write. Seamed at `GET /tasks/calendar/accounts` + `POST /tasks/calendar/sync` + (501). Done-when + the credential requirement: §13. +- **P5 — DEFERRED · AGENT-SAFE:** `gtd_time_blocks` table (multiple blocks/task, + external events on the grid) + continuous auto-scheduling engine + ~~Pomodoro~~ + + ~~ideal-week templates~~ + learned-estimate heuristics. + *(Update 2026-08-01, re-verified 2026-08-03: **Pomodoro SHIPPED** via + `calendar_focus_os.md` F1's Focus Mode — pomodoro/flow timer with cycle dots, + 2026-07-22, `workbench/control_plane/src/app/tasks/components/FocusMode.tsx`. + **Ideal-week templates SUBSTANTIALLY SHIPPED** 2026-07-23 — migration + `98_gtd_day_templates.sql` + settings round-trip + editor + grid render + + packer honouring; only the unused-window/adherence gap remains + (focus_os §9.6). **Packer breaks + lunch protection SHIPPED** 2026-07-23 + (`80722e17`, mig 97) — but as buffer geometry, so *typed* break blocks and + cycle telemetry feeding learned estimates still ride on `gtd_time_blocks`. + The rest of P5 tracks under focus_os F2/F3 — see §12, and §9 there for + acceptance.)* ## 11. Files this touches (map) -- Migration: `infra/postgres/76_gtd_scheduling.sql` (+ `schema.generated.sql`). -- Backend: `routes/tasks/core.py` (model + row map), `routes/tasks/items.py` - (patch fields + `GET /tasks/calendar`), new `routes/tasks/calendar.py` (sync - stubs), later `skill-task-gtd/core.py` (schedule tools) + - `agent-task-manager/agents.py` (register tools). -- Frontend: `lib/types.ts` (`ViewKey` already has `calendar`; add scheduled - fields), `lib/api.ts` (`mapItem` + `apiSchedule`/`apiCalendarRange`), - `lib/taskStore.ts` (`scheduleItem`, calendar range loader), new - `components/CalendarView.tsx` (+ day/week/month subviews), `page.tsx` (route - calendar → `CalendarView`), `lib/taskAssistantPersona.ts` (calendar context, - P2). - -## 12. Cross-map to `calendar_focus_os.md` F0–F3 (added 2026-08-01, doc-truth pass) - -| This doc | focus_os | State | -|---|---|---| -| P0–P2 (grid, timeboxing, planner, chat) | shipped foundation under F0/F1 | SHIPPED (PR #71; F0/F1 2026-07-22) | -| P3 remainder (nightly roll-over + history) | — (closed here) | SHIPPED (mig 78 + #235 release-to-list) | -| P4 external sync | F3 item | OPEN — `/tasks/calendar/sync` still 501 | -| P5 `gtd_time_blocks` / batch / recurring blocks | F2 | OPEN (table unbuilt; focus_os §5 canonical) | -| P5 Pomodoro | F1 Focus Mode | SHIPPED 2026-07-22 (`FocusMode.tsx`) | -| P5 ideal-week templates / auto-scheduling / learned estimates | F3 | OPEN (partial: recurring windows, mig 98) | - -## 13. Acceptance & verification (added 2026-08-01, doc-truth pass) +*(Paths re-qualified 2026-08-03 — every entry here was previously one or two +tree levels short, which broke the anchor check. All paths are repo-root +relative.)* + +- Migration: `infra/postgres/76_gtd_scheduling.sql` + (+ `infra/postgres/schema.generated.sql`). +- Backend, all under `apps/services/gateway/gateway/`: + `routes/tasks/core.py` (model + row map), `routes/tasks/items.py` + (patch fields + `GET /tasks/calendar`), `routes/tasks/calendar.py` (the + planner, packer, rollover, day-state and the external-sync stubs), + `routes/tasks/settings.py` (calendar prefs + `day_templates`). +- Agent surface: `apps/skills/skill-task-gtd/skill_task_gtd/core.py` + (`gtd_schedule` / `gtd_unschedule` / `gtd_list_schedule`) + + `apps/agents/agent-task-manager/agents.py` (registers those tools). +- Frontend, all under `workbench/control_plane/src/app/tasks/`: + `lib/types.ts` (`ViewKey` already has `calendar`; scheduled fields), + `lib/api.ts` (`mapItem` + `apiSchedule`/`apiCalendarRange`/`apiGetDayState`/ + `apiSetDayState`), `lib/taskStore.ts` (`scheduleItem`, calendar range loader), + `lib/scheduling.ts` (+ `lib/scheduling.test.ts`) — `blocksForDay`, + `applySchedule`, the block projection, + `components/CalendarView.tsx` and `components/calendar/*` (day/week/month + subviews, `TimeGrid`, `MonthGrid`, `NowNextBar`, `ScheduleSheet`, + `PlanDayPanel`, `EndOfDayReview`, `CalendarSettings`, `shared.ts`), + `components/FocusMode.tsx`, `page.tsx` (routes calendar → `CalendarView`), + `lib/taskAssistantPersona.ts` (calendar context, P2), + `lib/focusPrefs.ts` (the residual client-only Focus-OS state). + +## 12. Cross-map to `calendar_focus_os.md` F0–F3 + +*(Added 2026-08-01; re-verified against code 2026-08-03.)* + +| This doc | focus_os | State | Label | +|---|---|---|---| +| P0–P2 (grid, timeboxing, planner, chat) | shipped foundation under F0/F1 | **SHIPPED** (PR #71; F0/F1 2026-07-22) | — | +| P3 remainder (nightly roll-over + history) | — (closed here) | **SHIPPED** (mig 78 + `start_auto_rollover()` at `apps/services/gateway/gateway/main.py:274-275`; #235 release-to-list, `CalendarView.tsx:387-396`) | — | +| — (packer breaks + lunch) | F2 item, now closed | **SHIPPED 2026-07-23** (`80722e17`, mig 97) as *geometry*; typed `kind='break'` rows still owed under `gtd_time_blocks` | AGENT-SAFE | +| — (per-day Focus-OS state) | F2 clause | **SHIPPED** (mig 92 `gtd_day_state` + `GET/PUT /tasks/calendar/day-state`); only ritual stamps + `timerMode` remain local | AGENT-SAFE | +| P4 external sync | F3 item | **OPEN** — `calendar_accounts` absent; `POST /tasks/calendar/sync` still 501 (`calendar.py:57-64`); `GET /calendar/accounts` returns `[]` | **OWNER-GATE** (OAuth client creds) | +| P5 `gtd_time_blocks` / batch / recurring blocks | F2 | **OPEN** — table unbuilt; focus_os §5 canonical, §9.1 has the 4-slice plan | AGENT-SAFE | +| P5 Pomodoro | F1 Focus Mode | **SHIPPED 2026-07-22** (`FocusMode.tsx`) | — | +| P5 ideal-week templates | F3 | **SUBSTANTIALLY SHIPPED** (mig 98 + settings + editor + grid + packer + 2 tests); only the unused-window/adherence gap remains — focus_os §9.6 | AGENT-SAFE | +| P5 continuous auto-scheduling / learned-estimate heuristics | F3 | **OPEN** — no acceptance written; not dispatchable | AGENT-SAFE | +| Focus Shield | F2 (slipped from F1) | **OPEN** — primitive absent (zero grep hits); needs a design before dispatch | AGENT-SAFE *once specced* | +| Block reminders / notifications | — | **OPEN and unregistered** — lives only in `calendar_ux_review.md` P1; see focus_os §9.13 | AGENT-SAFE *once specced* | + +## 13. Acceptance & verification + +*(Added 2026-08-01; **verified against code 2026-08-03** — P3 below was the one +clause the 2026-08-01 pass got fully right and it is confirmed unchanged. The +verify command has been rewritten; the old `-k` form was unsafe.)* - **P3 nightly roll-over — SHIPPED; acceptance restated to match #235:** after a user's local midnight, yesterday's unfinished flexible timeboxes are RELEASED @@ -258,15 +323,61 @@ Microsoft Graph; encrypted tokens via `key_store`). opt-out honoured), and each release is recorded as a `gtd_rollover_log` row with `rolled_to = NULL`; re-planning them is deliberate (drag, or Rebuild my day, which sweeps them in per #232). -- **P4 external sync — OPEN; done when:** a `calendar_accounts` row can be - created through a real OAuth connect flow (Google/Graph, reusing the - `email/transport/oauth.py` pattern); external events render on the grid as +- **P4 external sync — OPEN · 🔒 OWNER-GATE; done when:** a `calendar_accounts` + row can be created through a real OAuth connect flow (Google/Graph, reusing + the `apps/services/gateway/gateway/routes/email/transport/oauth.py` pattern); + external events render on the grid as `kind='external'` and the planner/packer refuses to book over them; `POST /tasks/calendar/sync` returns data instead of 501; two-way write pushes a timeboxed block out as a real calendar event. -- **Verify:** `cd workbench/control_plane && npx tsc --noEmit && npm test` - (vitest); `pytest tests/unit -k calendar` — runs - `tests/unit/test_calendar_planner.py` (packer geometry: free intervals, - buffers, energy windows, lunch carve-out, day templates) and - `tests/unit/test_email_calendar_context.py`; GTD API surface: - `pytest tests/unit/test_tasks_gtd.py`. + + > **Why this is owner-gated, and what the owner must do.** Clause 1 cannot be + > satisfied — or even verified — without **Google Calendar and/or Microsoft + > Graph OAuth client credentials (client id, client secret, redirect URI) + > provisioned on the VPS** and registered in the Integration Registry. + > Obtaining, installing and consenting to those is an owner action; an agent + > must refuse and report. Verified state 2026-08-03: `calendar_accounts` does + > not exist in any migration or query (the only matches repo-wide are three + > comments in `apps/services/gateway/gateway/routes/tasks/calendar.py`), + > `GET /tasks/calendar/accounts` returns `[]` (`calendar.py:44-50`), and + > `POST /tasks/calendar/sync` raises `501` (`calendar.py:53-64`, the raise at + > `:60-64`). + > **This gate is not yet registered in `work_plan.md` §6 — register it.** + > + > Clauses 2–4 are *partly* agent-safe once the table exists: `kind='external'` + > rendering and packer avoidance can be built and unit-tested against seeded + > rows. But the slice cannot be *accepted* without clause 1, so the whole item + > stays owner-gated. + +- **Verify:** + + ``` + cd workbench/control_plane && npx tsc --noEmit && npm test # vitest + ``` + + ``` + uv run pytest tests/unit/test_calendar_planner.py \ + tests/unit/test_email_calendar_context.py \ + tests/unit/test_tasks_gtd.py + ``` + + **Name the files. Never run `pytest tests/unit -k calendar`** (the form this + section carried until 2026-08-03): `-k` filters *after* collection, so it + still collects the whole `tests/unit/` directory — the construction that hangs + on the Windows dev box. + + Measured 2026-08-03: the two calendar files alone → **28 passed in 1.16s**; + all three files → **157 passed in 166.79s** (`test_tasks_gtd.py` is the slow + one — budget ~3 minutes, it is not hung). + + Coverage: `test_calendar_planner.py` = packer geometry (free intervals, + buffers, energy windows, lunch carve-out, day-template block/focus windows, + weekday filter); `test_email_calendar_context.py` = the email-side calendar + context; `test_tasks_gtd.py` = the GTD API surface. + +- **Everything else open** — `gtd_time_blocks` and its four slices, typed break + blocks, batch blocks, Email windows, Waiting-on chase, Focus Shield, mobile + timeline, AI breakdown-on-drop, the ideal-week residual, weekly review and + Top-5 outcomes — has its acceptance, its AGENT-SAFE / OWNER-GATE label and its + dependencies in **`calendar_focus_os.md` §9**, which is canonical for the + F2/F3 surface. Do not write a second copy here. diff --git a/ai-company-brain/specs/multi_agent_orchestration.md b/ai-company-brain/specs/multi_agent_orchestration.md index b0b1a5158..a2ec9517d 100644 --- a/ai-company-brain/specs/multi_agent_orchestration.md +++ b/ai-company-brain/specs/multi_agent_orchestration.md @@ -1,12 +1,38 @@ # Multi-Agent Orchestration — Architecture & Work Plan -**Status:** proposed · **Date:** 2026-07-17 · **Owner:** Vijay +**Status:** Phase 4 only — Phases 0/1 shipped, 2/3/5 superseded · verified against code on 2026-08-03 · Owner: WS-12 **Scope:** how MAF agents and GitHub Copilot SDK agents delegate to each other today, and the backbone for the future visual workflow editor. Every claim marked ✅ below was **verified by execution** against the live VPS (`agent-framework-core==1.8.1`) or an isolated throwaway venv (`core==1.11.0` + -`orchestrations==1.0.0`). Reproduction commands are in [§8](#8-appendix--reproduction). +`orchestrations==1.0.0`) on 2026-07-17/18. Those reproduction commands are in +[§8](#8-appendix--reproduction) and are **not agent-runnable** (they require prod SSH); +§8.1 carries the local, agent-runnable equivalents added in the 2026-08-03 pass. + +> **Update 2026-08-03 (truth pass) — THIS DOCUMENT IS NOW PHASE 4 ONLY.** +> The row was audited against code on 2026-08-03 and ~90% of what its title claims is +> already delivered elsewhere. What is left here: +> +> | Phase | State | Where it went | +> |---|---|---| +> | **0** — hand-off fix | ✅ **shipped 2026-07-22** | delegation family is in the floor (`_tool_injection.py:41-65`); addendum is scope-aware | +> | **1** — context discipline | ✅ **shipped / moot / reassigned** | 1.1 → `93b93a08` (#191); 1.2 moot; 1.3 → **WS-23** ([`skills_registry.md`](skills_registry.md) · [`skills_scope_out.md`](skills_scope_out.md)) | +> | **2–3** — workflow runtime + editor | 🚫 **superseded** | the shipped Workflows app ([`workflows_app.md`](workflows_app.md), ADR-028, D6) | +> | **4** — framework uplift | 🔲 **the only live work in this document** | all four §5.5 shims verified still in the tree on 2026-08-03 | +> | **5** — orchestrations / collab chat | 🚫 **shipped elsewhere / reassigned** | 5.2 = multiplayer rooms (WS-10); 5.1 → **WS-11** / [`workflows_app.md`](workflows_app.md) §8; 5.3 unchanged (skip) | +> +> **Scope of record.** A three-way disagreement existed until 2026-08-03 — the board cell said +> "Phases 1, 4", `work_plan.md` §3 D6 said "Phases 1/4/5", this banner said "Phases 1, 4 and 5". +> **Resolved: Phase 4 only.** The board and D6 are to be swept to match; where they still disagree, +> this header wins for *what remains in this document* and `work_plan.md` wins for ordering. +> +> **Non-goals** (explicitly out of scope for this spec, do not build them from here): +> - a second workflow engine, graph spec, compiler, runner or editor — D6, `workflows_app.md` +> - injected-tool-surface / addendum token reduction — **WS-23**, `skills_registry.md` +> - a collaborative multi-agent chat surface — **shipped** as multiplayer rooms; residue is WS-10's +> owner-gated floor-control re-decision +> - choosing the uplift target (minimal vs full bump) — **OWNER-GATE**, see §6 Phase 4.0 > **Update 2026-08-01 (doc-truth pass) — Phases 2–3 SUPERSEDED. Do not build a second workflow > engine from this document.** Phases 2 and 3 below (workflow graph spec, compiler, runner, editor @@ -17,8 +43,6 @@ Every claim marked ✅ below was **verified by execution** against the live VPS > §7 open question 1 (agent-invocable workflows) was answered by **F13 workflows-as-tools** > (shipped: `list_workflows` / `run_workflow` / `get_workflow_run`, see `workflows_app.md` F13); > open question 2 (Postgres vs repo files) was answered: **Postgres** (`132_workflows.sql`). -> Phases 1 (context discipline), 4 (framework uplift) and 5 (orchestrations / collaborative chat) -> **remain live**, tracked as `work_plan.md` WS-12. --- @@ -43,9 +67,11 @@ Every claim marked ✅ below was **verified by execution** against the live VPS 5. **Phases 0–3 need no dependency change at all** ✅ — ship them on core 1.8.1 first. The framework uplift to current (core 1.11 + satellites) is a **separate, worthwhile cycle** — not just the price - of orchestrations, but a **maintenance dividend** that retires ≥4 live workarounds (§5.5). Its cost - is two forced vendor-SDK majors + one breaking AG-UI change (§4.0/§7). Sequence it after Phases 0–1, - before/merged-with Phase 5 — not folded into the bug-fix. + of orchestrations, but a **maintenance dividend** that retires ≥4 live workarounds (§5.5), **all + four of which were re-verified present in the tree on 2026-08-03**. Its cost is **one** forced + vendor-SDK major + one breaking AG-UI change (§4.0/§7) — *corrected 2026-08-03: the openai + 1.99 → 2.x major landed independently and is already in `uv.lock` at `openai 2.38.0`.* + Phases 0–1 have since shipped and Phase 5 has been reassigned, so this is now the whole document. --- @@ -64,9 +90,15 @@ planner needs to email a file Only the first link is real. Everything after is fallback flailing that generated unrelated errors. -### 2.2 Root cause A — `call_agent` is not in the guaranteed floor +### 2.2 Root cause A — `call_agent` is not in the guaranteed floor *(historical — fixed by Phase 0.1)* -`_CORE_STANDARD_TOOL_NAMES` ([`_tool_injection.py:41-48`](../../apps/services/orchestrator/orchestrator/_tool_injection.py#L41-L48)) +> **Anchor corrected 2026-08-03.** `_CORE_STANDARD_TOOL_NAMES` now spans +> [`_tool_injection.py:41-65`](../../apps/services/orchestrator/orchestrator/_tool_injection.py#L41-L65) +> and **does** contain `call_agent` / `call_agents_parallel` / `call_agent_background` (Phase 0.1, +> landed 2026-07-22, with the rationale below quoted in its own comment). The list quoted next is the +> floor **as it was when the bug happened** — read §2 as the incident record, not current state. + +At the time of the incident, `_CORE_STANDARD_TOOL_NAMES` guarantees every tool an agent needs to **work alone** — and not one tool it needs to **hand off**: ``` @@ -189,26 +221,40 @@ Handoff MAF-only -> ValueError: Handoff workflows require all participant ag Confirmed by the docs: *"Handoff orchestration only supports `Agent` and the agents must support local tools execution."* -### 3.4 Dependency reality ✅ (PyPI, 2026-07-18) +### 3.4 Dependency reality (PyPI snapshot 2026-07-18 — **stale, must be re-resolved**) + +> ⚠️ **This table is a 2026-07-18 PyPI snapshot, not current state.** Version numbers on PyPI move; +> nobody may bump against these numbers. **Phase 4.1 must re-resolve the whole set from PyPI at +> build time** and record what it actually got — the table below is background only. ``` agent-framework-orchestrations latest 1.0.0 requires core <2,>=1.9.0 prod today: agent-framework-core 1.8.1 ← below that floor + (re-verified 2026-08-03: uv.lock and the repo .venv both carry core 1.8.1) Latest satellites all pin core >=1.11.0 → coupled, move together: - agent-framework-openai 1.10.1 requires core>=1.11.0 + openai>=2.25 (was 1.99 → MAJOR) + agent-framework-openai 1.10.1 requires core>=1.11.0 + openai>=2.25 agent-framework-github-copilot 1.0.0rc3 requires core>=1.11.0 + github-copilot-sdk==1.0.2 (was 0.1.32 → MAJOR) agent-framework-ag-ui 1.0.0rc8 requires core>=1.11.0 agent-framework-redis 1.0.0b260521 already current, needs only core>=1.6.0 ``` +> **Correction 2026-08-03 — the openai major already landed; Phase 4 drags ONE SDK major, not two.** +> This section, §4.0 and §7 all billed Phase 4 for `openai 1.99 → 2.x`. That is no longer true: +> `uv.lock` and the repo `.venv` both carry **`openai 2.38.0`** under an unchanged +> `agent-framework-openai 1.7.0` (whose only openai constraint is unpinned `{ name = "openai" }`), +> so the openai 2.x major came in independently of the framework bump. The **only** remaining +> forced vendor-SDK major is `github-copilot-sdk 0.1.32 → 1.0.2`, pinned exactly by +> `agent-framework-github-copilot`. §7's "two forced SDK majors" risk row is retired accordingly. + **Two shapes of upgrade.** (a) *Minimal:* core→1.9/1.10 + orchestrations, satellites untouched (they -only need `core <2`) — unlocks Phase 5, dodges both SDK majors. (b) *Full:* core→1.11 + all satellites -latest — required to reach the copilot-side fixes in §5.5, but drags **two vendor-SDK majors** (openai -1.99→2.x, github-copilot-sdk 0.1.32→1.0.2) and one breaking AG-UI change. The full-set resolves to -`core 1.11.0` in an isolated venv ✅. Prove which path resolves before committing (Phase 4.1). +only need `core <2`) — dodges the copilot-SDK major, and unlocks the orchestrations package that +**only WS-11 now consumes** (5.1, reassigned). (b) *Full:* core→1.11 + all satellites latest — +required to reach the copilot-side fixes in §5.5, but drags the **one remaining vendor-SDK major** +(github-copilot-sdk 0.1.32→1.0.2) and one breaking AG-UI change. The full-set resolved to +`core 1.11.0` in an isolated venv on 2026-07-18 ✅ — re-prove it (Phase 4.1) before acting on it. (Note: `OpenAIChatCompletionClient` already takes `model=` on 1.8.1 — no `model_id` churn; the real -churn is the two underlying SDK majors, not the framework client surface.) +churn is the underlying copilot-SDK major, not the framework client surface.) ### 3.5 Capacity ✅ (my earlier memory concern was overblown) @@ -312,15 +358,16 @@ control over what each agent sees. That is a first-class knob that directly serv uplift to current (core 1.11 + satellites) is **not** merely the price of the orchestrations package, as an earlier draft framed it. Reading the actual changelogs (core 1.9→1.11, copilot-sdk 0.1.32→1.0.x), the release wave fixes a cluster of bugs we currently **hand-work-around** — so the uplift lets us -*delete* shim code, not just add features. Confirmed still-live in our tree: +*delete* shim code, not just add features. **All four shims re-verified present 2026-08-03** (exact +anchors below) — this is the reason WS-12 stays open at all: -| Upstream fix | Version | Shim it retires (verified present) | +| Upstream fix | Version | Shim it retires (verified present 2026-08-03) | |---|---|---| -| Copilot SDK exposes `tokenPrices` + **context-window limits** on public types | copilot-sdk 1.0.2 | `COPILOT_INFINITE_SESSIONS` window-guessing (`_copilot_session.py`) | +| Copilot SDK exposes `tokenPrices` + **context-window limits** on public types | copilot-sdk 1.0.2 | `COPILOT_INFINITE_SESSIONS` window-guessing — `_copilot_session.py:75` (also read at `:122`, `:158`) | | "Disable harness compaction when max tokens not provided" (#6410) | core 1.9.0 | same false "context length exceeded", framework side | -| github-copilot function approval via `on_pre_tool_use` hook (#6750) + tool-approval middleware (#6414/#6522) | core 1.10/1.9 | `_gate_injected_tool` (exists only because `on_permission_request` skips injected tools) | -| Telemetry-context fixes: background ctx error (#6764), OTel parent ctx for deferred streams (#6709), span nesting (#6552) | core 1.10.0 | the **telemetry killswitch** — we disabled instrumentation over the ContextVar-reset bug | -| Message-injection middleware — enqueue into an active run (#6998) | core 1.11.0 | native-MAF `_nq` steering queue + write_artifact steering | +| github-copilot function approval via `on_pre_tool_use` hook (#6750) + tool-approval middleware (#6414/#6522) | core 1.10/1.9 | `_gate_injected_tool` — **defined** at `_tool_injection.py:280`, re-exported through `executor.py:85`; exists only because `on_permission_request` skips injected tools | +| Telemetry-context fixes: background ctx error (#6764), OTel parent ctx for deferred streams (#6709), span nesting (#6552) | core 1.10.0 | the **telemetry killswitch** — `executor.py:113-140` (`ENABLE_INSTRUMENTATION` read at `:138`) | +| Message-injection middleware — enqueue into an active run (#6998) | core 1.11.0 | native-MAF `_nq` steering queue — `executor.py:2680` — + write_artifact steering | | Structured-response parse fix — avoids spurious `ValidationError`/`JSONDecodeError` (#6383) | core 1.11.0 | JSON-mode fragility ([[llm-json-mode-required]]) | | `defer` / `toolSearch` native lazy tool loading + progressive MCP disclosure (#6850) | copilot-sdk 1.0.2/1.0.7, core 1.11 | hand tool-count management in `_CORE_STANDARD_TOOL_NAMES` | @@ -329,13 +376,25 @@ it is why the uplift is worth scheduling **sooner than "only when we need Magent release we skip, we keep maintaining shims for bugs already fixed upstream; the debt compounds. **The catch is real too:** the good fixes concentrate in core 1.11 + copilot-sdk 1.0.2 — i.e. the full -coordinated bump (§3.4), which drags **two vendor SDK majors** (openai 1.99→2.x, github-copilot-sdk -0.1.32→1.0.2) and **one breaking AG-UI change** (interrupt/resume canonicalization, #6925) against our -most-customized subsystem. So it is a genuine investment with a genuine payoff — scoped as its own -cycle (Phase 4), not folded into the Phase 0 bug-fix. When adopting orchestrations, still expose -Magentic/GroupChat as **node types inside a graph**, not a parallel top-level architecture. - -### 5.6 Collaborative multi-agent chat — the three shapes of "collaboration" *(deferred design note)* +coordinated bump (§3.4), which drags **one vendor SDK major** (github-copilot-sdk 0.1.32→1.0.2 — +*corrected 2026-08-03: the openai 1.99→2.x major already landed independently*) and **one breaking +AG-UI change** (interrupt/resume canonicalization, #6925) against our most-customized subsystem. So it +is a genuine investment with a genuine payoff — scoped as its own cycle (Phase 4), not folded into the +Phase 0 bug-fix. When adopting orchestrations, still expose Magentic/GroupChat as **node types inside a +graph**, not a parallel top-level architecture — **that item is now WS-11's** (see Phase 5.1). + +### 5.6 Collaborative multi-agent chat — the three shapes of "collaboration" *(design note — Shape C has SHIPPED)* + +> **Update 2026-08-03:** the analysis below stands and was **vindicated by what shipped**. Shape C +> exists today as **multiplayer rooms**, built the cheapest-first way this section recommends — a +> rule-based selector, not an LLM coordinator: `RoomAgent.role: "primary" | "mentioned"` in +> [`workbench/control_plane/src/lib/rooms.ts:31-37`](../../workbench/control_plane/src/lib/rooms.ts#L31-L37) +> ("`primary` answers an unaddressed turn; `mentioned` answers when @named") is exactly option 1 +> below, `floorMode: "open" | "driver"` (`rooms.ts:85`) is the turn discipline, and +> `apps/services/orchestrator/orchestrator/steer.py::route_turn` (`:123`) is the routing decision. +> It was built **without** `agent-framework-orchestrations` — that package is absent from `uv.lock` +> (0 occurrences, verified 2026-08-03) and never became a dependency. Keep this section as the +> reasoning record; **do not build Shape C from it.** Owner of the residue: **WS-10**. "Multiple agents collaborating" is not one thing. It is three, and only the third needs a runtime coordinator ("orchestrator"). Getting this distinction wrong leads to building L3 machinery for @@ -386,8 +445,10 @@ dynamic and open-ended. > tools actually injected (per-variant lru_cache keeps each agent's prefix > byte-stable); 0.3 `executor.tool_scope_unknown_entry` warning (catches > `ask_user`); 0.4 tests in `tests/unit/test_tool_scope_addendum.py` + -> `test_core_tool_floor.py`. Phase 1 (design.md gating, registry trim, -> re-measure) remains open. +> `test_core_tool_floor.py`. **Re-verified 2026-08-03** — the floor at +> `_tool_injection.py:41-65` contains all three delegation tools. +> *(The trailing "Phase 1 … remains open" sentence was true on 2026-07-22 and false +> by 2026-08-03; see Phase 1's own note below.)* | # | Change | File | |---|---|---| @@ -402,15 +463,41 @@ stays bounded — each target's own `request_confirmation` gate still requires a **Done when:** the planner can email a file via `call_agent("email-assistant", …)`, attaching `technical-project-planner:outputs/…` (that cross-workspace syntax **already works**). -### Phase 1 — Context discipline *(≈1 day · no deps)* - -- **1.1** Gate `design.md` (4,078 tok) on need — skip for agents that never render documents/UI. -- **1.2** Trim registry descriptions to one line. `technical-project-planner`'s entry is a - ~150-token paragraph of trigger keywords inflicted on **every other agent**; that belongs in its - own instructions. This is what makes the mesh scale past 50 agents. -- **1.3** Re-measure. Target: **7,827 → under 2,000** for a scoped agent. - -### Phase 2 — Workflow runtime *(≈1 week · no deps — core 1.8.1 suffices)* +### ~~Phase 1 — Context discipline~~ — ✅ STRUCK 2026-08-03 (shipped / moot / reassigned) + +> **Nothing in Phase 1 is work. Do not dispatch it.** Each item below is struck with its evidence. +> **Context discipline is owned by WS-23** — [`skills_registry.md`](skills_registry.md) + +> [`skills_scope_out.md`](skills_scope_out.md). Take any further prompt-budget work there, not here. + +- ~~**1.1** Gate `design.md` (4,078 tok) on need — skip for agents that never render documents/UI.~~ + **✅ SHIPPED ~6 weeks ago** as `93b93a08` *"feat(agents): design.md on demand via + `load_design_system()` — off every prompt (#191)"* — `packages/acb_skills/acb_skills/design_tools.py` + + `tests/unit/test_design_tools.py`, and `_tool_injection.py:595-597` now reads *"the full ~16KB + design.md is no longer injected into every prompt."* + ⚠️ **It shipped as a different mechanism than proposed here.** The proposal was *per-agent gating* + ("skip for agents that never render UI"); what shipped is **progressive disclosure** — `design.md` + is off *every* prompt for *every* agent and is pulled on demand via the `load_design_system` tool, + which now sits in the core floor (`_tool_injection.py:46`). Do not re-open this as "gating was + never built": the goal was met by a better mechanism. +- ~~**1.2** Trim registry descriptions to one line.~~ **MOOT — the subject does not exist.** + `technical-project-planner` is in neither `_AGENT_REGISTRY` nor `apps/agents/` (the six live agents + are `apis-config`, `app-builder`, `email-assistant`, `orchestrator`, `task-manager`, + `whatsapp-assistant`); its only trace is a stale diagnostic comment at `_copilot_session.py:59`. + All six live `config.json` descriptions are already single-line — measured 2026-08-03 at + 130–222 chars (≈32–55 tokens), zero newlines. Nothing to trim. +- ~~**1.3** Re-measure. Target: **7,827 → under 2,000** for a scoped agent.~~ + **DELIVERED BY WS-23 (S1 baseline, S4 diet).** Measured: full injected surface **19,259 → 12,644** + tokens; addendum **5,697 → 570** behind `SKILLS_INDEX_ONLY` (shipped **OFF**, OWNER-GATE). + See `skills_registry.md` §S1/§S4 and `skills_scope_out.md` §7. + 🚫 **The bare "under 2,000" target is withdrawn — it was ambiguous and kept being misread.** + Restated precisely: the target was the **addendum**, and it is **met at 570 tokens behind + `SKILLS_INDEX_ONLY`**. It was **never** a target for the whole injected-tool surface, and must not + be quoted as one: `skills_scope_out.md` §7.4 proves that reading unreachable — the 22 core-floor + schemas cost **1,252 tokens with every description deleted**, so ≤2k on the full surface is + arithmetically out of reach without progressive tool disclosure (designed and costed in + `skills_scope_out.md` §7.5, deliberately not built). + +### ~~Phase 2 — Workflow runtime~~ — 🚫 SUPERSEDED (D6) > **Update 2026-08-01 (doc-truth pass):** SUPERSEDED by the shipped Workflows app — see the banner > at the top of this document (decision D6). Kept for the record; do not implement. @@ -431,83 +518,280 @@ stays bounded — each target's own `request_confirmation` gate still requires a **Editor vocabulary is already covered by core:** `add_chain`, `add_edge`, `add_fan_out_edges`, `add_fan_in_edges`, `add_switch_case_edge_group`, `add_multi_selection_edge_group`. -### Phase 3 — Workflow editor UI *(≈1 week)* +### ~~Phase 3 — Workflow editor UI~~ — 🚫 SUPERSEDED (D6) > **Update 2026-08-01 (doc-truth pass):** SUPERSEDED by the shipped Workflows app's `/workflows` > editor — see the banner at the top of this document (decision D6). Do not implement. Node palette from the live registry · canvas → graph spec · save/load · run + live trace. -### Phase 4 — Framework uplift & migration to latest *(≈1–2 weeks · its own hardening cycle)* - -Migrate the whole `agent-framework` stack to current. Justified by the **workaround dividend** (§5.5), -not just orchestrations. Do this **after Phases 0–1 ship** (they need no deps) and **before/merged-with -Phase 5** (orchestrations needs core ≥1.9 anyway). Scope it as a standalone cycle — never fold it into -the Phase 0 bug-fix. - -**4.0 — Version target (verified on PyPI 2026-07-18).** The satellites are coupled: openai/copilot/ag-ui -*latest* all pin `core >=1.11.0`, so they move together. It is one coordinated bump, not piecemeal. - -| Package | Installed | Target | Bump drags in | +### Phase 4 — Framework uplift & migration to latest *(≈1–2 weeks · its own hardening cycle)* — **THE ONLY LIVE PHASE** + +Migrate the whole `agent-framework` stack to current. Justified by the **workaround dividend** (§5.5) — +all four shims re-verified in the tree 2026-08-03. Scope it as a standalone cycle. + +> ### ⚠️ Hazard — 4.1 must not mutate any venv it did not create +> +> The original instruction was **"never touch `/opt/acb/app/.venv`"** (the prod venv). That is still +> binding, and **on a dev box it is not enough**: 4.1 must also **never mutate the repo's own +> `.venv`**. `uv pip install`, `uv sync`, `uv lock` and `uv add` run against the project venv by +> default and would silently upgrade the tree an agent is testing against, invalidating every +> measurement in this document and breaking the local test suite. 4.1 is **evidence-only** — it +> creates a throwaway venv in a scratch directory, installs into it **by explicit +> `--python /bin/python`**, records the result, and deletes it. If a command in 4.1 would +> write to `/.venv` or `/uv.lock`, that command belongs in **4.2**, not 4.1. + +**4.0 — Version target.** *(Table below is the 2026-07-18 PyPI snapshot; §3.4's warning applies — +re-resolve before acting.)* The satellites are coupled: openai/copilot/ag-ui *latest* all pin +`core >=1.11.0`, so they move together. It is one coordinated bump, not piecemeal. + +**Choosing between the two shapes is an 🔒 OWNER-GATE.** *(Registered as an owner call, not a ticket.)* +It is a cost/risk trade the owner makes, not a fact an agent can derive. An agent may **produce the +evidence** (4.1) and must then **stop and report**. It may not run 4.2 until the owner picks a shape. +Rationale: minimal-bump leaves the copilot-SDK major and the four §5.5 shims in place; full-bump takes +the vendor major and the breaking AG-UI change against our most-customized subsystem. Neither is +"correct" — it is a schedule decision. + +| Package | Installed (verified 2026-08-03) | Snapshot target | Bump drags in | |---|---|---|---| | agent-framework-core | 1.8.1 | **1.11.0** | — | -| agent-framework-openai | 1.7.0 | **1.10.1** | **openai 1.99 → 2.x** (SDK major) | -| agent-framework-github-copilot | 1.0.0b260402 | **1.0.0rc3** | **github-copilot-sdk 0.1.32 → 1.0.2** (SDK major) | +| agent-framework-openai | 1.7.0 | **1.10.1** | — *(openai 2.x **already installed**: `uv.lock` = `openai 2.38.0`)* | +| agent-framework-github-copilot | 1.0.0b260402 | **1.0.0rc3** | **github-copilot-sdk 0.1.32 → 1.0.2** (the one remaining SDK major) | | agent-framework-ag-ui | 1.0.0rc3 | **1.0.0rc8** | breaking interrupt/resume (#6925) | | agent-framework-redis | 1.0.0b260521 | 1.0.0b260521 | **already current — no change** | -| agent-framework-orchestrations | *(none)* | **1.0.0** | needs core ≥1.9 (satisfied) | +| agent-framework-orchestrations | *(absent from `uv.lock`)* | **1.0.0** | needs core ≥1.9 (**not** satisfied at 1.8.1) | | github-copilot-sdk | 0.1.32 | 1.0.2 | **pinned exactly** by copilot rc3 (not 1.0.7) | -> **Minimal-bump fallback:** if the full jump proves too costly, orchestrations needs only **core ≥1.9**, -> and our *currently-installed* satellites only require `core <2` — so core→1.9/1.10 + orchestrations, -> leaving both vendor SDKs untouched, is a lighter path that still unlocks Phase 5 while dodging the two -> SDK majors. Prove which resolves in an isolated venv (4.1). - -**4.1 — Resolution proof (isolated venv, throwaway).** `uv venv` in `/tmp`; `uv pip install` the target -set; capture the fully-resolved version lock. **Never touch `/opt/acb/app/.venv`.** Confirm the two SDK -majors resolve and import. Decide full-bump vs minimal-bump here on evidence. - -**4.2 — Land the coordinated bump.** Update the four `pyproject.toml` pins (orchestrator, gateway, -agent-email-assistant, agent-task-manager); `uv sync`. redis unchanged. - -**4.3 — Absorb the two forced SDK majors.** openai 1.99→2.x and github-copilot-sdk 0.1.32→1.0.2 are the -real risk (7/8 agents ride the Copilot path). Re-verify against [[maf-agent-openai-client-choice]] and -[[copilot-sdk-context-window-unknown]]; check session/tool API shape on copilot-sdk 1.0.2. - -**4.4 — Migrate the one breaking AG-UI change (#6925).** Interrupt/resume is canonicalized around -`RUN_FINISHED.outcome.interrupts` + `ResumeEntry`. This hits our most-customized code — the HITL resume -path (`resolve_relay_thread_id`, `_pending_user_input` in `ask_tools.py`). Migrate deliberately; this is -where the schedule risk lives. **Gains that ride along:** SSE keepalive for silent streams (#6980 — -targets our idle-watchdog/HITL stalls), AG-UI thread snapshot persistence (#6471), clear-queued-approvals --on-cancel (#6947), preserve streamed text message id in mixed snapshots (#6269). - -**4.5 — Retire the shims, one at a time, each behind a verification.** Work the §5.5 table. For each -row, confirm the upstream fix actually covers *our* case before deleting the workaround — these are -strong candidates, not guarantees. Priority order: - 1. **Telemetry killswitch** → re-enable instrumentation, confirm the ContextVar-reset bug is gone - ([[chat-maf-telemetry-contextvar-bug]], `test_executor_telemetry_killswitch.py`). - 2. **`COPILOT_INFINITE_SESSIONS`** → read the real context window off `ModelBilling`; drop the guess. - 3. **`_gate_injected_tool`** → move to the native `on_pre_tool_use` hook (fires for injected tools too). - 4. **Native-MAF `_nq` steering** → evaluate message-injection middleware (#6998) as a replacement. - -**4.6 — Gate.** Full eval suite (21/21) + prod build + a manual soak of the Copilot streaming path -before merge. Deploy `git reset --hard`s and `uv sync`s, so the lock must be committed and clean. - -### Phase 5 — Pre-built orchestrations + collaborative chat *(depends on Phase 4)* - -Unlocks **Shape C** (free-form collaborative multi-agent chat, §5.6). Shapes A and B do **not** depend -on this. - -- **5.1** With orchestrations installed (Phase 4), expose Magentic/GroupChat as **node types inside a - graph**, not a parallel top-level architecture. -- **5.2 Collaborative chat surface (Shape C).** New chat mode where N registered agents share one - conversation. Coordinator picked cheapest-first (§5.6): start with a **`selection_func`** (round-robin - / rule-based, no LLM, no MAF-`Agent` constraint); add `orchestrator_agent` (MAF-typed) or a Magentic - manager only if dynamic routing is genuinely needed. Participants may be mixed runtime. Requires a - **termination condition** + turn cap up front (§5.6 reliability caveat). Reuses the Phase 2 multi-loader - and the SSE relay. -- **5.3** `HandoffBuilder`: MAF-only. Skip, migrate specific agents to native MAF, or use Magentic - instead. **Do not** rewrite all Copilot agents for this. +> **Minimal-bump fallback:** orchestrations needs only **core ≥1.9**, and our *currently-installed* +> satellites only require `core <2` — so core→1.9/1.10 + orchestrations, leaving the vendor SDK +> untouched, is a lighter path that dodges the copilot-SDK major. Note its payoff shrank on +> 2026-08-03: the orchestrations package it unlocks now has **exactly one consumer left** — WS-11's +> 5.1 — since Phase 5.2 shipped without it. Produce the evidence in 4.1; the owner picks. + +--- + +**4.1 — Resolution proof (isolated throwaway venv). 🟢 AGENT-SAFE — evidence only; produce it, do not choose.** + +**Done when:** a committed `docs/framework-uplift/framework-uplift-resolution.md` exists containing +**both** fully-resolved lock sets, each produced in its own throwaway venv: + +| Set | Contents | +|---|---| +| **minimal** | `agent-framework-core` (1.9/1.10 line) + `agent-framework-orchestrations`, satellites left at their currently-installed versions | +| **full** | `agent-framework-core` 1.11 line + `agent-framework-openai` + `agent-framework-github-copilot` + `agent-framework-ag-ui` + `agent-framework-redis` + `agent-framework-orchestrations` | + +and for **each** set the document records, verbatim: +1. the exact `uv pip install` command that produced it (including the explicit + `--python /…` that kept it off the repo venv); +2. the full `uv pip list` output of the resolved venv (this **is** the lock set — no hand-typed + version tables, and no version number quoted from `§3.4`'s stale snapshot); +3. an **import smoke** whose output is pasted in, run inside that venv, covering all three surfaces — + `import agent_framework`, `import agent_framework.openai`, `import agent_framework_github_copilot` + — printing each module's `__version__` (or `importlib.metadata.version`) and exiting non-zero on + any ImportError; +4. any resolution **conflict or backtrack** uv reported, quoted, or the explicit line + "no conflicts reported"; +5. the throwaway venv's path and proof it was deleted. + +**And the document must NOT contain a recommendation, a preference, or a chosen shape.** Its last +section is titled *"Evidence for the owner's 4.0 decision"* and states the trade-off neutrally. +An agent that finishes 4.1 **stops and reports** (🔒 the 4.0 choice). + +**Not done if:** `/.venv` or `/uv.lock` changed. Verify with `git status --short uv.lock` +(must be empty) and by confirming `uv pip list --python .venv/…` still shows `agent-framework-core +1.8.1` / `openai 2.38.0` after 4.1 completes. + +--- + +**4.2 — Land the coordinated bump. 🟢 AGENT-SAFE** *(but blocked until the owner resolves 4.0)* + +Update the `pyproject.toml` pins that carry `agent-framework-*` and `uv sync`; redis unchanged. + +**Done when:** (a) `uv.lock` is committed and `git status --short uv.lock` is empty after a fresh +`uv sync`; (b) every `agent-framework-*` version in `uv.lock` matches the owner-chosen lock set from +4.1 **exactly**, with no package resolved outside it; (c) the same three-import smoke from 4.1 passes +against the repo venv; (d) the Phase-4 verification block below is green (see §6.4v). +**Anti-drift:** the four `pyproject.toml` files were named as *orchestrator, gateway, +agent-email-assistant, agent-task-manager* on 2026-07-17 — **re-derive the list at build time** +(`grep -rln "agent-framework" --include=pyproject.toml .`) rather than trusting that list; the +`apps/agents/` tree has changed since. + +--- + +**4.3 — Absorb the forced SDK major. 🟢 AGENT-SAFE** + +`github-copilot-sdk 0.1.32 → 1.0.2` is the real risk — **6 of 6** in-repo agents ride the Copilot +path. *(The "openai 1.99→2.x" half of this item is struck: already at 2.38.0.)* Re-verify against +[[maf-agent-openai-client-choice]] and [[copilot-sdk-context-window-unknown]]. + +**Done when:** (a) a written diff-review of the copilot-SDK **session and tool-call API shape** +(`create_session` / session options / tool-registration / permission-hook signatures) between 0.1.32 +and 1.0.2 is recorded in the same `docs/framework-uplift/` file, naming every call site in +`_copilot_session.py` and `copilot_agent.py` that the change touches, or stating "no signature +change" per surface; (b) `uv run python -m pytest tests/unit/test_hitl_both_runtimes.py +tests/unit/test_ask_user_hitl.py tests/unit/test_permission_policy.py -q` is green; (c) the model-tier +resolution path still resolves — `uv run python -m pytest tests/unit/test_model_resolution.py -q` +(substitute the real filename if it has moved; derive it, do not trust this line). + +--- + +**4.4 — Migrate the one breaking AG-UI change (#6925). 🟢 AGENT-SAFE** + +Interrupt/resume is canonicalized around `RUN_FINISHED.outcome.interrupts` + `ResumeEntry`. This hits +our most-customized code — the HITL resume path. + +> **Anchors corrected 2026-08-03.** The previous text placed `resolve_relay_thread_id` and +> `_pending_user_input` in **`ask_tools.py`** — **that file does not exist anywhere in the repo.** +> Both live in `apps/services/orchestrator/orchestrator/executor.py`: +> `resolve_relay_thread_id` at **`:220`** and `_pending_user_input` at **`:339`** +> (*not* `:257` — line 257 is a comment inside the neighbouring `_active_elicitation_request_id` +> ContextVar that merely mentions the name). Re-derive both with +> `grep -n "def resolve_relay_thread_id\|^_pending_user_input" apps/services/orchestrator/orchestrator/executor.py` +> before editing; this file changes often. + +**Done when:** (a) the HITL resume path builds and parks/resumes on the new +`RUN_FINISHED.outcome.interrupts` + `ResumeEntry` shape with no compatibility shim left behind; +(b) `uv run python -m pytest tests/unit/test_hitl_both_runtimes.py tests/unit/test_ask_user_hitl.py +tests/unit/test_hitl_heartbeat.py tests/unit/test_hitl_stall_suppression.py +tests/unit/test_genui_hitl.py -q` is green **without** any test being skipped or xfailed as part of +the migration (a skipped HITL test is a failed 4.4); (c) `uv run python -m pytest +evals/trajectories/test_hitl_trajectory.py evals/trajectories/test_stream_replay_trajectory.py -q` +is green. +**Gains that ride along:** SSE keepalive for silent streams (#6980 — targets our idle-watchdog/HITL +stalls), AG-UI thread snapshot persistence (#6471), clear-queued-approvals-on-cancel (#6947), +preserve streamed text message id in mixed snapshots (#6269). + +--- + +**4.5 — Retire the shims, one at a time, each behind its own verification.** + +Work the §5.5 table. For each row, confirm the upstream fix actually covers *our* case before +deleting the workaround — these are strong candidates, **not guarantees**, which is why each row +below carries its own done-when rather than one gate for the set. + +**The shape of every done-when here is the same:** the named test is green **today with the shim in +place**; it must be green **after the shim is deleted**, with the assertions that pin the shim's +existence replaced by assertions that pin the upstream behaviour — not deleted, not skipped. + +- **4.5.1 — Telemetry killswitch** (`executor.py:113-140`). 🔒 **OWNER-GATE — do not flip.** + Already registered in `work_plan.md` §6 under **WS-6 observability activation** + (*"re-enabling the MAF telemetry kill switch … it hides a known ContextVar-reset bug"*; §6 cites + `executor.py:114`, the comment banner — the env read is at `:138`). + **An agent may:** delete the shim's *code* behind an unchanged default and demonstrate the + ContextVar-reset bug is gone upstream. **An agent may not:** make instrumentation on-by-default, + or set `ENABLE_INSTRUMENTATION=1` in any committed env/deploy file. + **Done when:** `tests/unit/test_executor_telemetry_killswitch.py` goes from asserting *"we disable + agent_framework instrumentation unless opted in"* to asserting the post-uplift contract, and is + green with `_disable_agent_telemetry_once` removed; plus a recorded streamed-run trace showing no + `"Token was created in a different Context"` at end-of-run ([[chat-maf-telemetry-contextvar-bug]]). + Command: `uv run python -m pytest tests/unit/test_executor_telemetry_killswitch.py -q` +- **4.5.2 — `COPILOT_INFINITE_SESSIONS`** (`_copilot_session.py:75`). 🟢 AGENT-SAFE. + **Done when:** the real context window is read off the SDK's public model type (`ModelBilling` / + `tokenPrices`) instead of guessed, all three env overrides (`COPILOT_INFINITE_SESSIONS`, + `COPILOT_COMPACTION_THRESHOLD`, `COPILOT_BUFFER_THRESHOLD`) are gone from + `_copilot_session.py`, and a test asserts the window comes from the SDK for a gateway-routed BYOK + model. Command: `uv run python -m pytest tests/unit/test_copilot_session.py -q` + *(derive the real filename; if no such test exists, adding it is part of 4.5.2.)* +- **4.5.3 — `_gate_injected_tool`** (`_tool_injection.py:280`, re-exported at `executor.py:85`). + 🟢 AGENT-SAFE. **Done when:** injected tools are gated by the native `on_pre_tool_use` hook and + `_gate_injected_tool` is deleted, with **`tests/unit/test_permission_policy.py` green** — that + file is the shim's pin (it is the only test referencing `_gate_injected_tool` by name) and its + `test_inject_rewraps_repo_baked_tools`-style assertions must be **rewritten to assert the hook + fires for injected tools**, not removed. Fail-closed behaviour must not regress (root AGENTS.md + harness rule 2). Commands: `uv run python -m pytest tests/unit/test_permission_policy.py -q` and + `uv run python -m pytest evals/trajectories/test_permission_trajectory.py -q` +- **4.5.4 — Native-MAF `_nq` steering queue** (`executor.py:2680`). 🟢 AGENT-SAFE. + **Done when:** message-injection middleware (#6998) replaces the hand-rolled queue, `_nq` and + `_active_run_queue` are gone from `executor.py`, and the steer contract is unchanged — + `tests/unit/test_steer_routing.py` (DROP/ENGAGE/ABORT/STEER, `202 {"steered": true}`, + `409 steer_outside_run_floor`) and `tests/unit/test_supersede_guard.py` green with **no assertion + weakened**. This one is coupled to WS-10: steer shipped on `_nq` (`15c8933f`), so 4.5.4 is a + behaviour-preserving swap under a shipped feature — treat any assertion change as a red flag. + Command: `uv run python -m pytest tests/unit/test_steer_routing.py tests/unit/test_supersede_guard.py -q` + +--- + +**4.6 — Gate.** + +> **Corrected 2026-08-03: there is no "21/21 eval suite".** The offline trajectory suite collects +> **135 tests** (`uv run python -m pytest evals/trajectories/ -q --collect-only`, measured +> 2026-08-03). The "21/21" figure was stale and must not be used as a pass criterion. + +**Done when:** +1. `uv run python -m pytest evals/trajectories/ -q` shows **no new failures relative to the + pre-uplift baseline recorded in `docs/framework-uplift/`** — record the baseline *before* 4.2. + ⚠️ *Baseline is not zero on Windows:* measured 2026-08-03 on this box, **6 failed / 129 passed**, + 5 of them `ValueError: preexec_fn is not supported on Windows platforms` (the Module Studio + subprocess runner) plus `test_chat_fold_trajectory.py::test_cancelled_run_still_persists_partial_turn`. + Take the baseline on the **same** machine you will re-measure on. +2. the Phase-4 unit block below (§6.4v) is green; +3. the control-plane build is clean (`npm run build` in `workbench/control_plane`); +4. `uv.lock` is committed and clean — deploy `git reset --hard`s and `uv sync`s, so a dirty lock + breaks prod; +5. 🔒 **OWNER-GATE — the manual soak.** A human drives the Copilot streaming path (a streamed run, + an `ask_questions` park-and-resume, a steer mid-run) and signs off. **An agent cannot perform, + simulate or self-certify this**, and must not mark 4.6 done without a recorded human sign-off. + An agent that reaches this point stops and reports. + +#### §6.4v — Phase-4 verification block (run these; never `tests/unit/` as a directory) + +> ⚠️ **Never run `tests/unit/` as a directory on a dev box.** It hangs — this box's `.env` has Mem0 +> enabled and `test_memory_integration.py` blocks on the live DB. Name files. Also never run +> `test_owner_bootstrap.py`, `test_memory_integration.py`, `test_memory_e2e.py`, +> `test_run_agent_stream_e2e.py`, `test_debug_routes.py` locally. CI runs the directory; you do not. + +```bash +# Shim pins — must be green BEFORE the uplift (baseline) and AFTER each 4.5 deletion. +uv run python -m pytest \ + tests/unit/test_executor_telemetry_killswitch.py \ + tests/unit/test_core_tool_floor.py \ + tests/unit/test_tool_scope_addendum.py \ + tests/unit/test_own_tool_scope.py -q +# measured 2026-08-03, pre-uplift baseline: 21 passed in 74.40s + +# Permission gate + HITL/steer contracts (4.3 / 4.4 / 4.5.1 / 4.5.3 / 4.5.4). +uv run python -m pytest \ + tests/unit/test_permission_policy.py \ + tests/unit/test_hitl_both_runtimes.py \ + tests/unit/test_ask_user_hitl.py \ + tests/unit/test_steer_routing.py -q +# measured 2026-08-03, pre-uplift baseline: 81 passed in 74.92s + +# Offline trajectory evals (4.6 gate 1) — compare to the recorded baseline, not to zero. +uv run python -m pytest evals/trajectories/ -q +# measured 2026-08-03 on Windows: 6 failed, 129 passed (see 4.6 note 1) + +# Lock hygiene (4.1 must leave this empty; 4.2 must leave it empty after committing). +git status --short uv.lock + +uv run ruff check . +``` + +### ~~Phase 5 — Pre-built orchestrations + collaborative chat~~ — ✅ STRUCK 2026-08-03 (shipped / reassigned) + +> **No item in Phase 5 is WS-12 work.** 5.1 moved to WS-11, 5.2 shipped as multiplayer rooms, +> 5.3 was always "skip". Nothing here is dispatchable from this document. + +- **5.1 — ➡️ REASSIGNED to WS-11 / [`workflows_app.md`](workflows_app.md) §8.** *(Expose + Magentic/GroupChat as **node types inside a graph**, not a parallel top-level architecture.)* + It belongs there because the graph, its node catalog and its compiler are all owned by the + Workflows app under D6 — adding a node type to someone else's engine from this document is exactly + the parallel seam D6 exists to prevent. **It is the only remaining consumer of + `agent-framework-orchestrations`, and therefore sequences AFTER Phase 4** (the package needs + core ≥1.9; we are on 1.8.1 and the package is absent from `uv.lock`). No acceptance is written for + it here — WS-11 writes it, against its own node-catalog contract. +- ~~**5.2 Collaborative chat surface (Shape C).**~~ **✅ SHIPPED as multiplayer rooms — and shipped + the way this spec recommended.** The §5.6 "cheapest-first" guidance was to start with a rule-based + `selection_func` rather than an LLM coordinator; what shipped is exactly that, expressed natively: + `RoomAgent.role: "primary" | "mentioned"` (`workbench/control_plane/src/lib/rooms.ts:31-37` — + *"`primary` answers an unaddressed turn; `mentioned` answers when @named"*), the turn discipline as + `floorMode: "open" | "driver"` (`rooms.ts:85`), and the routing decision as + `orchestrator/steer.py::route_turn` (`:123`, DROP/ENGAGE/ABORT/STEER). + **It was built without the orchestrations package** — absent from `uv.lock`, verified 2026-08-03 — + so Phase 5.2 never depended on Phase 4 after all. Residue (the floor-control re-decision) is + **WS-10's and 🔒 OWNER-GATE**, registered in `work_plan.md` §6 by name. Do not re-open it here. +- ~~**5.3** `HandoffBuilder`: MAF-only.~~ **Unchanged and still "skip"** — not work, a standing + decision (§5.1: handoff is the wrong pattern for our case). **Do not** rewrite Copilot agents for it. --- @@ -515,12 +799,15 @@ on this. | Risk | Severity | Mitigation | |---|---|---| -| Two forced SDK majors (openai 1.99→2.x, copilot-sdk 0.1.32→1.0.2) | **high** | Phase 4.1 isolated-venv proof + 4.3; minimal-bump fallback dodges both; not needed for 0–3 | -| Breaking AG-UI interrupt/resume (#6925) vs our custom HITL resume | **high** | Phase 4.4 deliberate migration; the schedule risk lives here | -| Shim removal deletes a workaround the fix doesn't fully cover | med | 4.5 verifies each fix against our case *before* deleting; one at a time | -| Floor-wide `call_agent` widens reach | low | per-target confirm gate + depth/cycle guards already exist | -| N live agents exhaust 4 GB | low–med | in-process shared `sys.modules`; measure in 2.6; 4 GB swap available | -| Graph spec churn after editor ships | med | version the spec in 2.3 **before** UI work | +| ~~Two forced SDK majors (openai 1.99→2.x, copilot-sdk 0.1.32→1.0.2)~~ | ~~**high**~~ **RETIRED 2026-08-03** | **Half of this risk expired on its own.** `uv.lock` + the repo `.venv` both carry `openai 2.38.0` under an unchanged `agent-framework-openai 1.7.0` — the openai major landed independently of Phase 4. Successor row below. | +| **One** forced SDK major (github-copilot-sdk 0.1.32→1.0.2) | **high** | 6/6 in-repo agents ride the Copilot path. Phase 4.1 isolated-venv proof + 4.3's API-shape diff; the minimal-bump fallback dodges it entirely (🔒 owner picks, 4.0) | +| Breaking AG-UI interrupt/resume (#6925) vs our custom HITL resume | **high** | Phase 4.4 deliberate migration; the schedule risk lives here. Its anchors were wrong until 2026-08-03 (`ask_tools.py` does not exist) — re-derive before editing | +| Shim removal deletes a workaround the fix doesn't fully cover | med | 4.5 verifies each fix against our case *before* deleting; one at a time, each with its own named test | +| 4.5.4 regresses shipped steer (WS-10) | med | `_nq` is load-bearing for a **shipped** feature (`15c8933f`); 4.5.4 is a behaviour-preserving swap — any weakened assertion in `test_steer_routing.py` is a red flag | +| 4.1 mutates the repo `.venv` / `uv.lock` while "just proving resolution" | med | the hazard note at the head of Phase 4; `git status --short uv.lock` must be empty when 4.1 ends | +| Bumping against §3.4's stale PyPI snapshot | med | §3.4 and §4.0 are marked 2026-07-18 snapshots; 4.1 must re-resolve from PyPI and record what it got | +| ~~Floor-wide `call_agent` widens reach~~ | low | **Shipped 2026-07-22 (Phase 0.1)** and the mitigation held: per-target confirm gate + depth/cycle guards | +| ~~N live agents exhaust 4 GB~~ · ~~Graph spec churn after editor ships~~ | — | **Moot** — both belonged to Phases 2–3, superseded by the Workflows app (D6) | | `HandoffBuilder` never supports Copilot | low | we don't need handoff semantics (§5.1) | **Open questions** @@ -528,13 +815,50 @@ on this. *(Answered — see the banner at the top: F13 workflows-as-tools shipped.)* 2. Where do workflow definitions live — Postgres (survives `git reset --hard`) or repo files? Precedent says **Postgres**. *(Answered — see the banner at the top: Postgres, migration 132.)* -3. Does a node need HITL? `AgentExecutor` supports `request_info`; confirm it survives our SSE relay. -4. Do we cap live nodes per workflow (`commandcenter-dev` alone is a 2.1 GB clone)? +3. ~~Does a node need HITL? `AgentExecutor` supports `request_info`; confirm it survives our SSE relay.~~ + *(Moot here — Phases 2–3 superseded. Workflow-node HITL shipped as the Action Broker inbox + pause/resume in `workflows_app.md`. The HITL-vs-SSE-relay question that **is** still live is + Phase 4.4's, and it is scoped there.)* +4. ~~Do we cap live nodes per workflow (`commandcenter-dev` alone is a 2.1 GB clone)?~~ + *(Moot here — belongs to `workflows_app.md` under D6.)* + +**No open questions remain in this document.** Every decision Phase 4 needs is either recorded above +or explicitly marked 🔒 OWNER-GATE (the 4.0 target choice, 4.5.1's killswitch flip, 4.6's manual soak). +If an agent finds itself needing a decision that is not one of those three, the spec has drifted — +stop and report rather than choosing. --- ## 8. Appendix — reproduction +> ⚠️ **§8 is NOT agent-runnable and is NOT the verification block.** Every command below requires +> prod SSH (`ssh acb@…`) — an agent has no such reach and must not attempt it, and probes A/B were +> run against a **2026-07-17 prod state that has since changed** (Phase 0 landed; the floor now +> contains `call_agent`, so probe A would now print `True`). Kept as the incident record. +> **The commands you actually run are §6.4v** (Phase-4 verification block) and §8.1 below. + +### 8.1 Local, agent-runnable equivalents + +```bash +# Shim + version facts this document depends on — all offline, all local. +grep -n "^_CORE_STANDARD_TOOL_NAMES" -A 25 apps/services/orchestrator/orchestrator/_tool_injection.py +grep -n "def resolve_relay_thread_id\|^_pending_user_input" apps/services/orchestrator/orchestrator/executor.py +grep -n "ENABLE_INSTRUMENTATION" apps/services/orchestrator/orchestrator/executor.py +grep -n "COPILOT_INFINITE_SESSIONS" apps/services/orchestrator/orchestrator/_copilot_session.py +grep -n "def _gate_injected_tool" apps/services/orchestrator/orchestrator/_tool_injection.py +grep -n "_nq: asyncio.Queue" apps/services/orchestrator/orchestrator/executor.py + +# Installed versions — the §3.4 / §4.0 "Installed" column. Read-only. +uv pip list | grep -E "agent-framework|^openai |github-copilot-sdk" +grep -c "agent-framework-orchestrations" uv.lock # 0 today — the package is absent + +# Shape C's shipped selector (Phase 5.2). +sed -n '31,37p;85p' workbench/control_plane/src/lib/rooms.ts +grep -n "def route_turn" apps/services/orchestrator/orchestrator/steer.py +``` + +### 8.2 Original prod probes (2026-07-17 · prod SSH · **not agent-runnable**) + ```bash ssh acb@187.127.179.143 @@ -585,3 +909,13 @@ rm -rf /tmp/orchprobe # ALWAYS clean up Related specs: [`agent_file_and_memory_framework.md`](agent_file_and_memory_framework.md) · [`core_module_map.md`](core_module_map.md) · [`harness_hardening_2026-07.md`](harness_hardening_2026-07.md) + +**Where this document's struck phases went** (added 2026-08-03 — follow these, not the struck text): + +| Struck here | Now owned by | +|---|---| +| Phase 1 (context discipline / prompt budget) | **WS-23** — [`skills_registry.md`](skills_registry.md) · [`skills_scope_out.md`](skills_scope_out.md) | +| Phases 2–3 + §5.3 (graph, compiler, runner, editor) | **`workflows_app.md`** per D6 | +| Phase 5.1 (Magentic/GroupChat as graph node types) | **WS-11** — [`workflows_app.md`](workflows_app.md) §8; sequences after Phase 4 | +| Phase 5.2 (Shape C collaborative chat) | **WS-10** — shipped as multiplayer rooms (`docs/multiplayer/README.md`); floor-control residue is 🔒 OWNER-GATE | +| **Phase 4 (framework uplift)** | **stays here — WS-12** | diff --git a/ai-company-brain/specs/permissions_sandbox_b6.md b/ai-company-brain/specs/permissions_sandbox_b6.md index e48f84fe6..caa564c73 100644 --- a/ai-company-brain/specs/permissions_sandbox_b6.md +++ b/ai-company-brain/specs/permissions_sandbox_b6.md @@ -1,7 +1,26 @@ # B6 — Permissions & Sandboxing (HH-6) -> **Status:** Near-term handler **shipped (2026-07-03)** — B6 grade C → B−. **Phase 5 (isolation) in progress (2026-07-04)** — see the "Phase 5" section below. +> **Status: verified against code on 2026-08-03** (truth pass, WS-3). Near-term +> risk-aware permission handler **shipped 2026-07-03** and still wired at all five +> executor sites. Phase 5 (isolation): **P5-a shipped** (per-run credential scoping, +> 2026-07-04) · **P5-b partially shipped** (container cap/resource ceilings landed +> 2026-07-27; egress + read-only rootfs unbuilt) · **P5-c (T2) parked as a +> deprioritised sub-project** under the internal-tool threat model (owner decision +> 2026-08-03) · **P5-d not started.** The two dispatchable slices are **WS-3a** +> (§P5-a.2) and **WS-3b** (§P5-b.2). Board row: `work_plan.md` §2 WS-3. > **Module:** B6 (core_module_map.md). +> +> **Isolation ladder (R2).** This doc's Phase-5 build order is lettered **P5-a/b/c/d**. +> The **isolation-strength ladder is T0/T1/T2**, defined once in +> [`agent_platform_hardening_2026-07.md`](agent_platform_hardening_2026-07.md) §1.2 and +> implemented in `AgentManifest.isolation_tier()`. Until 2026-08-03 this doc used +> "Tier 0/1/2/3" for its build order while the hardening doc used "T0/T1/T2" for +> isolation strength — two incompatible ladders inside one board cell. They are not the +> same thing and the numbers never lined up: this doc's "Tier 2" (generalise the +> container to live runs) is the hardening doc's **T2**, but this doc's "Tier 1" (harden +> the mutation container) has **no T-equivalent at all** — it hardens an existing +> container rather than choosing a tier for a run. Say **T0/T1/T2** when you mean +> isolation strength; say **P5-a…d** when you mean this doc's build order. > **Scope of THIS pass:** replace the blanket `PermissionHandler.approve_all` > with a **risk-aware allowlist handler** that gates shell / file-write / > network / tool operations using the SDK's own request classification + our @@ -9,10 +28,15 @@ > isolation for normal runs — the in-process `importlib` execution model stays; > that's a much larger infra change tracked separately. -## The gap (audited 2026-07-03) +## The gap (audited 2026-07-03 — **closed**; kept as the record of what was wrong) -- **Copilot-SDK agents run with `PermissionHandler.approve_all`** — set at FIVE - sites in `executor.py` (`~1190, ~2572, ~3023, ~3796, ~4297`), always as +> **Anchor refresh 2026-08-03.** The five sites are now +> `executor.py:632, 2483, 3011, 3909, 4442` and every one of them installs +> `_copilot_permission_handler()`, not `_PH.approve_all`. The paragraph below +> describes the **pre-2026-07-03** state. + +- **Copilot-SDK agents ran with `PermissionHandler.approve_all`** — set at FIVE + sites in `executor.py` (then `~1190, ~2572, ~3023, ~3796, ~4297`), always as `if agent._permission_handler is None: agent._permission_handler = _PH.approve_all`. `approve_all` returns `PermissionRequestResult(kind="approved")` for EVERY request: every shell command, file write, and network fetch the model decides @@ -62,6 +86,14 @@ The handler consults `tool_annotations.get_annotations` for named tools and the workspace root from `write_artifact._WRITE_ARTIFACT_CONTEXT` for the file-write-scope check (the same plain-dict context the tools already use). +> **Gate labels (added 2026-08-03, contract point 7).** The handler, its policy +> table, and its wiring are **AGENT-SAFE** and shipped. Moving +> `AGENT_PERMISSION_MODE` off `audit` to the enforcing mode is **OWNER-GATE** +> (`work_plan.md` §6) — an agent must refuse it and say so. Note the honest +> discrepancy: the code's *default* is the enforcing mode, but **prod is pinned +> to `audit`** (see the 2026-07-03 production-verification entry below), so +> reading this section's "default" as the live posture is wrong. + ## Wiring Replace `_PH.approve_all` at all five executor sites with our handler (guarded: if `AGENT_PERMISSION_MODE=approve_all`, keep `_PH.approve_all`). Handler lives @@ -82,6 +114,14 @@ shell/file/network requests surface. decision table is locked as the contract. ## Status + +> **A third "Tier" lives in this section (R2 warning).** The 2026-07-03 entry +> below says "Native-MAF **Tier-2** `_make_tool_shim`" and "Native-MAF **Tier-1** +> streaming". Those are the **MAF runtime tiers** (which execution path a run +> takes), not isolation strength (`T0/T1/T2`) and not this doc's build order +> (`P5-a…d`). Three unrelated ladders share the word. Left as-is because the +> entry is a historical record, but do not read them across. + - 2026-07-03 — Design from the B6/HH-6 audit. Building the handler + wiring. - 2026-07-03 — **Shipped.** `acb_skills/permission_policy.py` (`decide` pure fn + `risk_aware_permission_handler`). Wired into all FIVE @@ -120,39 +160,106 @@ shell/file/network requests surface. # B6 Phase 5 — Isolation for normal agent runs -> **Status:** In progress (2026-07-04). This is the deferred deep-isolation -> work — the "real residual excessive-agency exposure" the module map flags. -> The near-term permission handler above is the *policy* layer inside the -> process; Phase 5 adds the *boundary*. +> **Status: verified against code on 2026-08-03.** P5-a shipped · P5-b partly +> shipped · P5-c parked · P5-d not started. See §"What actually shipped" below — +> that table, not the prose, is the state of record. The near-term permission +> handler above is the *policy* layer inside the process; Phase 5 adds the +> *boundary*. -## The exposure, precisely (audited 2026-07-04) +## What actually shipped (verified against code on 2026-08-03) + +The prose in this section was written 2026-07-04 and went 30 days without a +reconciliation pass while four separate things landed. Everything below was +re-checked against the tree at `2ccff9e0` before being written here. + +| Slice | State (2026-08-03) | Evidence | +|---|---|---| +| **P5-a — per-run credential scoping** (was "Tier 0") | ✅ **SHIPPED** | `_inject_integrations_to_env` now returns a **restore token** and `_restore_integration_env` tears it down at run end — `executor.py:4340-4389` (fn) / `:4392` (restore). Called + restored on all three run paths: `_run_sub_agent_streaming` (`:599` / `:843`), `run_agent_stream` (`:2335` / `:4053`), `_run_with_maf_agent` (`:4516`). Pinned by `tests/unit/test_integration_env_scoping.py` + `evals/trajectories/test_integration_env_scoping_trajectory.py` | +| **P5-b — container resource + capability ceilings** (was part of "Tier 1") | ✅ **SHIPPED 2026-07-27** | `mutation.py:700-722` and `copilot_sandbox.py:153-171` both pass `--cap-drop ALL`, `--cap-add DAC_OVERRIDE`, `--security-opt no-new-privileges`, `--memory`, `--cpus`, `--pids-limit`, all settings-overridable. Pinned by `tests/unit/test_mutation_sandbox_hardening.py` and `tests/unit/test_copilot_sandbox.py` | +| **P5-b — egress + read-only rootfs** | 🔲 **UNBUILT** | Neither `docker run` passes `--network`, `--read-only`, or any allowlist. This is **WS-3b** (§P5-b.2) | +| **P5-b — scoped gateway key for the sandbox** | 🔲 **unbuilt and undesigned** | `mutation.py:700-722` still passes `GATEWAY_API_KEY` straight through. TTL, issuance and revocation are all unanswered — **OWNER-GATE** (see §P5-b.3) | +| **Copilot-CLI containerization** (T2-*shaped*, but not T2) | ✅ **SHIPPED, wired at 2 call sites, ships OFF** | `orchestrator/copilot_sandbox.py` + `Dockerfile.copilot-sandbox`; call sites `code_session.py:109-120` (`code_task`) and `executor.py:1070-1080` (`_maybe_sandbox_session_workspace`, App Workshop app-builder). Gated on `settings.copilot_sandbox_scope` (`acb_common/settings.py:222`, default `""`), hard fallback to in-process on any spawn failure. It containerizes the **`copilot` CLI binary**, not the agent run — the host still owns orchestration, tools and permissions, so it is not T2 | +| **`isolation_tier()` derivation** | ⚠️ **SHIPPED AS A LOG LINE ONLY** | `manifest.py:273-287` computes T0/T1/T2 from the resolved surface; pinned by `tests/unit/test_agent_manifest.py:224-252`. Its only non-test consumer is `declarative.py:210`'s `_log.info("declarative.agent_built", …, tier=…)`, plus a registration warning at `manifest.py:370-374`. **Computed and thrown away** | +| **Tier record on `agent_run` + T2-run refusal** | 🔲 **UNBUILT** | No `tier` column exists — checked `infra/postgres/`; highest migration on disk is 142. Nothing refuses a run. This is **WS-3a** (§P5-a.2) | +| **P5-c (T2 proper)** | 🔲 **untouched, and now parked** | `loader.py:1300`'s in-process `spec.loader.exec_module` is what `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO‑7 and `competitive_hardening_2026-07.md` CH‑1 actually name, and neither 2026-07-27 pass touched it. See §P5-c | + +**Cross-doc pointer (contract point 6).** The **build record** for the 2026-07-27 work is +not in this file — it lives in +[`competitive_hardening_2026-07.md`](competitive_hardening_2026-07.md)`:119-141` (the +`2026-07-27 — BO-7 progress, in two passes` log entry). That is a fourth doc describing +this board cell, alongside this spec, `agent_platform_hardening_2026-07.md` Part 1, and +`FOUNDATION_BUILDOUT_CHECKLIST.md` §BO‑7. **This spec is the owner**; the other three +should link here and add nothing. Recommended `work_plan.md` §4 row: +*"Isolation ladder (BO-7 / HH-6 / T0–T2) → owner **`permissions_sandbox_b6.md`**; +mirrors: hardening Part 1 (ladder definition only) · checklist §BO‑7 · competitive CH-1 +(build log)."* + +## The exposure, precisely (audited 2026-07-04 — **reconciled against code 2026-08-03**) Everything about a normal agent run executes **in the single gateway/orchestrator interpreter**, and that interpreter's `os.environ` holds **every decrypted integration secret**. Concretely, from the recon: -1. **Shared ambient credentials — the top standing exposure.** - `executor._inject_integrations_to_env` (`executor.py:4509`) writes every - resolved credential into `os.environ` (`ZOHO_REFRESH_TOKEN`, - `CLICKUP_API_TOKEN`, `SMTP_PASSWORD`, `APIFY_API_TOKEN`, `INSTANTLY_API_KEY`, - the Gmail/Sheets SA-json paths, …). It's called on all three run paths - (sub-agent `:1419`, streaming `:2769`, batch `:4655`) and the guard is only - `if val and not os.environ.get(env_var)` — so creds are written once and - **never cleared**. They **accumulate globally** across every run and every - agent for the process lifetime. **Any agent — or any prompt-injected agent — - can read any other integration's secret today** with `os.getenv(...)` or a - shell `env`, regardless of its own `config.json` scope. -2. **Arbitrary code in-process.** `loader._import_module_file` - (`loader.py:1240-1247`) `exec_module`s the agent repo's `agents.py` in the - gateway interpreter; imported modules persist process-wide (cleanup only pops - the run module + sys.path entries). -3. **Shared venv.** `_install_agent_deps` (`loader.py:1095`) and the runtime - `install_dependency` tool (`dep_tools.py:79`) both `uv pip install --python - sys.executable` — into the gateway's own interpreter. One agent's deps can - shadow/break another agent's or the gateway's. -4. **No resource/network limits anywhere** — even the *mutation* container - (our only existing isolation) runs with **zero** `--memory`/`--cpus`/ - `--pids-limit`/`--network`/`--cap-drop`/`--read-only` flags (grep-confirmed). +1. **Shared ambient credentials — ✅ CLOSED by P5-a (2026-07-04).** + `executor._inject_integrations_to_env` (**`executor.py:4340`** — the old + `:4509` anchor is stale) exports this run's resolved credentials + (`ZOHO_REFRESH_TOKEN`, `CLICKUP_API_TOKEN`, `SMTP_PASSWORD`, + `APIFY_API_TOKEN`, `INSTANTLY_API_KEY`, the Gmail/Sheets SA-json paths, …) + into `os.environ`. It is called on all three run paths — **`:599` sub-agent, + `:2335` streaming, `:4516` batch** (the old `:1419 / :2769 / :4655` anchors + are stale) — and it now **returns a restore token** that + `_restore_integration_env` (`executor.py:4392`) consumes at teardown + (`:843`, `:4053`). The pre-2026-07-04 behaviour, described below as the + exposure, was write-once-never-clear: creds **accumulated globally** for the + process lifetime and any agent could read any other integration's secret with + `os.getenv(...)`. **Residual, unchanged and honest:** `os.environ` is + process-global, so *concurrent* in-process runs still share the env for the + overlap window. Only a real per-run boundary (P5-c) closes that. +2. **Arbitrary code in-process — open.** `loader._import_module_file` + (**`loader.py:1287`**, the `exec_module` call at **`loader.py:1300`**; the old + `:1240-1247` anchor is stale) `exec_module`s the agent repo's `agents.py` in + the gateway interpreter; imported modules persist process-wide (cleanup only + pops the run module + sys.path entries). **This is the line `FOUNDATION_BUILDOUT_CHECKLIST.md` + §BO‑7 and CH‑1 actually name, and nothing has touched it.** +3. **Shared venv — open, but the install-time RCE is closed.** + `_install_agent_deps` (**`loader.py:1137`**; the old `:1095` anchor is stale) + and the runtime `install_dependency` tool (`dep_tools.py:79`) both + `uv pip install --python sys.executable` — into the gateway's own + interpreter. One agent's deps can still shadow/break another agent's or the + gateway's. **What the 2026-07-04 text did not know:** since 2026-07-27 + `_install_agent_deps` defaults to **`--only-binary=:all:`** + (`loader.py:1213-1215`, wheels only, escape hatch + `settings.agent_deps_allow_source_builds`), which closes the + arbitrary-code-at-install-time gap that ran *ahead* of any tool-call gate. + `dep_tools.py:79` does **not** carry that guard — noted, not scoped here. +4. **Resource/capability ceilings — ✅ CLOSED 2026-07-27. Egress + rootfs still open.** + > ⚠️ **The sentence that used to sit here was false and dangerous.** It read: + > *"even the mutation container … runs with **zero** `--memory`/`--cpus`/ + > `--pids-limit`/`--network`/`--cap-drop`/`--read-only` flags (grep-confirmed)."* + > That has been untrue since **2026-07-27**. An implementer trusting it would + > most likely have re-added `--cap-drop ALL` to `mutation.py`, duplicating a + > flag that is already there. **Struck.** + + Actual state, verified 2026-08-03. Both containers carry four of the six + flags, settings-overridable: + + | Flag | `mutation.py:700-722` | `copilot_sandbox.py:153-171` | + |---|---|---| + | `--cap-drop ALL` | ✅ | ✅ | + | `--cap-add DAC_OVERRIDE` | ✅ (root-in-container vs. host-owned bind mount) | ✅ (same reason) | + | `--security-opt no-new-privileges` | ✅ | ✅ | + | `--memory` | ✅ `settings.mutation_memory_limit`, default `2g` | ✅ `settings.copilot_sandbox_memory_limit`, default `768m` | + | `--cpus` | ✅ `mutation_cpu_limit`, default `2` | ✅ `copilot_sandbox_cpu_limit`, default `1` | + | `--pids-limit` | ✅ `mutation_pids_limit`, default `512` | ✅ `copilot_sandbox_pids_limit`, default `256` | + | `--network` | 🔲 **absent** — default bridge, unrestricted egress | 🔲 **absent** | + | `--read-only` | 🔲 **absent** — writable rootfs | 🔲 **absent** | + + The two remaining gaps are **WS-3b** (§P5-b.2). Note the constraint any + `--network` work must respect: `mutation.py:716` passes + `--add-host host.docker.internal:host-gateway` because the sandbox reaches the + gateway `/v1` over the host, and `copilot_sandbox.py:163` publishes + `-p 127.0.0.1::` because the host drives the CLI over loopback TCP. + `--network none` breaks both; the posture has to be an allowlist, not a cut. The one clean seam: the **model call already goes over loopback HTTP** to the gateway `/v1` (native MAF `OpenAIChatCompletionClient(base_url=…/v1)`; @@ -178,16 +285,29 @@ container — is the *destination*, but shipping it as step 1 is wrong here: every one of these back over RPC** — that's the bulk of the work and it's orthogonal to which isolation mechanism wraps it. - **The mutation container is batch-only.** It communicates by parsing stdout - sentinels after the process *exits* (`mutation.py:665`); normal runs need the - **live AG-UI SSE relay** (`stream_relay.py`). So even reusing the skeleton, we'd + sentinels after the process *exits* (`mutation.py:748-765`; the old `:665` + anchor is stale); normal runs need the **live AG-UI SSE relay** + (`orchestrator/stream_relay.py`). So even reusing the skeleton, we'd be building a new live host↔sandbox event channel. -So Phase 5 is **tiered** — ordered by (exposure removed) ÷ (infra cost), so each +So Phase 5 is **stepped** — ordered by (exposure removed) ÷ (infra cost), so each step is independently shippable and de-risks the next. -## Tiered plan - -### Tier 0 — Per-run credential scoping (kill the shared-env exposure) ← **Phase-5 step 1, implementing now** +## The Phase-5 plan — P5-a … P5-d + +> **Naming (R2).** These were "Tier 0/1/2/3" until 2026-08-03 and are now +> **P5-a/b/c/d**, because "Tier n" already means isolation strength in +> `agent_platform_hardening_2026-07.md` §1.2 (**T0/T1/T2**) and the two ladders +> do not correspond. Mapping, for anyone reading an old link: +> +> | Old name | New name | T-ladder relation | +> |---|---|---| +> | Tier 0 | **P5-a** | none — it is credential hygiene inside T0/T1, not a tier | +> | Tier 1 | **P5-b** | none — it hardens the containers we already run; a T2 *prerequisite*, not a tier | +> | Tier 2 | **P5-c** | **is** the hardening doc's **T2** | +> | Tier 3 | **P5-d** | none — it is a permission-*policy* change unlocked by T2 | + +### P5-a — Per-run credential scoping (kill the shared-env exposure) — ✅ **SHIPPED 2026-07-04** · AGENT-SAFE The single highest-value slice, and it needs **no container at all** — it directly closes exposure #1 above, which is the concrete "any agent reads any secret" hole. @@ -204,8 +324,8 @@ a **scoped, per-run** materialization that is torn down when the run ends: - **Concurrency caveat, stated honestly:** `os.environ` is process-global, so under *concurrent* in-process runs this scoping is best-effort — two runs overlapping still share the env for the overlap window. This is a real limit of - the in-process model and is exactly what Tier 2+ (a real process/container - boundary, each with its **own** env) fixes permanently. Tier 0's win is + the in-process model and is exactly what P5-c (a real process/container + boundary, each with its **own** env) fixes permanently. P5-a's win is removing the **permanent accumulation** (the steady-state where every secret ever used is always present) and scoping to the run's own declared integrations — a large, real reduction, not a complete fix. The residual @@ -221,22 +341,203 @@ a **scoped, per-run** materialization that is torn down when the run ends: This is a contained executor change with unit-test coverage and no infra dependency — ships first. -### Tier 1 — Egress-scoped model key + resource ceilings on the mutation container -Before generalizing the container, **harden the one we already have** (it's the -template Tier 2 reuses, and it currently has zero limits): -- Add `--memory`, `--cpus`, `--pids-limit`, `--cap-drop=ALL` - (+ re-add only what's needed), and a `--read-only` rootfs with a writable - workspace mount, to the `docker run` in `mutation.py:626`. Sane defaults tuned - for the 4GB box, env-overridable. -- Give the sandbox a **scoped gateway key** (not the `sk-local` master key) with - a short TTL / run-scoped identity, so a leaked sandbox key can't act as the - gateway. (Ties to B5 on-behalf-of vs fixed-credential.) -- Constrain egress: the sandbox needs the gateway `/v1` + (for the self-heal - agent) GitHub; everything else can go through a default-deny with an allowlist. -These flags are pure additions to the existing invocation and carry into Tier 2. - -### Tier 2 — Generalize the container to a live, streaming run sandbox (the big lift) -Lift a **normal** Copilot/MAF run into the (now hardened) container: +**Shipped as described** — `executor.py:4340-4389` (token) + `:4392` (restore), +teardown at `:843` and `:4053`, pinned by `tests/unit/test_integration_env_scoping.py` +and `evals/trajectories/test_integration_env_scoping_trajectory.py`. The last +bullet (shrink the env surface to subprocess-callers only) was **not** done and is +not part of WS-3a; it stays as an unowned residual. + +### P5-a.2 — **WS-3a · Record the derived tier, and refuse a run we cannot isolate** — 🔲 **AGENT-SAFE, dispatchable** + +The tier is already derived and immediately discarded (see §"What actually +shipped"). This slice makes it a **record** and a **gate**. It builds no +container and changes no isolation mechanism — it makes the ladder observable +and makes the one posture we cannot honour refuse itself instead of proceeding +silently. Nothing here needs P5-c to exist. + +**Scope.** `manifest.py` (no change expected), `declarative.py`, `executor.py`, +`gateway/run_trace.py`, one new migration, one new test file. +**Non-goals.** No container. No change to which tools any agent receives — the +tier is *derived from* the resolved surface, never the other way round. No +change to `AGENT_PERMISSION_MODE` behaviour. + +**Done when — all five, each independently testable:** + +1. **Derived on all three run paths, not just `declarative.py`.** + `isolation_tier()` is resolved once per run and available to the run in each + of the three executor entrypoints that already call + `_inject_integrations_to_env` — `_run_sub_agent_streaming` (`executor.py:599`), + `run_agent_stream` (`executor.py:2335`), `_run_with_maf_agent` + (`executor.py:4516`). `declarative.py:210`'s existing log field stays and + keeps reading the same function. *Test:* each of the three paths emits the + tier for a manifest fixture whose expected tier is known (reuse the four + fixtures already pinned in `tests/unit/test_agent_manifest.py:224-252`). +2. **Persisted on the `agent_run` trace.** A new nullable `text` column on + `agent_run` (name it `isolation_tier`), added by **the next free migration + number — determine it by listing `infra/postgres/` at build time; do not + copy a number out of this document**. `gateway/run_trace.py::_persist_row` + (~`:169`) writes it in both the INSERT and the `ON CONFLICT DO UPDATE` + branch, and `build_run_trace_row` (`:67`) carries it. + *Test:* `build_run_trace_row(...)` returns the tier in its row dict. + **Known limit, state it in the PR rather than chasing it:** `agent_run` rows + are written only from `chat_fold.py:467-478` (the streamed chat path), so + batch `/agent/run` and sub-agent runs get the log line and the refusal but no + row. Widening trace coverage is a separate, unowned item. + **Recommended seam:** the streaming path already emits + `{"type": "RUN_STARTED", "runId", "threadId"}` at `executor.py:2311`, and + `run_trace` already derives fields from that same replayed event list + (`_derive_status`, `:23`). Carrying the tier as one extra field on + `RUN_STARTED` needs no new plumbing and is unit-testable against a synthetic + event list. An explicit `record_run_trace(..., isolation_tier=…)` kwarg + (`run_trace.py:220`) is an acceptable alternative; pick one and say which. +3. **A T2 run that is not covered by the sandbox is refused — before any tool + injection.** When the resolved tier is `T2` and + `settings.copilot_sandbox_scope` (`acb_common/settings.py:222`, default `""`) + does not cover that run, the run raises a **named** error — + `IsolationTierUnavailable` — carrying the agent slug, the derived tier, and + the configured scope. It is raised **before** `_inject_agent_tools` + (`_tool_injection.py:639`) runs, so an un-isolatable run never receives the + shell tools that made it T2. + *Test:* a T2 manifest with an empty `copilot_sandbox_scope` raises + `IsolationTierUnavailable`; the same manifest with a covering scope does not; + a T0 and a T1 manifest never raise regardless of scope. +4. **It ships OFF, and the switch is named.** Because *today* every unscoped + agent derives T2 (`manifest.py:281-282` — an open scope means the shell tools + are injected), enforcing the refusal on day one would refuse most real runs. + So the refusal is behind an env switch (`ISOLATION_TIER_ENFORCE`, default + off) and defaults to **log-and-proceed** with a `WARNING` naming the tier and + the missing coverage — the same audit-then-enforce shape + `AGENT_PERMISSION_MODE` already uses, and for the same reason. + **Flipping it on is OWNER-GATE**; register it in `work_plan.md` §6 in the + same change. *Test:* with the switch off, a T2/no-scope run proceeds and logs; + with it on, the same run raises. +5. **Pinned in a named new test file:** `tests/unit/test_isolation_tier_record.py`, + covering all of 1–4. Existing pins must stay green — + `tests/unit/test_agent_manifest.py` in particular, because it asserts + `resolve_tool_surface` ≡ `_resolve_injected_scope`, and that equivalence is + what makes the derived tier trustworthy. + +**Verification commands (WS-3a):** +``` +uv run ruff check . +uv run python -m pytest tests/unit/test_isolation_tier_record.py \ + tests/unit/test_agent_manifest.py tests/unit/test_declarative_builder.py -q +``` + +### P5-b — Ceilings, egress and a scoped key on the containers we already run + +Before generalizing the container, **harden the ones we already have** (they are +the template P5-c reuses). There are now **two**: `mutation.py`'s batch mutation +sandbox and `copilot_sandbox.py`'s Copilot-CLI sandbox. + +#### P5-b.1 — Resource + capability ceilings — ✅ **SHIPPED 2026-07-27** · AGENT-SAFE +~~Add `--memory`, `--cpus`, `--pids-limit`, `--cap-drop=ALL` (+ re-add only what's +needed) … to the `docker run` in `mutation.py:626`.~~ **Done.** Both containers +carry `--cap-drop ALL` + `--cap-add DAC_OVERRIDE` + +`--security-opt no-new-privileges` + `--memory` / `--cpus` / `--pids-limit`, all +settings-overridable — `mutation.py:700-722`, `copilot_sandbox.py:153-171`. Do +**not** re-add these; see the struck sentence in exposure #4. Pinned by +`tests/unit/test_mutation_sandbox_hardening.py` and +`tests/unit/test_copilot_sandbox.py`. + +#### P5-b.2 — **WS-3b · Read-only rootfs + a stated network posture** — 🔲 **AGENT-SAFE, dispatchable** + +The two flags the 2026-07-27 pass did not add. Pure additions to two existing +`docker run` invocations; no new infrastructure, no new call site. + +**Scope.** `apps/services/orchestrator/orchestrator/mutation.py`, +`apps/services/orchestrator/orchestrator/copilot_sandbox.py`, +`packages/acb_common/acb_common/settings.py`, the two existing test files. +**Non-goals.** No scoped gateway key (that is P5-b.3, OWNER-GATE). No egress +*proxy* — a proxy is P5-c infrastructure. No change to what either container runs. + +**Done when — all four:** + +1. **Both containers pass `--read-only` with a named writable mount.** The + rootfs is read-only and the workspace is the declared exception: + `mutation.py` keeps `-v {agent_dir}:/workspace/repo` writable and adds a + `--tmpfs /tmp` (the `copilot` CLI, `git`, and `uv` all write there); + `copilot_sandbox.py` keeps its two existing `-v` mounts + (`{workspace}:{CONTAINER_WORKSPACE}` and the state dir) writable and adds the + same `--tmpfs /tmp`. Both are settings-overridable + (`mutation_readonly_rootfs` / `copilot_sandbox_readonly_rootfs`, **default + `True`**) so a single env var reverts the posture without a deploy. +2. **Both containers pass an explicit `--network` posture with a stated + default.** Default `bridge` — i.e. **today's behaviour, made explicit and + overridable** (`mutation_network` / `copilot_sandbox_network`, default + `"bridge"`). Deny-by-default is **not** in this slice, and the reason is + recorded rather than assumed: `mutation.py:716` needs + `host.docker.internal:host-gateway` to reach the gateway `/v1`, and + `copilot_sandbox.py:163` publishes `-p 127.0.0.1::` for host→container + RPC, so `--network none` breaks both. The slice's value is that the posture + becomes a **named, overridable, tested setting** an operator can narrow to a + custom docker network — not that it is narrowed here. +3. **Assertions land in the existing test files.** + `tests/unit/test_mutation_sandbox_hardening.py` and + `tests/unit/test_copilot_sandbox.py` each gain: the flag is present with the + default; the setting overrides it; the writable mount / tmpfs is present + alongside `--read-only`; and the existing cap/limit assertions still pass + unchanged. +4. **No behaviour change at defaults.** A run with untouched settings produces + the same container behaviour as today apart from the read-only rootfs — which + means the honest risk of this slice is *"something in the image writes outside + the mounts and now fails"*. Both images must be exercised once before merge + and the result stated in the PR. + +**Verification commands (WS-3b):** +``` +uv run ruff check . +uv run python -m pytest tests/unit/test_mutation_sandbox_hardening.py \ + tests/unit/test_copilot_sandbox.py tests/unit/test_code_session_sandbox.py \ + tests/unit/test_app_builder_sandbox.py -q +``` + +#### P5-b.3 — Scoped gateway key for the sandbox — 🔲 **OWNER-GATE · unbuilt and undesigned** +Give the sandbox a **scoped gateway key** (not the master key) with a short TTL / +run-scoped identity, so a leaked sandbox key can't act as the gateway. (Ties to +B5 on-behalf-of vs fixed-credential.) Today `mutation.py:700-722` passes +`GATEWAY_API_KEY` straight through. + +**This has no acceptance and should not be given any by an agent.** Three +questions are open and every one of them is an owner decision, not an +implementation detail: what the TTL is, who issues the key (the gateway at spawn +time? a pre-provisioned service identity?), and how it is revoked mid-run. It +also touches credential issuance, which is in `work_plan.md` §6's gate list. +An agent asked to "finish P5-b" builds **P5-b.2 only** and refuses this by name. + +### P5-c — Generalize the container to a live, streaming run sandbox — 🔲 **PARKED SUB-PROJECT** (owner decision 2026-08-03) · **OWNER-GATE to un-park** + +> **Why this is parked, not cancelled.** Command Center is an **internal Fracktal +> tool**. The team uses it; there are no external tenants and no customer-authored +> agents. So the isolation ladder has to hold up to **trusted colleagues, not +> hostile users** — and against that threat model the failure modes that matter +> are mistakes and blast radius (a runaway loop on a 4GB box, an agent reading a +> credential outside its declared scope, a write in the wrong tree), all of which +> are addressed by P5-a's credential scoping, P5-b's ceilings, and WS-3a/WS-3b. +> None of them needs a container around a normal run. +> +> The design below is still the right destination and is kept intact. What it +> loses is its **schedule** and its old justification: it was previously gated on +> *"before the Agent Workshop opens to non-engineers"*, which assumed the Workshop +> would hand agent authorship to people outside the engineering team. It will not — +> the Workshop's users are colleagues who could already open a PR against this +> monorepo. See `agent_platform_hardening_2026-07.md` §1.5. +> +> **P5-c has no acceptance criteria, and none should be written for it** until +> either (a) a **second organisation** runs on this platform, or (b) agent +> authorship opens to someone **outside Fracktal**. At that point it is re-costed +> from scratch — the 2026-07-04 estimates below are a year stale by then. +> Un-parking it is an **owner decision**. An agent asked to "finish the isolation +> ladder" builds WS-3a and WS-3b and refuses P5-c by name. +> +> **Do not confuse P5-c with what shipped.** `copilot_sandbox.py` containerizes +> the **`copilot` CLI binary** and is wired at two call sites behind a scope +> setting that ships empty. The host still owns orchestration, tool execution and +> permission handling. That is T2-*shaped* reuse of the mutation container's +> hardening — it is **not** P5-c, and it does not isolate a normal agent run. + +Design of record (unchanged, 2026-07-04) — lift a **normal** Copilot/MAF run into +the (now hardened) container: - New `sandbox_runner.py` (generalize `mutation_runner.py`) that runs the agent turn and **streams AG-UI events live** to the host over a real channel (Redis Stream keyed by thread_id — reuse `stream_relay.py`'s contract directly, @@ -248,25 +549,96 @@ Lift a **normal** Copilot/MAF run into the (now hardened) container: `--python sys.executable` shared-venv risk). - **Warm-pool** execution model for the 4GB box (a small pool of pre-started sandbox containers claimed per run), not cold-container-per-run. -This is genuinely multi-step infra and is scoped as its own sub-project; Tier 0 -+ Tier 1 remove the concrete standing exposures and de-risk it. +This is genuinely multi-step infra and is scoped as its own sub-project; P5-a ++ P5-b remove the concrete standing exposures and de-risk it. -### Tier 3 — Default-deny tightening + intent-level auth -Once Tier 2 gives real isolation, flip the near-term handler's *unknown → +### P5-d — Default-deny tightening + intent-level auth — 🔲 **blocked on P5-c (parked)** · **OWNER-GATE** +Once P5-c gives real isolation, flip the near-term handler's *unknown → approve-open-but-logged* to *default-deny* (the honest reason it's fail-open today, per the near-term section, is that a hard deny on an in-process model that already runs arbitrary code gives false assurance — a real boundary removes that objection). Layer intent-level authorization over allow-everything. -## Grade movement -Tier 0 alone: B6 stays **B−** but closes the single worst concrete hole (shared -ambient secrets). Tier 0+1: **B** (limits + scoped key + no permanent cred -accumulation). Tier 2: **B+/A−** (real isolation boundary for normal runs). The -map's "container isolation for normal runs" open item is fully closed only at -Tier 2; Tiers 0–1 are the shippable de-risking that gets us there safely. +> **Re-framed 2026-08-03 under the internal-tool threat model.** P5-d inherits +> P5-c's parking: it is explicitly conditioned on *"once P5-c gives real +> isolation"*, and P5-c is parked. **Do not build P5-d in the meantime**, and do +> not "partially" default-deny an in-process run to make progress — that is +> precisely the false assurance the paragraph above warns about, and against +> colleagues rather than attackers it buys nothing while breaking real work. +> +> The one piece of P5-d that is separable is the **near-term handler's mode**, +> which already exists: prod runs `AGENT_PERMISSION_MODE` in `audit` and moving +> it to enforcement is **OWNER-GATE** (`work_plan.md` §6). That flip does not +> need P5-c and does not need this section — it needs someone to read the +> decision stream. Everything else here (intent-level authorization over a +> default-deny surface) stays parked with P5-c and gets **no acceptance**. + +## Grade movement — *re-scored 2026-08-03* +P5-a alone: B6 stays **B−** but closes the single worst concrete hole (shared +ambient secrets). **P5-a + P5-b.1 (both shipped) + WS-3a + WS-3b: B** — ceilings, +a recorded and enforced tier, a stated egress/rootfs posture, and no permanent +credential accumulation. The `sk-local`-class scoped key (P5-b.3) is the one +piece of the original "B" bundle still missing, and it is owner-gated. + +**The A− line is now conditional, not scheduled.** The old text put **B+/A−** at +P5-c ("real isolation boundary for normal runs") and treated it as the +destination. Under the internal-tool threat model that grade is only *worth +buying* when the trust boundary moves — a second org, or authorship outside +Fracktal. Against colleagues, **B is the right resting grade**, and the module +map's "container isolation for normal runs" item should be read as *"open, and +deliberately parked"* rather than *"open, in progress"*. ## Status (Phase 5) - 2026-07-04 — Design from the B6 Phase-5 recon (mutation-container primitive + in-process/credential boundary analysis). Tiered plan authored. Implementing **Tier 0** (per-run credential scoping) first — the highest exposure-removed ÷ infra-cost slice, no container dependency. +- 2026-07-04 — **Tier 0 (now P5-a) shipped.** `_inject_integrations_to_env` + returns a restore token consumed by `_restore_integration_env` at run + teardown; wired on all three run paths. Never logged at the time — recovered + from code on 2026-08-03. +- 2026-07-27 — **Tier 1's ceilings (now P5-b.1) shipped**, plus two things this + spec never mentioned. Recorded here on 2026-08-03 from + `competitive_hardening_2026-07.md:119-141` and re-verified against code: + (a) `--cap-drop ALL` / `--cap-add DAC_OVERRIDE` / + `--security-opt no-new-privileges` / `--memory` / `--cpus` / `--pids-limit` on + the mutation container; (b) the **dep-install RCE fix** — + `_install_agent_deps` defaults to `--only-binary=:all:` (`loader.py:1213-1215`); + (c) **Copilot-CLI containerization** (`copilot_sandbox.py` + + `Dockerfile.copilot-sandbox`) wired at `code_session.py:109` and + `executor.py:1070`, gated on `settings.copilot_sandbox_scope` and shipping OFF. + Egress, read-only rootfs and the scoped key were **not** part of that pass. +- 2026-08-03 — **Truth pass (WS-3), verified against code at `2ccff9e0`.** No + code changed; this is a documentation reconciliation. What changed here: + 1. **Struck a false, dangerous claim.** Exposure #4 asserted the mutation + container ran with *"zero `--memory`/`--cpus`/`--pids-limit`/`--network`/ + `--cap-drop`/`--read-only` flags (grep-confirmed)"*. Untrue since + 2026-07-27; four of the six have been present for five weeks. An + implementer dispatched on this row would most likely have re-added + `--cap-drop ALL` to `mutation.py`. Replaced with a per-flag table naming + both containers and the two flags that really are missing. + 2. **Renamed Tier 0/1/2/3 → P5-a/b/c/d** (R2) and adopted + `agent_platform_hardening_2026-07.md` §1.2's **T0/T1/T2** as the single + isolation ladder. The two were incompatible numberings inside one board + cell (WS-3). + 3. **Fixed six stale anchors**: `executor.py:4509`→`:4340`; the three run-path + anchors `:1419/:2769/:4655`→`:599/:2335/:4516`; `mutation.py:626`→ + `:700-722`; `loader.py:1240-1247`→`:1287`/`:1300`; `loader.py:1095`→`:1137` + (plus the previously-unrecorded `--only-binary=:all:` guard at `:1213-1215`); + `mutation.py:665`→`:748-765`; and the five near-term permission sites + `~1190/~2572/~3023/~3796/~4297`→`632/2483/3011/3909/4442`. + 4. **Recorded what shipped and was never written down** — see the + §"What actually shipped" table. + 5. **Wrote acceptance for exactly two slices**, WS-3a (§P5-a.2) and WS-3b + (§P5-b.2). Both AGENT-SAFE, both dispatchable, neither needing a container. + 6. **Parked P5-c and P5-d** under the owner's 2026-08-03 internal-tool threat + model, with the un-parking condition stated. Neither gets acceptance. + 7. **Struck WS-3's claim on `tool_scope` deny** — that is built and + owner-gated under **WS-23** (`_tool_injection.py:101-117` + `:214-224`, + spec `skills_scope_out.md` §4), not this row. + **Still owed by the owner** (recorded, not actioned, because they are outside + this doc): the `work_plan.md` §2 WS-3 title correction, a §4 single-owner row + for the isolation ladder, `copilot_sandbox_scope` registration in §6, and the + `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO‑7 correction (its stale + `loader.py:1247` / `:1095` anchors, and its CH‑1 note recommending a flag set + that has already been adopted). diff --git a/ai-company-brain/specs/workflows_app.md b/ai-company-brain/specs/workflows_app.md index 21ff7b29e..9a8f35471 100644 --- a/ai-company-brain/specs/workflows_app.md +++ b/ai-company-brain/specs/workflows_app.md @@ -1,7 +1,8 @@ # Workflows App — Project Plan (deterministic automation over the agent fleet) -> **Product:** CommandCenter · **Feature:** Workflows app (`/workflows`) · **Updated:** 2026-07-30 · **Version:** 0.2 +> **Product:** CommandCenter · **Feature:** Workflows app (`/workflows`) · **Updated:** 2026-08-03 · **Version:** 0.3 · **verified against code on 2026-08-03** > **Status:** 🔄 Slices 1+2 built — data model (migration 132) + gateway API + MAF compiler/engine + `/workflows` visual editor + Module Studio + **Workflow Copilot (F14)** + **keyword capability search (F15; semantic → BO‑22)** + **event triggers (F10)** + **approval node with pause/resume via the Action Broker inbox (F11)** + **workflows as agent tools (F13)** + **run-history drill-in (F9 complete: a history row replays its recorded node results onto the canvas)** + **F1/F6 complete (gallery search + duplicate/delete, version rollback via the status-badge popover)** + **F3's logic vocabulary complete (wait node — inline under a minute, durable pause above it; approval and wait now both in the catalog/palette)**. All five trigger kinds live: manual, api, webhook, schedule, event. Engine semantics are locked by a golden trajectory eval (`evals/trajectories/test_workflow_engine_trajectory.py`, CI-blocking); orphaned `running` rows are swept to `failed` at gateway startup (paused runs survive — resume rebuilds from the pause snapshot). **R2 is mitigated (migration 134):** a published workflow whose unattended runs fail `AUTO_DISABLE_AFTER` times consecutively disables itself with a recorded reason, and `POST /{id}/enable` is the one-click way back. +> **Slice 3 re-scoped 2026-08-03 (truth pass, §8.3).** The one-line Slice 3 asked for three things and **one of them is already shipped**: describe→generate→refine full-graph authoring landed as F14 (`39b1e17a`) and is **struck**. "Parallel fan-out" is also shipped (`engine/graph.py:17`; MAF's superstep scheduler routes it) — the unbuilt half is **fan-in/join**, restated as such. What genuinely remains is **fan-in/join (8.3b), loops (8.3c), and a template gallery (8.3a — nothing exists)**. Two owner decisions recorded the same day: Command Center is an **internal Fracktal tool** (§1.4) and **loops are approved** despite §11 R1 (§8.3c). > **Parent RFC:** [`docs/workflow-editor/README.md`](../../docs/workflow-editor/README.md) — stack selection (React Flow), the compile-to-MAF-Workflows decision, data model, editor UX, trigger taxonomy. Read it for *how*; this doc is *what, why, and why now*. Interactive mockup: `docs/workflow-editor/mockup.html`. > **Reference precedents:** [`task_manager_app.md`](task_manager_app.md) (app spec shape) · [`docs/app-workshop/README.md`](../../docs/app-workshop/README.md) §4.0 (the platform contract this app also enforces). > **Policy amendment:** ADR-028 (see `system_architecture.md`) amends ADR-014 and `project_plan.md` C-09 / §2 non-goals — see §10. @@ -54,6 +55,8 @@ The missing quadrant is **deterministic + self-serve**: the ops owner defines th - **Not multi-tenant marketplace tooling.** Workflows are org-internal; sharing/templates beyond this org are Phase 4+. *(Clarification 2026-08-01: the org-internal template gallery is Slice 3 — this non-goal refers to cross-org marketplace sharing, not in-org templates.)* - **No autonomous outward writes.** Same rule as everywhere else: write-class integration actions require the approval node / Action Broker disposition until BO‑1 lands fully. +**OWNER DECISION 2026-08-03 — Command Center is an internal Fracktal tool.** The team uses it; there are no external tenants and none are planned in this app's horizon. Scope is weighed accordingly: features whose only justification is *someone else's org* (template marketplaces, per-tenant template stores, sharing permissions on content) are out, and "one org, engineers in the room, ships with the code" is a legitimate answer to a storage or distribution question — see the §8.3a decision, which is decided on exactly that basis. This does **not** relax the platform contract (§3.2), the approval gates (G4), or capability checks (Q3): internal does not mean unguarded, it means un-multi-tenanted. + --- ## 2. Feature set (prioritized) @@ -90,7 +93,7 @@ The RFC §3 mapping table is the source of truth; summary of the seams this buil - **Agent invocation** — `call_agent` / orchestrator executor (`apps/services/orchestrator/`). The agent node is a thin adapter; agents remain code-authored. - **Integration actions** — the integrations registry in `packages/acb_skills` (13 registered services today: zoho-crm, clickup, gmail, gmail-send, smtp, google-sheets, apollo, serpapi, apify, instantly, anymailfinder, google-maps, litellm) resolves credentials at run time; nodes never see secrets (D4). Write-class actions dispatch through the Action Broker's handler registry so disposition/approval semantics are the broker's, not the engine's. - **Trigger plumbing** — gateway webhook receivers, ingestion normalizers, Redis activity feed; the workflows scheduler is the platform's first real cron loop (D6). -- **HITL** — approvals inbox + `workflow_run_pause` snapshots; Action Broker disposition once BO‑1 wires the write path. +- **HITL** — approvals inbox + `workflow_run_pauses` snapshots; Action Broker disposition once BO‑1 wires the write path. - **Streaming** — run events over the existing SSE relay pattern; runs appear in `/observability`. - **New**: the `workflow*` tables, the graph→MAF compiler, the node handler set, the module library + generator, and the `/workflows` UI. @@ -113,7 +116,7 @@ Enforcement ladder (each rung independent): (1) the editor and Module Studio **r ### 3.3a Trigger durability (what survives a restart, and what does not) -There is **no OS cron and no scheduler process**. A schedule is a `workflow_trigger` row (`config.cron`, `config.timezone`, `last_fired_at`); the *scanner* is one supervised asyncio loop in the gateway (30s). APScheduler's `CronTrigger` is used purely as an expression **parser** — importing a scheduler daemon would be the "second runtime" this design exists to avoid. Durability therefore comes from the row, not the loop: **the schedule is a database fact, the scanner is a stateless reader of it.** +There is **no OS cron and no scheduler process**. A schedule is a `workflow_triggers` row (`config.cron`, `config.timezone`, `last_fired_at`); the *scanner* is one supervised asyncio loop in the gateway (30s). APScheduler's `CronTrigger` is used purely as an expression **parser** — importing a scheduler daemon would be the "second runtime" this design exists to avoid. Durability therefore comes from the row, not the loop: **the schedule is a database fact, the scanner is a stateless reader of it.** | Situation | Behaviour | |---|---| @@ -139,7 +142,7 @@ An external caller must be given the **gateway's own** origin (`public_api_base_ ### 3.3 Trigger model -RFC §7 verbatim, plus one product rule: **all trigger kinds converge on one entrypoint** (`start_run`) that seeds `variables.trigger` with a typed payload and creates a `workflow_run`. Kinds: `manual`, `api`, `webhook` (per-workflow secret token URL), `schedule` (cron expression, croniter-driven asyncio loop), `event` (bindings against normalized ingestion events — the successor to `agent_registry.json.webhook_routes`). Durable queueing/backoff for high-volume event triggers is BO‑20's scope; v1 executes runs as supervised asyncio tasks in-process and says so honestly in run status. +RFC §7 verbatim, plus one product rule: **all trigger kinds converge on one entrypoint** (`start_run`) that seeds `variables.trigger` with a typed payload and creates a `workflow_runs` row. Kinds: `manual`, `api`, `webhook` (per-workflow secret token URL), `schedule` (cron expression parsed by APScheduler's `CronTrigger` inside the gateway's supervised asyncio scan loop — **not croniter**, which is not a dependency of this repo; see §3.3a and D6), `event` (bindings against normalized ingestion events — the successor to `agent_registry.json.webhook_routes`). Durable queueing/backoff for high-volume event triggers is BO‑20's scope; v1 executes runs as supervised asyncio tasks in-process and says so honestly in run status. ### 3.4 Code modules — scope and the sandbox line @@ -149,10 +152,10 @@ The v1 module runtime is **restricted-execution, not a sandbox**: AST-allowliste ## 4. Data model -Migration `infra/postgres/132_workflows.sql` (next free number after `131_integration_memory_permissions.sql`). Tables per RFC §4 — `workflow`, `workflow_version`, `workflow_trigger`, `workflow_run` (+ `node_results`), `workflow_run_pause` — plus one addition: +Migration `infra/postgres/132_workflows.sql` (next free number after `131_integration_memory_permissions.sql`). **Table names are plural** — corrected 2026-08-03 against `132_workflows.sql:27/45/56/70/94/107`, which is the authority; every singular form previously written here (`workflow`, `workflow_version`, `workflow_trigger`, `workflow_run`, `workflow_run_pause`, `workflow_module`) was wrong and would send a reader to a table that does not exist. The real set is `workflows`, `workflow_versions`, `workflow_triggers`, `workflow_runs` (whose per-node history is the `node_results` JSONB **column**, `:81` — not a table), `workflow_run_pauses` — plus one addition: ``` -workflow_module -- the org module library (Module Studio) +workflow_modules -- the org module library (Module Studio) id (uuid, pk) name (unique per org), description language text -- 'python' (only value in v1) @@ -164,9 +167,9 @@ workflow_module -- the org module library (Module Studio) created_by / created_at / updated_at ``` -`workflow.graph` is React-Flow-native JSON persisted verbatim (edit-model); `workflow_version.serialized` is the compiled flat DAG (run-model). Node schema and edge `sourceHandle` branching per RFC §4. +`workflows.graph` is React-Flow-native JSON persisted verbatim (edit-model); `workflow_versions.serialized` is the compiled flat DAG (run-model). Node schema and edge `sourceHandle` branching per RFC §4. -Migration `134_workflows_automation_health.sql` adds three columns to `workflow` for the R2 mitigation: `disabled_reason` / `disabled_at` (why it is off — written identically by a human hitting Disable and by the auto-disable policy) and `health_since` (the instant the failure streak is counted from; publish, rollback, and enable all stamp it). The streak itself is deliberately **not** a counter column — it is derived from `workflow_run` on demand, so it can never drift from the history a human reads, and one success breaks it with no bookkeeping. +Migration `134_workflows_automation_health.sql` adds three columns to `workflows` for the R2 mitigation: `disabled_reason` / `disabled_at` (why it is off — written identically by a human hitting Disable and by the auto-disable policy) and `health_since` (the instant the failure streak is counted from; publish, rollback, and enable all stamp it). The streak itself is deliberately **not** a counter column — it is derived from `workflow_runs` on demand, so it can never drift from the history a human reads, and one success breaks it with no bookkeeping. --- @@ -211,10 +214,91 @@ Deferred to `FOUNDATION_BUILDOUT_CHECKLIST.md` per planning rules — not re-des Aligned to RFC §9, resequenced so each slice ships value: -- **Slice 1 (this build):** migration 132 · gateway `routes/workflows/` + `workflows/` engine package (compiler → MAF `WorkflowBuilder`, handlers: trigger/agent/tool/condition/transform-module/set-variable/http/output) · module validator + restricted runner + conversational generator · manual + webhook + schedule triggers (croniter loop) · `/workflows` UI: gallery, editor (palette/canvas/inspector/console), Module Studio, run history · catalog endpoint · feature slug + nav. +- **Slice 1 (this build):** migration 132 · gateway `routes/workflows/` + `workflows/` engine package (compiler → MAF `WorkflowBuilder`, handlers: trigger/agent/tool/condition/transform-module/set-variable/http/output) · module validator + restricted runner + conversational generator · manual + webhook + schedule triggers (APScheduler `CronTrigger` as an expression parser inside one supervised asyncio scan loop — **not croniter**; `scheduler.py:57-61`, dependency at `apps/services/gateway/pyproject.toml:37`) · `/workflows` UI: gallery, editor (palette/canvas/inspector/console), Module Studio, run history · catalog endpoint · feature slug + nav. - **Slice 2 (shipped):** event triggers via `/agent/webhook/{source}` + the ClickUp receiver's event-hook sink; approval node via `workflow_run_pauses` snapshots + the Action Broker approvals inbox, with cached-replay resume. Also landed: the publish-gate write-class check (`write_without_approval`) and F13 workflow-as-tool. Remaining: streaming from the raw MAF event stream (engine-level per-node events stream today). -- **Slice 3:** describe→generate→refine full-graph authoring; loops/parallel fan-out in the compiler; template gallery. (Workflow-as-tool for the orchestrator shipped early — F13, Slice 2.) -- **Slice 4 (post-BO‑20/BO‑7):** durable queued runs; sandboxed module execution; MCP exposure; retention policies. +- **Slice 3:** three items — **8.3a templates**, **8.3b fan-in/join**, **8.3c loops**. Fully specified with per-item acceptance, gate labels and verification in **§8.3**; the old one-line version was 16 words and asked for one thing that already shipped. (Workflow-as-tool for the orchestrator shipped early — F13, Slice 2.) +- **Slice 4:** blocked. Named dependencies and the reason in **§8.4** — it is *not* "post-BO‑20" in the vague sense. + +### 8.3 Slice 3 — specified (truth pass, verified against code 2026-08-03) + +**What was struck.** *"Describe→generate→refine full-graph authoring"* is **DONE — delivered by F14 in commit `39b1e17a`** ("feat(workflows): Workflow Copilot + semantic capability search"). `POST /workflows/{id}/copilot` (`copilot.py:1-12`) emits the **FULL updated graph** — the system prompt says so literally at `copilot.py:51` (`"graph": {...} // FULL updated graph, or null if no change`) — with a named-issue repair round against the same validators publish uses, and auto-creates the modules the graph needs. §2 already records this twice (F12 *"Superseded by F14"*, F14 *"Must (shipped)"*). Dispatching it would have sent an implementer to rebuild a live endpoint. **Do not re-open it.** + +**What "loops/parallel fan-out" actually meant.** **Fan-out already ships** — `engine/graph.py:17`: *"Fan-out from a node is allowed (parallel branches)"*, and `runner.py:7` records that MAF's superstep scheduler does the routing, fan-out and completion detection. The unbuilt halves are the two things the validator still rejects: **fan-in** (`graph.py:303-311`, `"a node may have only one incoming edge (v1)"`) and **cycles** (`graph.py:326-329`, comment *"loops arrive in a later slice"*). They are split into 8.3b and 8.3c because they are different problems with different risks. + +**The acceptance standard for all three items — the F14 lesson, stated once.** F14's acceptance is **mechanical, not vibes**: the emitted graph must pass `validate_graph`, and generated module code must pass `validate_module_code` (`copilot.py:30-31` imports exactly those two). The LLM's output is judged by a deterministic validator, never by an eyeball. Every done-when below is written to the same standard — an assertion a test can make, on a validator or a status code, not a screenshot. **Corollary, and the specific failure mode to design against:** 8.3b and 8.3c both *relax a rejection that is currently pinned by a passing test*. A ticket that adds a join executor or a loop bound but leaves `test_fan_in_rejected_v1` / `test_cycle_rejected` asserting rejection closes **green while delivering nothing**. Each done-when therefore names the test that must **invert**. + +**Verification (all three items, and never `tests/unit/` as a directory — the full directory hangs on Windows):** + +``` +uv run pytest tests/unit/test_workflows_engine.py tests/unit/test_workflows_slice2.py \ + tests/unit/test_workflows_trigger_reliability.py \ + evals/trajectories/test_workflow_engine_trajectory.py -q +``` + +Baseline on this branch, run 2026-08-03: **`4 failed, 69 passed, 2 warnings in 17.70s`** on Windows. ⚠️ **All four failures are the same known Windows-only defect and are green in CI** — `engine/modules.py:296` passes `preexec_fn=_limit_resources` to the module subprocess, and CPython raises `ValueError: preexec_fn is not supported on Windows platforms`, so every test whose graph contains a **module node** fails locally: `test_workflows_engine.py::test_module_node_runs_generated_code` plus, in the golden eval, `test_high_priority_run_pauses_at_the_gate`, `test_resume_replays_without_repeating_side_effects` and `test_tool_failure_surfaces_the_node_and_skips_downstream`. CI runs `tests/unit/` on `ubuntu-latest` (`pr-check.yml:84,101`) and `evals/trajectories/` on `ubuntu-latest` (`skill-eval.yml:29,47`, triggered by the `apps/services/gateway/gateway/routes/workflows/**` path filter), where `preexec_fn` is supported. **Do not report these four as a regression, and do not "fix" them by removing the rlimits.** On Windows the honest local signal is **69 passed / 4 known-Windows-fail** (73 in CI); a fifth failure is yours. + +#### 8.3a — Template gallery ✅ **AGENT-SAFE** + +**State: nothing exists.** No `workflow_template` table in migrations `132`/`133`/`134`, and zero `template` matches under `workbench/control_plane/src/app/workflows/`. This is a greenfield item, unlike the rest of Slice 3. + +**DECISION (agent-proposed, owner may overrule) — templates are repo JSON fixtures, not a DB table.** A template ships as a versioned file in the gateway route package (proposed home: `apps/services/gateway/gateway/routes/workflows/templates/.json` + a `templates.py` loader beside `catalog.py`, which is the module that already answers "what can the palette offer"). Rationale, resting directly on the 2026-08-03 owner decision in §1.4: templates are **product content that ships with the code**, not user data. Fixtures are reviewed in the PR that adds them, cannot drift between dev and prod, need no migration, no seeding path and no admin CRUD screen. A table would need all four — plus a migration whose number must be found by listing `infra/postgres/` at build time, never written in advance — to buy one capability nobody has asked for: **in-app "save as template" authoring**, which is also the direction §1.4 rules out as marketplace tooling. *Accepted cost, stated plainly:* adding a template is an engineer's PR, not a maker's button. If the owner wants maker-authored templates, this decision inverts and the table comes back — but then the ticket is a different, larger ticket and should be re-scoped, not stretched. + +**DECISION (agent-proposed, owner may overrule) — instantiation is a body field on `POST /workflows`, not a new route.** `WorkflowCreate` (`crud.py:35-37`) today carries only `name` + `description`; the field is `template: str | None = None`. The seam already exists: `duplicate_workflow` (`crud.py:141-202`) is *exactly* "insert a new draft carrying a graph + variables + triggers, regenerating the hook token because it is a credential" — instantiating a template is duplicating from a fixture instead of from a row. A third near-identical INSERT under a new route would be a parallel seam for no gain. Same rule applies: **the hook token is regenerated, never carried in a fixture** — a template file must not contain one, and the loader should refuse a fixture that does. + +**Done when:** +1. `POST /workflows` with `{"name": "...", "template": "lead-intake"}` returns **201** with a graph that `validate_graph` accepts with **zero issues**, and `POST /workflows/{id}/publish` on that new workflow **succeeds**. +2. An unknown `template` slug returns **422** naming the slug (not a 500, not a silent empty draft). +3. `POST /workflows` **without** `template` behaves byte-identically to today (the field is additive and optional) — pinned, because `create_workflow` currently has **no covering test at all**. +4. Every shipped fixture passes `validate_graph` in a parametrized test that walks the templates directory — so a broken template is caught at CI time, not at maker time. A fixture containing a `hook_token` fails the same test. +5. All of the above pinned in a new **`tests/unit/test_workflows_templates.py`**. + +**Template *content* is an owner input, not an implementer's guess.** Which Fracktal processes deserve a starter graph is a business question; the engineer can ship the mechanism against one throwaway fixture and the real set lands after. ⚠️ **A report/weekly-digest template is NOT this row's artifact** — `work_plan.md` §4 assigns digest workflows to **WS-15** (where they double as this spec's G1 launch metric). Building one here duplicates WS-15's deliverable under a second owner; leave it to WS-15 and consume it. + +#### 8.3b — Fan-in / join ✅ **AGENT-SAFE** + +**Prescribe the state-bus semantics before an agent touches the compiler.** The engine's shared state bus keys **one slot per node**: `state[node_id] = output` (`runner.py:135` on replay, `:179` on live execution), and `node_results[node_id]` is likewise a single slot that becomes the run's persisted per-node history (`workflow_runs.node_results`, `132_workflows.sql:81`). The message routed along edges, `_Token` (`runner.py:55-59`), carries **only** `branch: str | None` — data never travels on the edge, only the permission to proceed. And the compiler adds one MAF edge per connection (`_build_maf_workflow`, `runner.py:236-246`). Consequence, and the reason this is not a one-line validator change: **wire two edges into one node today and MAF delivers two messages, so the executor body runs twice** — the second pass overwrites both `state[node_id]` and `node_results[node_id]`, and the run history silently shows only the last pass. A join therefore needs a *defined merge shape*, not a relaxed check. + +The merge shape must be recorded in the spec (here) before implementation, and must answer all three of: +- **Where merged inputs live.** Proposed: the join node's own slot holds a dict keyed by **incoming source node id** — `state[join_id] = {src_a: out_a, src_b: out_b}` — so existing `{{join.src_a.field}}` reference resolution keeps working unchanged and nothing about `templating.py` has to learn a new shape. (Ordered-list-by-edge is the rejected alternative: edge order is not stable in the edit-model, so refs would silently re-bind when a maker re-draws an edge.) +- **Quorum.** *Which* branches must arrive. "All incoming edges" is **wrong on its face** — a condition sends down exactly one of `true`/`false` (`_branch_condition`, `runner.py:241-244`), so a join downstream of a condition would wait forever on an edge that is structurally dead for that run. The rule must be expressed in terms of branches that can still arrive, and the deadlock case must be a **named run failure with the waiting node id**, never a hang to `RUN_TIMEOUT_SECS`. +- **Interaction with pause/replay.** `_mark_unrun` currently marks never-run nodes `pending` while paused and `skipped` on a finished run (`runner.py:200-202`). A half-arrived join is neither; the resume path (`precomputed`, `runner.py:133-139`) must be able to rebuild a partial merge from the pause snapshot, or approval-under-a-join must be explicitly refused at publish. + +**Done when:** +1. **`test_fan_in_rejected_v1` (`tests/unit/test_workflows_engine.py:155`) inverts** — the graph it feeds now validates, and the `fan_in` `GraphIssue` (`graph.py:303-311`) is either deleted or narrowed to the shapes still refused. *An un-inverted pinned test is how this ticket closes green while doing nothing.* +2. A two-branch fan-out→join graph executes and `state[join_id]` contains **both** branch outputs under their source node ids; `node_results` records the join **once**, not twice. +3. A join whose second branch is unreachable (condition false) resolves per the quorum rule and does **not** hang — asserted against a bounded `run_timeout`, so a regression fails fast rather than sleeping 15 minutes. +4. A join that can never satisfy quorum fails the **publish gate** with a named issue, not at run time. +5. The golden eval (`evals/trajectories/test_workflow_engine_trajectory.py`) gains a fan-out→join trajectory; the six existing trajectories stay green unchanged. + +#### 8.3c — Loops ✅ **AGENT-SAFE** · **APPROVED BY THE OWNER 2026-08-03** + +**Owner decision, recorded against §11 R1.** §11's standing risk R1 warns against *"scope creep toward n8n"*. **The owner has explicitly decided that loops are worth the engine complexity** — real automations iterate ("for each row in this sheet…", "retry until the CRM accepts it"), and an automation platform that cannot iterate pushes makers back to the toil quadrant §1.2 exists to close. **R1 is not a blocker on this item and must not be cited as one.** R1 keeps its original and unchanged meaning: *a node exists only if the Integration Registry has the integration* — it governs the **node catalog**, not the **control-flow vocabulary**. This is a deliberate call, dated, not drift. + +**The cost the owner accepted, stated honestly.** Today the engine's simplicity *is* its correctness argument: a DAG with one slot per node terminates by construction, and MAF's completion detection needs no help. Loops give that up. The run model acquires iteration state, run history acquires cardinality, `RUN_TIMEOUT_SECS` stops being the only bound that matters, and the graph validator loses "no cycles" as a cheap universal safety net — every later engine change must now reason about non-terminating graphs. That is real, permanent complexity in the one subsystem that is CI-locked by a golden eval, and it was accepted with open eyes. + +**What must be designed, not discovered:** +- **Iteration state.** `state[node_id]` is a single slot overwritten on every pass (`runner.py:179`), and so is `node_results[node_id]` — which *is* the run history a human reads (`132_workflows.sql:81`). Unmodified, a 50-pass loop persists one pass and silently discards 49. The model must say what a node's slot means inside a loop (current pass? accumulated list?), what `{{node.field}}` resolves to for a node inside vs outside the loop body, and what run history records per pass. This is the load-bearing decision of the item. +- **A max-iteration bound.** Mandatory, a **literal in code**, enforced by the engine (not only by the wall clock), and exceeded ⇒ a **named run failure**, not a timeout. Whether it is also per-workflow configurable is secondary; the unconditional ceiling is not. +- **`foreach`-over-a-list vs true cyclic edges — DECISION required, and the two docs currently disagree.** The parent RFC §6 promises MAF handles cycles; `graph.py:326-329` forbids them outright. Both cannot stand. A `foreach` **body-scoped node** (bounded by the list, no cycle in the graph, `validate_graph`'s cycle check survives untouched, the golden eval's termination argument survives) is the smaller change and covers the "for each row" case that motivates the feature; true cyclic edges additionally cover "retry until", at the cost of the validator's cycle rejection and of every termination guarantee resting on the iteration bound alone. **Whichever is chosen, record it here with its rejected alternative before writing code** — and reconcile the RFC §6 sentence in the same change, because leaving it is how the next auditor finds a fourteenth contradiction. + +**Done when:** +1. **`test_cycle_rejected` (`tests/unit/test_workflows_engine.py:148`) inverts** for the shape the chosen design admits, and stays red for the shapes still refused. *Same trap as 8.3b: leave it asserting rejection and the ticket closes green having built nothing.* +2. A loop over an N-element list executes the body **exactly N times**, and the run's `node_results` accounts for **all N passes** (not one) in whatever shape the iteration-state model prescribes. +3. Exceeding the max-iteration bound produces a **named** failure (the message identifies the loop node and the bound) — asserted, not inferred from a timeout. +4. Publish refuses an unbounded / non-terminating loop shape with a named `GraphIssue`, so the failure lands at design time. +5. **The golden eval gains a bounded-loop trajectory** — the eval is the engine's semantic lock (`skill-eval.yml`'s path filter fires it on every `routes/workflows/**` edit, blocking, on `ubuntu-latest`), and after this item it is the *only* CI artifact asserting that a workflow with a cycle still terminates. Without that trajectory the termination guarantee is untested; the six existing trajectories must stay green unchanged, proving loops cost nothing on the loop-free path. + +### 8.4 Slice 4 — durable queued runs; sandboxed module execution; MCP exposure; retention policies. **BLOCKED — do not absorb any of it into Slice 3.** + +Gate labels: the build work is ✅ **AGENT-SAFE**, but Slice 4 cannot be *activated* by an agent — 🔴 **OWNER-GATE** on the `INGESTION_CONSUMER=1` flip (registered in `work_plan.md` §6) that the durable path rides. + +The old anchor read "post-BO‑20/BO‑7", which is too vague to sequence against. Precisely, Slice 4 needs: + +- **BO‑20b slice 2 → BO‑20c → (BO‑20d, BO‑20e)** — retry via PEL reclaim + an honest `XACK` + DLQ hand-off, a drainable/visible DLQ, per-source rate limiting, bounded concurrency. BO‑20a (the drain loop) and BO‑20f (Gmail/Zoho receiver parity) are built; **BO‑20b slice 1 (the `emit_event` strict mode) is built**; slice 2 and c–e are open. Until BO‑20b lands, **a failed dispatch is `XACK`ed and lost** — BO‑20a's deliberate interim. "Durable queued runs" on that substrate would be a lie in the status field. +- **The consumer is inert everywhere.** `INGESTION_CONSUMER` is unset in every environment, so the drain loop never starts and the receivers still emit inline. Flipping it is an owner gate and is not merely "start a loop": the same flag cuts all three provider receivers over to enqueue-only, so **Redis down = provider events dropped**. +- **BO‑7** — still ☐ (`FOUNDATION_BUILDOUT_CHECKLIST.md:110`). Sandboxed module execution graduates into it; §3.4's restricted-execution runtime is documented as insufficient for untrusted code until then. + +Anything in Slice 4 that looks reachable today is reachable only because its dependency is being skipped. ## 9. Key design decisions @@ -223,7 +307,9 @@ Aligned to RFC §9, resequenced so each slice ships value: - **D3 — Edit-model ≠ run-model.** React Flow JSON verbatim for editing; compiled serialized DAG per published version; runs pin versions. - **D4 — Nodes never hold secrets.** Integration resolution happens server-side at execution; graph JSON is safe to export/share by construction. - **D5 — Modules are pure transforms.** Import-free, I/O-free, time-boxed; everything bigger is a skill repo. The generator enforces this at authoring time, the validator at save time, the runner at run time. -- **D6 — APScheduler's `CronTrigger` as a parser inside a supervised asyncio loop** (the canonical gateway scheduler shape — no APScheduler *process*). Already in the dependency tree via ingestion; due ticks are CAS-claimed on `last_fired_at` so multiple workers can't double-fire, and downtime collapses to one catch-up fire. Revisit under BO‑20. +- **D6 — APScheduler's `CronTrigger` as a parser inside a supervised asyncio loop** (the canonical gateway scheduler shape — no APScheduler *process*). Already in the dependency tree via ingestion; due ticks are CAS-claimed on `last_fired_at` so multiple workers can't double-fire, and downtime collapses to one catch-up fire. Revisit under BO‑20 — **still correct and still unspent as of 2026-08-03**: `scheduler.py:57-61` imports `apscheduler.triggers.cron.CronTrigger` inside `compute_due_fire()` purely as an expression parser, and the one supervised `_scheduler_loop()` (`:270-284`) drives it via `_scan_once()`. The dependency is declared for exactly that and says so: `apps/services/gateway/pyproject.toml:35-37`. + **The house style now has two independent subsystems.** BO‑20a's ingestion drain (`apps/services/ingestion/ingestion/consumer.py`) shipped the identical shape — a supervised asyncio loop in the gateway lifespan rather than a scheduler/worker *process* — which strengthens D6 rather than dating it. **D6 needs no edit.** + ⚠️ **ID collision, flagged not resolved:** `work_plan.md` §3 also defines a **D6** — *"The Workflows app won"* (which names this spec as the winner over `multi_agent_orchestration.md` Phases 2–3). Two different live D6s are reachable from this row, so "D6" is ambiguous in any cross-doc sentence. Same class of defect `work_plan.md` §2 **R2** forbids for phase IDs (no ID reuse across docs), applied to decision IDs. Renaming either is a cross-doc edit that touches `work_plan.md` and is **not** in this spec's gift; until an owner picks, always qualify — *"D6 (`workflows_app.md` §9)"* vs *"D6 (`work_plan.md` §3)"*. - **D7 — The catalog is served, not hard-coded.** The palette's agents/integrations/modules come from the same registries the runtime uses, so the builder can never offer a capability the platform doesn't actually have (G3). ## 10. Policy reconciliation @@ -238,7 +324,7 @@ Aligned to RFC §9, resequenced so each slice ships value: - **Q3 — RESOLVED (implemented).** Any `workflows`-granted member may draft, validate, Test-run, and duplicate; **publish, rollback, and disable require `workflows:publish`** (`acb_auth` capability; migration 133 seeds it to owner/admin/manager — `member`/`guest` drop to draft-and-test, and an admin can hand it back per-user with an override). The line is drawn at *arming*: a draft fires no triggers and its writes are still broker-held, while publishing starts webhooks/cron/events running it unattended. `/auth/me` now returns resolved `capabilities` so the editor greys out Publish with an explanation instead of surfacing a bare 403. Still worth validating against real usage: whether `manager` is the right default tier. - **Q4** — Event-trigger volume before BO‑20: in-process runs are honest-but-lossy on restart; cap per-workflow concurrency and surface "missed while down" in run history, or hold F10 GA until BO‑20? - **Q5** — Module review policy: is generator + validator + test-before-save enough, or should `ready` status require a second human (approver) before a module is usable in published workflows? **Sharpened by F14:** copilot-created modules save as `ready` immediately (provenance `auto_created: true` makes them auditable and filterable); if review-before-ready is adopted, the copilot path should queue them as `draft` and say so in its reply. -- **R1** — *Scope creep toward n8n*: the catalog makes it tempting to add generic SaaS nodes. Rule: a node exists only if the Integration Registry has the integration — the registry is the roadmap. +- **R1** — *Scope creep toward n8n*: the catalog makes it tempting to add generic SaaS nodes. Rule: a node exists only if the Integration Registry has the integration — the registry is the roadmap. **Scope clarified by owner decision 2026-08-03 (§8.3c): R1 governs the node *catalog*, not the control-flow *vocabulary*.** Loops are approved and R1 is not a blocker on them; the engine-complexity cost was stated and accepted in §8.3c. R1 continues to bind unchanged everywhere else — a generic-SaaS node with no registry entry is still refused. - **R2 — MITIGATED (implemented).** *Silent automation drift*: published workflows keep running while the business changes. All three mitigations are in: run-history visibility (F9 + the gallery's last-run dot), per-workflow `owner_email`, and a **disabled-on-repeated-failure policy** — `AUTO_DISABLE_AFTER` (5) consecutive failed runs from *unattended* triggers (`schedule`/`webhook`/`event`) flips the workflow to `disabled` with the reason recorded. Three narrowings keep it from firing on a working system: manual/api runs never count (a maker debugging must not take production down, and one agent passing bad arguments must not take it from everyone else); the streak is consecutive, derived from run history, so any success breaks it; and only runs after `health_since` count, which is why re-enabling sticks instead of re-disabling on the next failure. Notification is in-product — persisted reason on the card and an editor banner, a `workflows.auto_disabled` warning log, and a `disabled` event on the activity feed (/observability). Outward notification (email the owner) is an outward write and belongs on the Action Broker path, not in the run's `finally` block. **Open:** whether 5 is the right threshold under real webhook volume, and whether a high-volume workflow should get a per-workflow opt-out — dropping events is not obviously better than failing them, and today the policy chooses to stop. ## 12. Success criteria (v1) @@ -248,4 +334,12 @@ Aligned to RFC §9, resequenced so each slice ships value: 3. A module generated in Module Studio from a plain-English description passes validation, runs against sample data, and is reused in a second workflow unchanged. 4. Every agent in the live registry and every integration action in the registry appears in the catalog with typed config — nothing hard-coded in the frontend. 5. No workflow can be published with an unresolved `{{ref}}`, a secret-shaped string in node config, or a write-class node without a Human-approval ancestor (`write_without_approval` — enforced at publish/validate/copilot; draft Test runs stay permissive since the runtime broker still holds any real write). -6. All engine/validator/generator logic is covered by unit tests that run without Docker; `uv run pytest tests/unit` green. +6. All engine/validator/generator logic is covered by unit tests that run without Docker. **Verification is the named four-file command, never `tests/unit/` as a directory** (the whole directory hangs on a Windows dev box and is not a usable signal): + + ``` + uv run pytest tests/unit/test_workflows_engine.py tests/unit/test_workflows_slice2.py \ + tests/unit/test_workflows_trigger_reliability.py \ + evals/trajectories/test_workflow_engine_trajectory.py -q + ``` + + Green means **73 passed** in CI (`ubuntu-latest`). On Windows the honest expectation is **69 passed / 4 failed**, all four being the `preexec_fn` module-sandbox defect catalogued in §8.3 — see that section before reporting a regression. Items 1–5 above are Slice-1/2 criteria and are met; Slice 3's criteria are per-item in §8.3a/b/c, not here. diff --git a/ai-company-brain/work_plan.md b/ai-company-brain/work_plan.md index e5bd342c6..f73e8a590 100644 --- a/ai-company-brain/work_plan.md +++ b/ai-company-brain/work_plan.md @@ -1,6 +1,8 @@ # Work Plan of Record — the dispatch board -**Status:** Active · **Date:** 2026-07-31 · **Owner:** vjvarada +**Status:** Active · **Date:** 2026-08-03 (six-row truth pass: WS-1, WS-3, WS-8, +WS-11, WS-12, WS-21 swept to match their rewritten specs; D10 records two owner +calls) · **Owner:** vjvarada **Purpose:** the single sequencing document from which independent agents are dispatched. Content lives in the owning specs; *this* doc owns ordering, ownership, and the rules that make a spec executable without questions. @@ -68,9 +70,9 @@ gap; calendar P3 was found already shipped (with revised roll-over semantics). | WS | Workstream | Owning spec | State | Next / notes | |---|---|---|---|---| -| WS-1 | **Action Broker truth + completion** (BO-1) | `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO-1 (corrected 2026-08-01) | 🟢 | Broker loop LIVE (inbox, `/actions`, ClickUp + WhatsApp + workflow handlers). Remaining: email/Zoho handlers, verify vs live DB. **OWNER-GATE:** flipping `ACTION_BROKER_ENFORCE` on. | +| WS-1 | **Action Broker truth + completion** (BO-1) | `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO-1 (rewritten + verified against code 2026-08-03) | 🟢 | Broker loop LIVE and writing (inbox, `/actions`, ClickUp + WhatsApp + workflow + app-publish handlers). **Handlers register at FIVE sites, not the three this row claimed:** `gateway/main.py:983-985` (the four ClickUp task actions), `main.py:1067-1069` (`workflow.resume_run`), `routes/whatsapp/scheduler_hooks.py:30` (`whatsapp.broadcast`) — those three at startup — plus `routes/apps/tools.py:211` and `:261`, which register **at module import**, not startup. ~~"Remaining: **Zoho** handlers"~~ **struck — the work does not exist.** `apps/services/ingestion/ingestion/sources/zoho/client.py` is read-only: six `list_*` functions and exactly two CRM calls, both `GET /crm/v2/*` (`:109`, `:152`); the one `POST` (`:58`) is the OAuth token refresh. There is no Zoho write path anywhere in the repo to route through the broker, so this is not BO-1 work until a Zoho write client is specced and built elsewhere. ~~"verify vs live DB"~~ → **OWNER-GATE, and the "already done 2026-07-13" claim is UNSUPPORTED** — `FOUNDATION_CONTINUATION.md:145` records it outstanding and nothing since records it executed; no agent may claim it done or reach prod to do it, and it is not an acceptance criterion for anything below. **Three new tickets in §BO-1, all AGENT-SAFE, one PR each — the first two are flip-blockers, both new findings:** **BO-1a** — `providers.py` routes **six** ClickUp action names through `_broker_gate` but `broker_handlers._WRITERS` registers **four**, and the two missing are the two *irreversible* ones (`clickup.delete_task` `:551`, `clickup.archive_task` `:575`); under enforcement, approving one falls into `broker.execute()`'s no-handler branch (`broker.py:155-166`) and the row is marked **`failed`**. **BO-1b** — `_broker_gate` returns `{"pending": True, …, "provider_task_id": ""}` (`providers.py:171-172`) and `items._push_pending_item` ignores the marker, writing `sync_state='synced'` with an empty `provider_task_id` — under enforcement the user sees a green "synced" task that exists in no workspace. **BO-1c** — email handlers (zero `action_broker` wiring under `email_ingestion/`), buildable but blocked on §BO-1's recorded decision naming which of the base class's **14** mutating verbs are broker actions. **OWNER-GATE:** flipping `ACTION_BROKER_ENFORCE` on — **not until BO-1a and BO-1b are both in**, for the two reasons above. | | WS-2 | **Secrets** (BO-8: rotate Zoho token, purge history, fail-closed) | checklist §BO-8 + `FOUNDATION_CONTINUATION.md` | 🔴 | **OWNER-GATE end-to-end** (force-push, rotation). Standing P0 since 2026-07-11. | -| WS-3 | **Isolation ladder** (BO-7 / HH-6 / B6 Tier 1→2, `tool_scope` deny, T2 for non-first-party agents) | `permissions_sandbox_b6.md` + `agent_platform_hardening_2026-07.md` Part 1 | 🟢 Tier 1 | Tier 1 container flags partially landed 2026-07-27 (competitive log) — reconcile B6 first. T2 is its own sub-project; required before Agent Workshop opens to non-engineers. **OWNER-GATE:** `AGENT_PERMISSION_MODE=enforce` flip. | +| WS-3 | **Isolation ladder** (BO-7 / HH-6 — T0/T1/T2 per `agent_platform_hardening_2026-07.md` §1.2) | `permissions_sandbox_b6.md` | 🟢 **WS-3a** (record + refuse, §P5-a.2) · 🟢 **WS-3b** (rootfs + network posture, §P5-b.2) | P5-a (per-run credential scoping, 2026-07-04) + P5-b.1 (cap/resource ceilings, 2026-07-27) shipped. **T2 / P5-c PARKED** under the internal-tool threat model (owner decision 2026-08-03, D10) — the ladder must hold against trusted colleagues, not hostile users; **un-parking is OWNER-GATE**, and no acceptance should be written for P5-c until it happens. P5-d is blocked behind it. **Two claims struck from the old title:** `tool_scope` deny belongs to **WS-23** (shipped there), and "T2 for non-first-party agents" named a distinction the code does not carry — no `first_party` field exists on any manifest, config or column; the phrase occurs only in comments and one test helper. **OWNER-GATE:** the `AGENT_PERMISSION_MODE` enforcement flip · P5-b.3's scoped gateway key (unbuilt *and* undesigned) · the new `ISOLATION_TIER_ENFORCE` flip WS-3a introduces. | | WS-4 | **Event-bus consumer + durable queue** (BO-20) | `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO-20 — **the file is at the REPO ROOT, not under `ai-company-brain/`** (this row's old anchor was wrong) | 🟢 a+f built · b slice 1 built · b slice 2 + c–e open | **§BO-20.0 IS ANSWERED — `BO-20 = Option A (in-process)`, owner, 2026-08-02.** Nothing in this row is blocked on a decision any more; the recorded rejection of Option B (a separate `python -m ingestion.worker`: needs a systemd unit no agent can deploy, and a separate process starts with an empty `event_hooks._SINKS`, so it would `XREADGROUP`, `XACK` and dispatch to nothing) is kept in §BO-20.0 as the reasoning, not deleted. **BO-20a BUILT 2026-08-02, pending review:** `apps/services/ingestion/ingestion/consumer.py` — `XGROUP CREATE cc-ingest $ MKSTREAM` on all three streams (`$` = tail, so the ~10k buffered entries per stream are skipped, not replayed into real workflow runs), a supervised `XREADGROUP` drain loop (`_GROUP="cc-ingest"`, `_BLOCK_MS=5_000`, `_READ_COUNT=8`, per-worker consumer name `gw--` because BO-20b's `XAUTOCLAIM` identifies a dead worker by it) decoding `{event_type, JSON data}` into `event_hooks.emit_event(source, event_type, dict)` and `XACK`ing, a long-lived pooled `redis.asyncio` client per `acb_common/activity.py:66-76`, `start/stop_ingestion_consumer()` + `consumer_status()` in the gateway lifespan (start `main.py:307`, stop `:364` — **unconditional**, like `stop_whatsapp_enrichment`), and the **§BO-20 Q1 cutover in all three receivers**: flag ON ⇒ enqueue-only, flag OFF ⇒ **dispatch-identical** to before (not byte-identical — each receiver now also does one function-body import + one `os.environ` read per request). Packaging defect closed: `ingestion` is now a declared gateway dependency (`pyproject.toml` + `uv.lock`), not an inheritance from the root workspace umbrella. Pinned by `tests/unit/test_ingestion_consumer.py` (41 tests; **77 passed** across the four-file fence — 41 + 10 + 22 + 4, the other three unmodified), no Redis/DB/network. **Adversarial review 2026-08-03 → APPROVE, no P0/P1;** the four P2s were repaired in-branch: a `asyncio.timeout(_DISPATCH_TIMEOUT_SECS=30.0)` around `emit_event` (one serial loop drains all three streams, so an unbounded await turned a per-event hang into a **bus-wide, silent** stall — strictly worse than the pre-cutover `BackgroundTasks` hang it replaces), a test pinning the lifespan start/stop wiring itself, one shared ordered timeline so criterion A can tell ack-after-dispatch from ack-before-dispatch (the line BO-20b edits), and `assert task.cancelled()` instead of the weaker `task.done()` — **the reviewer's last item was half a fix**: cancelling a task that has never been stepped makes asyncio raise `CancelledError` above the loop's `try`, so `task.cancelled()` passes against a swallowing loop too; the test now waits for the loop to reach its first read before stopping, and was verified red against a deliberately-swallowing `_consumer_loop`. ⚠️ **Ships OFF and is inert in every environment:** `INGESTION_CONSUMER` is unset everywhere, so the loop never starts and the receivers still emit inline. **OWNER-GATE:** flipping `INGESTION_CONSUMER=1` (registered in §6) — it is not just "start a loop": the same flag cuts the three provider receivers over to enqueue-only, so **Redis down = provider events dropped** rather than dispatched inline. That drop is now logged loudly (`.queue.dropped`, warning) instead of being silent, and must not be "fixed" by re-emitting inline. **Interim semantics, deliberate:** BO-20a acks after dispatch regardless of outcome — honest `XACK` + retry + DLQ is **BO-20b**, now split in two. **BO-20b slice 1 BUILT 2026-08-03:** `event_hooks.emit_event` gained a **keyword-only** `raise_on_error: bool = False` — the strict mode the consumer needs to observe a failure at all, since `emit_event` swallowed every sink exception by design and BO-20b's retry logic is dead code without it. Default unchanged (swallow, log `event_hooks.sink_failed`, run the next sink — a webhook must never 5xx); `raise_on_error=True` propagates the **first** sink exception and skips the remaining sinks. Keyword-only so the three receivers' three-positional-arg `add_task(emit_event, source, event_type, payload)` can never reach it, and the default is pinned as the literal `False` via `inspect.signature` so a later PR cannot flip provider-facing behaviour silently. `consumer.py` is **untouched** — it still acks regardless of outcome. Three new tests (`tests/unit/test_ingestion_consumer.py` §J), four-file fence **80 passed** (44 + 10 + 22 + 4, the last three unmodified); both mutants (drop the `raise`, flip the default) verified red. **BO-20b slice 2 is open, and its SCOPE GREW on 2026-08-03** (adversarial review, repair round 1): slice 1 is *necessary but not sufficient*. `main.py:1074` registers exactly **one** sink, `workflows.triggers.dispatch_event`, and its whole body sits inside a `try/except Exception` that logs `workflows.event_dispatch_failed` and returns `[]` (`triggers.py:45-46`, `:90-104`) — so `raise_on_error=True` is a **no-op on the real registry**: slice 2 would have called it, `dispatch_event` would have swallowed, `emit_event` would have returned normally, the loop would have `XACK`ed, and the event would be **gone** with no retry, no PEL entry and no DLQ row — with every test green, because the suite registers a *raising fake* sink, a shape production does not have. Slice 2 therefore also owns a keyword-only strict path in `dispatch_event` (`triggers.py` joins its Files list; `tests/unit/test_workflows_slice2.py` joins its regression fence, 80 → 90 passed), with the failure boundary prescribed in §BO-20b: **propagate** the `_get_db`/trigger-query failure and `RunRejected` (raised at `service.py:193-196` *before* the run row and the task, so nothing ran), **never** the per-run execution failures (fire-and-forget via `create_task` at `service.py:226` — re-delivering would start a *second* run of the same workflow on the same payload), and raise **after** the row loop so a partial dispatch is not made worse. §BO-20's non-goal "Not a change to `dispatch_event`" is **struck and qualified** accordingly — that is a third `DECISION (agent-proposed, owner may overrule)` on this row; the rejected alternative was to leave `dispatch_event` untouched and accept that the consumer cannot distinguish "dispatched" from "swallowed", i.e. BO-20b cannot deliver its guarantee. Slice 2 also carries two `DECISION (agent-proposed, owner may overrule)` entries recorded in §BO-20b, because the ticket as written was *satisfiable while doing nothing*: (i) **retry is PEL-and-reclaim, not an in-loop `asyncio.sleep`** — the prescribed `_backoff` schedule (1,2,4,8,16 s) was dominated by the same section's `_RECLAIM_MIN_IDLE_MS = 60_000`, so the two constants could not both be true; `_backoff` is **struck** (it was also unpinned at its *call site*, so it could be defined, satisfy all four asserted properties, never be called, and close green), a `_RECLAIM_EVERY_SECS = 30.0` periodic cadence is prescribed with a done-when that the periodic pass **exists**, and the attempt counter is `XPENDING`'s `times_delivered` (an in-process dict resets on restart ⇒ a poison entry never reaches the DLQ). The rejected in-loop model would have blocked **all three streams for ~165 contiguous seconds** per poison entry, reintroducing exactly what BO-20a added `_DISPATCH_TIMEOUT_SECS` to prevent; the accepted cost of the chosen model is retry latency quantised to the reclaim cadence (~5 min to succeed on the 5th attempt, ~6 min to DLQ). (ii) **a dispatch `TimeoutError` is a FAILED dispatch** (retry, then DLQ) — acking it is a silent drop, which is the thing this ticket abolishes; consequence: BO-20a's `test_a_hung_sink_times_out_and_the_bus_keeps_draining` must be **rewritten** by slice 2 (its ack assertion inverts; its bus-keeps-draining half is preserved). Also recorded: the DLQ write must **not** call the sync `queue.enqueue_dlq` from the async loop (fresh sync client per call at `queue.py:49`, blocks the loop, invisible to the `consumer._get_client` fake), and `XAUTOCLAIM`'s **third** reply element — ids whose stream entry `_MAXLEN` trimmed away — must be unpacked and logged, because on redis-py 7.1.1 the common two-element unpack raises `ValueError` and wedges the whole **drain loop** every cycle — the `try` at `consumer.py:294-298` spans `_ensure_groups` *and* `_drain_once`, so a failing top-of-iteration reclaim stops the bus draining entirely, at ~1 Hz, forever (the reclaim pass must be wrapped so its failure degrades to "no reclaim this cycle"). Also newly recorded in §BO-20b: `JUSTID` is **forbidden** (it suppresses the very delivery-counter increment the retry design rests on, and `redis-py` returns a bare id list that unpacks into three names *without raising*); the `XPENDING`-before-`XAUTOCLAIM` read order is pinned (the other order moves the observable DLQ threshold from 5 deliveries to 6 and no fake-backed test can tell); `times_delivered` counts **deliveries, not failures**, so a crash-loop burns retry budget on a healthy event (mitigated by recording it on the DLQ row); the reclaim's 60 s min-idle bound is **per entry, not per batch** and is safe today only because the loop is serial — a constraint now sits on **BO-20e** to bound per-entry idle before concurrency is enabled, or the same event runs twice; **per-stream ordering is given up** by PEL-and-reclaim and is now listed as an accepted cost (a stale `taskUpdated` can start a run after a fresher one); and the attempt counter survives a *gateway* restart but **not a Redis** one (`xgroup_create(id="$")` re-creates the group at the tail after a flush, and `infra/` sets no `appendonly`). ⚠️ **Two further "enqueued but never dispatched" states are now recorded in §BO-20a** beyond that accepted drop: the `XACK` is deliberately unguarded (a raising `xack` means Redis is gone and must reach the backoff, not hot-loop), and the loop only ever reads `">"`, so an ack failure or a SIGTERM **mid-batch** strands the rest of that `XREADGROUP` reply in the PEL under the old pid's consumer name. Only BO-20b's reclaim pass recovers them, and only until `queue._MAXLEN` trims — so **BO-20b's done-when now requires the reclaim pass to run at startup**, not only on the periodic cadence, and carries an explicit open sub-question about the min-idle bound at startup. **BO-20f (Gmail + Zoho receivers reach ClickUp enqueue+emit parity) shipped 2026-08-02** and is what multi-channel event triggers actually needed; it is still **inert in prod** — `zoho_webhook_secret` and `gmail_pubsub_token` default to `""`, both receivers fail closed, and **OWNER-GATE (an agent can do neither):** provision `ZOHO_WEBHOOK_SECRET` + `GMAIL_PUBSUB_TOKEN` on the VPS (`.env.example` is itself OWNER-GATE under WS-2 — the plan-guard hook blocks agent writes to it) **and** point the provider subscription/webhook at `/webhooks/{zoho,gmail}`. The fail-closed posture is correct and must not be changed. ⚠️ **Not a greenfield build:** webhook→run was ALREADY wired — ClickUp → `ingestion/event_hooks.emit_event` → `workflows/triggers.dispatch_event` → `start_run` since commit `e20ea830`, and `/agent/webhook/{source}` (`routes/agent.py:3476-3478`) is a second live path that calls `dispatch_event` **directly** and is **untouched by the cutover** — so §BO-20 Q1's old "the consumer becomes the single dispatch path" was loose and is corrected there to "the only caller of `emit_event`". **Remaining: BO-20b slice 2 → c → (d, e)** — retry via PEL reclaim + honest `XACK` + DLQ hand-off, a drainable/visible DLQ, per-source rate limiting, bounded concurrency; all ✅ AGENT-SAFE, each waiting only on its predecessor. **WS-11 Slice 4 still waits**: `workflows_app.md:217` defines it as "(post-BO-20/BO-7): durable queued runs; …", and durable means a–e — without BO-20b a failed dispatch is acked and lost. BO-9 resolved as **not blocking** (the consumer owns its own long-lived async client; the producer's per-call sync `queue._client` stays BO-9's, untouched here). | | WS-5 | **CI gates real** (BO-17/BO-18) | checklist §F | 🟡 Docs | Un-gate evals, blocking gitleaks, coverage floor. ~~AGENT-SAFE~~ → **mixed: the highest-value item is a GitHub *settings* change an agent cannot make.** **Audited 2026-08-01 → NO-GO**: §F has zero testable "done when" ("per the existing plan", "a few green PRs", "for foundation packages"), its ratchet-plan anchor points at a path that moved to `specs/archive/` (3 stale citations live *in the workflow files*), and BO-17 reads ☐ while half of it shipped (blocking ruff-correctness + xenon, a frontend tsc/vitest job, gitleaks, per-PR health). **THE MISSING ITEM — why the 2026-08-01 F821 escape happened, in no doc today:** (1) `main` has **no branch protection** (`gh api …/branches/main/protection` → 404) — every "blocking" gate in these YAMLs is decorative; (2) commits pushed straight to main get **zero check-runs** (`15c8933f` had none); (3) `deploy.yml:56-58` lints with the *non-blocking full* `ruff check .`, **not** the `--select F821,…` correctness gate, so deploy went green over a broken tree; (4) PR #318's `pr-check` **failed on that exact F821 and merged anyway**. **Slice when specced (BO-17a "main-guard"):** add a `correctness` job to `deploy.yml` on push-to-main running the `--select` gate, deliberately NOT in the deploy job's `needs:` — loud, not blocking. AGENT-SAFE. **OWNER-GATE:** enabling branch protection / required checks, wiring any gate into `needs:`, removing `skip_tests`; BO-18's purge+rotation is WS-2's, not this row's. Refuted two long-standing beliefs: pr-check **does** cover the frontend, and it **does** run on non-main branches. | | WS-6 | **Observability wiring + attribution** (BO-5 + decision D1) | `observability_e2.md` **§7** | 🟡 partial | **Docs gate CLEARED** (PR #319 added the numbered §7 with nine lettered tickets WS-6a–i, per-item done-whens and gate labels). **Re-audited 2026-08-02 → GO-NARROWED to WS-6a+WS-6c only.** ✅ **BUILT 2026-08-02, pending review:** D1's attribution stamp exists as a substrate — `instance` joins `_RUN_CONTEXT_KEYS`/`bind_run_context`, resolved once in `run_agent_stream` via a **second additive bind** after `load_agent` (the early bind stays: it is what correlates a failure *during* load; moving it would trade 5 fields for 1), and `_emit_usage` carries the full (run, member, agent, instance) tuple with **zero call-site changes** — it arrives by inheritance via `activity._INHERIT`. Shared agents produce an **absent key, never `''`** (double-guarded + pinned). `refresh_run_presence()` patches `cc:activity:live:{run_id}` after the late bind, so `/observability/active` + `/roster` carry it; interim `by_instance` cost dimension added to the Redis rollup. **Nothing durable is written yet** — logs + Redis feed only. **🔴 WS-6b/6d/6e HELD, still NO-GO:** WS-6b's security amendment names *no workable mechanism* — `bind_run_context` has one call site (`executor.py`), contextvars do not cross the HTTP hop to `v1_compat`, and `agent_run` rows are written at the run *boundary* so a mid-run join finds nothing. **The only mechanism the code supports at request time is the presence key `cc:activity:live:{run_id}`**, which for the orchestrator path carries a server-established `user`; §7 must name it (or name another) before WS-6b dispatches. WS-6e has no token source (`build_run_trace_row` is pure over events+folded) so it sequences *after* WS-6b, not independently; WS-6d additionally waits on the retention/PII answer (Q3). **Two recorded asymmetries** — the `phase="start"` event predates the bind, and **a delegated sub-run inherits the caller's partition** while its blobs key to `''`, so WS-6d must not treat `instance` as a foreign key onto `agent_blob.instance`. **OWNER-GATE:** WS-6f/g/h/i (Langfuse keys, `--profile obs`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `LLM_USAGE_AUDIT`, the MAF telemetry kill switch) — all now listed in §6. | @@ -80,11 +82,11 @@ gap; calendar P3 was found already shipped (with revised roll-over semantics). | WS | Workstream | Owning spec | State | Next / notes | |---|---|---|---|---| -| WS-8 | **Agent architecture A0→C** (single runtime, manifests + `agent_defs`, generic declarative builder, Agent Workshop describe-to-create) | `agent_architecture.md` §12 | 🟢 A0/A1 | A0 items partially done (approve_all fixed 2026-07-26 — three states in one doc, see §5). Phase A unblocks D3's long-term form. | +| WS-8 | **Agent architecture A0→C** (single runtime, manifests + `agent_defs`, generic declarative builder, Agent Workshop describe-to-create) | `agent_architecture.md` **§12.2** (the lettered tickets WS-8a…WS-8n) | 🟡 | A0's `approve_all` half done 2026-07-26. ~~"three states in one doc, see §5"~~ **repaired 2026-08-03** — §5 doc-remediation item 14 is closed (one A0 status; the F/G dependency split is written). **~60% of Phases A+B is unwired substrate — read §12.1 before dispatching anything from this row**, or an implementer will rebuild `manifest.py` / `declarative.py`, both of which are complete, documented and tested with zero production callers. ~~"Phase A unblocks D3's long-term form"~~ **struck — verified false in the direction that matters:** `config.json`-based instancing already ships via `AgentManifest.instance_key()` (`manifest.py:235`, live at `executor.py:917-937` and `routes/workspace.py:247-256`, with a `sharing` block on all six first-party agents), so **WS-14 is NOT waiting on WS-8 Phase A** (§12.5). D7's MAF-side MCP gap is now a ticket here — **WS-8c**. | | WS-9 | **Memory tiers 3b/3c/4** (budgeted file-tier header, provenance markers, correction UX, supersession) | `memory_architecture.md` §9 (corrected 2026-08-01) | 🟡 Docs | 3a′ substrate shipped (migs 136–139). §6.7 correction UX is the highest-leverage UX item in the corpus. **Audited 2026-08-02 → NO-GO**: §9 gives acceptance for **3a′ only** (which is WS-10's, already shipped) — 3b/3c/4, the whole of WS-9, have none; §6.7 is experience prose with no endpoint, model or assertion; §6.5 ends "there are two honest paths… don't do both", an owner call presented as acceptance. Header still says `Draft / RFC · 2026-07-26` over a body stamped 2026-08-01 (R4). Paths are bare filenames whose line numbers have moved (`routes/memory.py` gate is now `_authorize_scope` :128-167; `_tool_injection.py:488-493` moved to `acb_skills/addendum.py` in WS-23 S3). **Verified substrate:** `MemoryClient` has search/add/get_all/delete and **no `update`**; the API has no PUT/PATCH; `/memory` already does list + semantic search + delete + clear-all (§5.5 understates it) but has **no edit, no provenance**, and hardcodes one of the **five** scope shapes (`` · `prefs:` · `room:` · `agent:` · `org:global`). No provenance/supersession fields exist anywhere. **NOT owner-gated** — the gate logic is testable against a fake with Mem0 disabled (41 tests, 0.58s); the real trap is inverted: **this box's `.env` already has Mem0 enabled, and `tests/unit/test_memory_integration.py` HANGS (measured exit 124); assume `test_memory_e2e.py` does too — name test files, never `tests/unit/`.** **D4 constraint:** `orchestrator/agents.py:520-534` reads only the user scope, so correcting an `org:global` fact would show fixed in the UI and change nothing on that path — PR-1 must restrict to ``/`prefs:`/`agent:` or say so. **Slice when specced (3c-0, AGENT-SAFE):** `PATCH /memory/{scope}/{memory_id}` reusing `_authorize_scope(write=True)` + the 404-not-403 membership probe at `memory.py:237-240`; `MemoryClient.update`; provenance in **Mem0's own metadata** (`corrected_by`/`corrected_at`/`supersedes`) — no new table; PATCH in the Next proxy; inline edit + compartment selector on `/memory`. **Scope creep to cut:** §6.1 instance-keying is WS-14's and the 3a′ remainder is WS-10's — this row should stop claiming both. | | WS-10 | **Multiplayer remainder** — S1 `subject:` compartments · floor-control re-decision · `prefs`/`user` backfill | `docs/multiplayer/memory-clearance.md` §7 + §7.1 (**the dispatchable slice**) · `docs/multiplayer/README.md` §8 (room-side index) — *`specs/multiplayer_prior_art_qm_2026-08.md` is reference-only per §4 and supplies acceptance for nothing* | 🟡 Docs → S1 | **Steer is SHIPPED — struck from this row's title** (`15c8933f`, ancestor of `main`: `orchestrator/steer.py::route_turn` → DROP/ENGAGE/ABORT/STEER, durable `cc:steer:` signals, `202 {"steered": true}` stand-down, `409 steer_outside_run_floor`, plus the two-layer supersede guard; `tests/unit/test_steer_routing.py` + `test_supersede_guard.py` green). **Audited 2026-08-01 → NO-GO on 5 of 7 contract points; §5-style remediation applied 2026-08-02** (both docs re-headered "verified against code on 2026-08-02", §3.5's 5 stale anchors fixed, gate labels added, verification blocks added). **That remediation was then independently verified and returned FAIL; repair round 1 landed the same day.** The P0 was the remediation's own new claim that `mark_active(reset=True)` raises `SupersedeRefused` — **it does not**: `mark_active` (`stream_relay.py:343-405`) deletes the stream at `:377` with no ownership check, and the only `raise` is at `:895` inside **`run_detached`** (`:823`), before it calls `mark_active` at `:909`. So the guard covers `run_detached`'s callers, **not** the destructive statement; both docs now say so, and README §12.3 carries an anchor grep that shows the line ordering. Six smaller defects fixed with it: `feature:memory` is `permissions.py:68` (not `:70`); §7.1.3 dw1 said "member" where §7.1.5 allows members (now **non-member**); dw5's `409` now matches its own precedent's **400** (`routes/rooms.py:533-538`); the slug grammar no longer claims `_clean_slug` (which forbids `.`, allows a leading `-`/`_` and unicode alnum) — it is `_SEGMENT_RE`'s shape plus a 64-char bound; `subject_ref` now reads as a **compartment scope key** everywhere (§3.2/§3.4/§4.1/§7.1.8), not an entity ref; and three already-green done-whens (§7.1.1 dw2/dw3, §7.1.5's miscounted row) were **replaced with criteria that require the work**, not merely labelled. Residual recorded, not built: moving the ownership check into `mark_active` would make it an invariant over the statement — no ticket minted for it here. **The row is now three things, and only one is work:** ① **`subject:` compartments = WS-10 S1, the dispatchable slice.** It is the one item with real query-layer acceptance (`memory-clearance.md` §7, kept verbatim) — it was NO-GO only because the surface it presumes was unspecified. **`memory-clearance.md` §7.1 now specifies it** (create/add-member endpoints and their gating, the `subject_ref` writer folded into the existing `PATCH /sessions/{id}/room`, the `_authorize_scope` rule, `audience='team'` → the shipped `org_group`, and a testable `sensitivity='restricted'` = *existence is confidential*, 404-not-403). Every decision there is marked `DECISION (agent-proposed, owner may overrule)` — **AGENT-SAFE once §7.1 is accepted**: dispatch after the owner reads it, or overrule and re-dispatch. (The owning spec's own Gate cell now carries that qualifier too — it read an unqualified "AGENT-SAFE" until 2026-08-02, and by this board's Authority rule the owning spec out-ranks this row for *what to build and how*, so the weakest of the three preconditions was the one that would have won.) **Repair round 2 (2026-08-02) — adversarial review returned REQUEST-CHANGES with no P0 and five P1s; all repaired in the same change.** The one that mattered: §7.1.4 specified the clearance cap as *"computed the way `_capability_cap` (`rooms.py:191`) already computes the credential cap"* — but `_capability_cap` **drops `group:` and `org` subjects by design** (`:207`, and short-circuits empty at `:208-212`, its own comment at `:209-211` saying so), so an implementer following that pointer would have turned the intersection into a **union** for exactly the rooms where a leak is widest: `[owner@x, group:sales]` bound to a restricted subject would come back with an empty cap, read as "no non-member participants", and admit the compartment to `Clearance.read` for forty people — while done-when 4 ("a non-member participant") passed green against two email addresses. §7.1.4 now names the site (`_subject_clearance_cap` beside `_capability_cap` in `routes/rooms.py`, consumed at the tree's only `resolve_clearance` call site, `routes/agent.py:1768-1774`), requires participants to be **expanded before** the intersection through one factored-out helper (`acb_auth.access.expand_session_subjects`, lifted out of `resolve_session_access` `:343-434` which already does the `group:`/`org` expansion at `:330-340`), and requires that expansion to **fail closed** — the opposite posture to `resolve_session_access`'s deliberate fail-open at `:417-426`, stated as such so nobody "fixes" it back. Done-when 4 is now four parts that cannot be satisfied without the `group:` and `org` cases. The other four P1s: the prior-art doc's QM-1 state cell still read "designed, unbuilt" for shipped steer (and QM-2 "✖" for built-but-off S4) while two other files in the same change said built; README §2's anchor table was **7 wrong of 8** under a "verified" header (fixed + caveat added, plus four stale repeats outside the table); §5.2 cited `test_reset_wipes_the_event_log` as demonstrating the `mark_active` bypass when that test seeds no `cc:runactor:` and so **cannot distinguish the two states** (README now says no test demonstrates it and describes the one that would); and §7.1.4 done-when 6 asserted a `422` on unknown `PATCH` keys that shipped code does not produce — `RoomPatch` (`routes/rooms.py:81-84`) is a plain `BaseModel`, no `extra="forbid"` anywhere in the gateway, verified against the repo interpreter (pydantic 2.13.4) — so it now pins the *real* behaviour and closing the model is filed as its own ticket in §7.1.9 rather than smuggled into this slice. ② **Floor control = OWNER-GATE, registered in §6 by name.** Per QM-1 steer dissolved most of the problem the baton was invented for; README §8 Phase 2 says whether the five modes still earn their place is *"pending the owner's re-decision"*. No acceptance is written for it on purpose — writing one would make an owner call look like queued work. ③ **`prefs`/`user` backfill** — classifier + **dry-run report** is AGENT-SAFE; **applying it is OWNER-GATE**, registered in §6 (mutates live Mem0). Verified: nothing writes a `prefs:` key anywhere today, so `prefs:` is permanently empty until this runs. **Two prior-art corrections (2026-08-02):** QM-3's *"rather than one `acting_identity`"* was factually wrong — there is no such column and never was (mig 138 `:26` rejects it explicitly), so QM-3 is net-new work with zero acceptance and maps to **WS-2 / WS-1, not here**; and the R2 phase-ID collision is resolved — the prior-art doc called `subject:` compartments "3b" while the owning spec puts them in the **3a remainder**, so the owning spec's ID wins and the board calls the slice **S1**. QM-5 (tenure narrows the model, not just the viewer) is a **real gap with an undone design**: viewer half built (mig 138 `:97-98` → `rooms.py:277-292` → `chat.py:314-316`), model half not (`_get_messages(thread_id, _hist_uid, …)` at `routes/agent.py:1947-1956` narrows by the acting caller only) — but README §6.5 says the two mechanisms are *"worth comparing before building either"*, which is a decision to record, not acceptance. | -| WS-11 | **Workflows Slice 3** (full-graph copilot authoring, loops/fan-out, template gallery); Slice 4 after WS-4 | `workflows_app.md` §8 (inconsistencies fixed 2026-08-01) | 🟢 | — | -| WS-12 | **Framework uplift + context discipline** | `multi_agent_orchestration.md` Phases 1, 4 (D6 banner added 2026-08-01) | 🟢 | Phases 2–3 marked superseded by the shipped Workflows app; Phase 5 orchestrations stay live. Phase 1's addendum-size target is delivered through WS-23. | +| WS-11 | **Workflows Slice 3** (template gallery, fan-in/join, loops); Slice 4 after WS-4 | `workflows_app.md` **§8.3** (re-scoped truth pass 2026-08-03) | 🟢 | Slice 3 = **8.3a** template gallery · **8.3b** fan-in/join · **8.3c** loops (**owner-approved 2026-08-03**, D10 — §11's standing anti-n8n rule R1 governs the node *catalog*, not the control-flow *vocabulary*, and must not be cited as a blocker on loops). All three AGENT-SAFE. **~1/3 of this row was struck:** "describe→generate→refine full-graph authoring" **shipped as F14** (`39b1e17a`) — dispatching it would have sent an implementer to rebuild the live `POST /workflows/{id}/copilot`; "parallel fan-out" also ships (`engine/graph.py:17`, MAF's superstep scheduler routes it), so the real remaining content is fan-**in** plus loops. Templates are greenfield — nothing exists. **8.3b and 8.3c each invert a pinned test** (`test_fan_in_rejected_v1`, `tests/unit/test_workflows_engine.py:155`; `test_cycle_rejected`, `:148`) — leave either asserting rejection and the ticket closes **green having built nothing**. Template *content* is an owner input; the report-digest template is **WS-15's** artifact, not this row's. Slice 4 stays blocked on **BO-20b slice 2 → BO-20c → (BO-20d, BO-20e) + BO-7** (§8.4), and its activation rides the OWNER-GATE `INGESTION_CONSUMER` flip. | +| WS-12 | **Framework uplift** | `multi_agent_orchestration.md` **Phase 4 only** (D6 banner 2026-08-01; shrunk 2026-08-03) | 🟡 Ph4 | **Audited NO-GO on all seven contract points; shrunk to Phase 4 only on 2026-08-03, not closed.** Ph0 shipped. **Ph1 struck** — 1.1 shipped as *progressive disclosure* (`93b93a08`, #191); 1.2 moot (`technical-project-planner` exists in neither `_AGENT_REGISTRY` nor `apps/agents/`); 1.3 delivered by **WS-23**. Ph2–3 superseded by the shipped Workflows app (D6). **Ph5 struck** — 5.2 shipped as multiplayer rooms *without* the orchestrations package, so it never depended on Phase 4; **5.1 is reassigned to WS-11**. **Ph4 is the genuinely undone part** — all four §5.5 shims re-verified in-tree 2026-08-03. **Drift correction: Phase 4 drags ONE SDK major, not two** — `uv.lock` and the repo `.venv` both carry `openai 2.38.0`, so the billed `openai 1.99 → 2.x` major already landed independently; only `github-copilot-sdk 0.1.32 → 1.0.2` remains. **0 PRs dispatchable today:** 4.0's target choice (minimal- vs full-bump) is **OWNER-GATE**; 4.1 (resolution proof in an isolated throwaway venv, evidence-only, AGENT-SAFE — it must never mutate `/.venv` or `uv.lock`) is what unblocks it. | | WS-23 | **Skills registry + per-agent skill toggles** (added 2026-08-01) | `specs/skills_registry.md` | 🟡 S1+S2 built | **S1+S2 shipped pending review 2026-08-01**. S1: `acb_skills/skill_families.py` registry + measured token-cost catalog, `GET /integrations/skills`, Integrations → Skills tab, drift test; measured baseline ≈19.3k tokens (core floor ≈15.1k). S2: `agent_skill_setting` table (override-shape provenance), `GET/PUT /agent/{name}/skills` (`admin:access:manage`; core/apps → 422), **intersection-only** enforcement in `_resolve_injected_scope` (no rows ⇒ byte-identical — regression-tested), Agents-page Skills panel with live token meter; decision note in spec §2: workflows toggle honored at its append site, Custom-App grants NOT toggle-governed. **S3 generation half + scope-out shipped pending review 2026-08-01**: addendum prose now GENERATED from family-tagged section registries in `acb_skills/addendum.py` (one renderer for injection AND catalog cost measurement; tool set byte-identical, text identical except the `App()Ellipsis` f-string fix); evidence-based scope-out in `specs/skills_scope_out.md` (GENERAL = core/memory/workflows/apps; SPECIALISED = history→orchestrator, coding→apis-config); `DEFAULT_PROFILE` + `SKILLS_FAIL_CLOSED` switch prepared and **shipped OFF**. Measured: all-families 19.3k → DEFAULT_PROFILE 17.8k → core floor 15.4k tokens — the ≤2k email target needs a core-floor diet, not toggles. Remaining: **OWNER-GATE** the `SKILLS_FAIL_CLOSED=1` flip (review dynamic agents first, `skills_scope_out.md` §4). Per-instance profiles defer to Centers C; manifest side lands with WS-8. **S4 core-floor diet BUILT 2026-08-01** (`skills_scope_out.md` §7): *Half A* `acb_skills/skill_index.py` — addendum becomes one line per family + `recall_notes("skills/.md")`, bodies materialized to `agent-data/skills/` content-hash-idempotently after the blob rehydrate, byte-preserved via the new `addendum.rendered_parts()`, index inside the prompt-cache-stable prefix, **`SKILLS_INDEX_ONLY` ships OFF**; *Half B* schema trim, live, **zero call-contract change** (pinned in `tests/unit/test_tool_schema_diet.py`). Measured: addendum 5,697 → **570**, core-floor schemas 9,998 → **8,510**, full surface 19,259 → **12,644**, email-assistant-recommended 17,757 → **11,337**. **≤2k still NOT met and unreachable by trimming** (22 schemas cost 1,252 tokens with descriptions deleted) — progressive tool disclosure + an `emit_generative_ui` schema pointer are designed and costed in `skills_scope_out.md` §7.5, **deliberately not built**. Remaining: **OWNER-GATE** the `SKILLS_INDEX_ONLY=1` flip. | ### Product — Centers (`department_centers.md` §3) @@ -104,7 +106,7 @@ gap; calendar P3 was found already shipped (with revised roll-over semantics). | WS-18 | **Tasks Phase 3** (Weekly Review, Waiting-For, ~~Horizons~~) | `task_manager_app.md` (corrected 2026-08-01) | 🟡 partial | **Audited 2026-08-02 → GO-NARROWED, and point 3 splits per view — the first row in four cycles to clear it.** ✅ **Waiting-For *surfacing* BUILT 2026-08-02, pending review** (`lib/waiting.ts` pure predicates + `WaitingForView.tsx` grouped by person + `ITEM_SELECT`/`GtdItemModel` now project the write-only mig-48 columns `expected_by`/`last_nudged_at`; **no migration — the substrate all shipped in mig 48**). Delegate now defaults `expected_by` from the item's own `due_at` (the in-app delegate path wrote NULL, so the headline §12 journey produced no flag at all). Fixed en route: a frozen `MOCK_NOW` (4 copies) that made the shipped overdue badge wrong by 33 days and growing, plus `mockData.ts`'s orphaned anchor. **🔴 Weekly Review = NO-GO** (§9.2 is a bare checkbox; `gtd_reviews.summary` is untyped JSONB — define the JSON contract + a per-movement done-when first). **🔴 Horizons = NO-GO and MIS-ASSIGNED** — no acceptance criterion exists anywhere, `gtd_horizons` has no link column to items/projects, and **the spec puts it in Phase 4, not 3**; strike it from this row's title or move it in the spec. **~~Open~~ CLOSED 2026-08-02 (follow-up):** `expected_by` now means exactly one thing — **an explicit human promise**. NULL ⇒ no promise was made, so the overdue line is the item's own `due_at` read **live** (nothing copied, nothing to go stale); non-NULL ⇒ a promise that stands independent of `due_at`. All four insert sites stopped deriving a copy (each was writing the item's own due date under another name), so the column is now written by exactly one path: `PATCH /tasks/items/{id}` with `expected_by` (ISO sets, `""` clears), which updates the open `gtd_waiting` row under a re-stated ownership `EXISTS`. Client judges `expectedBy ?? dueAt`. **No migration, no backfill** — rows delegated before this change keep their snapshot and stay judged on it; clearing one is a normal edit. **OWNER-GATE:** nudge drafting/sending (real-account email sends), delegation write-back to ClickUp (blocked on BO-1). **Drift found:** `gtd_reviews`/`gtd_horizons` have existed since mig 48 with zero gateway references — do NOT write a new migration for them; and the spec's `POST /tasks/projects/plan` was fiction (real: `POST /tasks/plan` + `/plan/apply`, shipped — only the ProjectPlanner UI is missing). **EVAL-LOCKED:** `propose()`/`propose_with_llm()` in `routes/tasks/ai.py`. | | WS-19 | **Notes + meeting bot** (share-to-chat, ask-during-recording; bot Phase 2 error codes AGENT-SAFE) | `note_taker_app.md` + `meeting_bot_platform_plan.md` | 🟡 | **OWNER-GATE:** bot Phase 1 needs a human-created Google account (`notetaker@fracktal.in`); share-to-chat needs a Slack integration that doesn't exist (scope call). | | WS-20 | **WhatsApp activation + remainder** (search UI 🟢 AGENT-SAFE; OCR needs a vision-tier decision; Odoo/Zoho-bound items blocked) | `whatsapp_message_manager.md` §11 (header fixed 2026-08-01) | 🟡 owner | **OWNER-GATE:** Meta env/app review, enrichment cost flags. | -| WS-21 | **Calendar F2/F3** (`gtd_time_blocks`, email windows, ideal week, external sync) | `calendar_focus_os.md` + `calendar_timeboxing.md` (acceptance + verification added 2026-08-01) | 🟢 F2 | P3 roll-over found ALREADY SHIPPED (released-to-unscheduled semantics, mig 78 + `start_auto_rollover`). Focus Shield stays blocked on the missing notification hold/release hook. Verify "breaks in the packer" state before dispatch (§5 residual 4). | +| WS-21 | **Calendar F2/F3** (`gtd_time_blocks`, email windows, mobile timeline, external sync) | `calendar_focus_os.md` **§9** (canonical for all F2/F3 acceptance; **§5** canonical for `gtd_time_blocks`) + `calendar_timeboxing.md` **§13** (canonical for P4) — both rewritten 2026-08-03 | 🟡 partial | **Re-audited 2026-08-03 → GO-NARROWED.** P3 roll-over was already shipped (released-to-unscheduled, mig 78 + `start_auto_rollover`). ~~"ideal week"~~ **struck — substantially shipped** (mig 98 + settings round-trip + editor + grid render + packer honouring + 2 unit tests); only the unused-focus-window / template-adherence gap remains (§9.6). **Breaks-in-the-packer SHIPPED 2026-07-23** (`80722e17`, mig **97**) as *packer geometry* — a widened buffer plus lunch protection, **a gap, not a `kind='break'` row**, which is exactly why F2 survives (§5 residual 4, now closed). **The 2026-08-01 acceptance was satisfiable by doing nothing** — 2 of its 3 `gtd_time_blocks` clauses were already green against shipped code; they are deleted and replaced with four that all fail today. **`gtd_time_blocks` is 4 slices, not 1 PR** (§9.1 S1–S4): the "non-breaking `TimeBlock[]` swap" claim was **FALSE** — the measured blast radius is 17 TS files + 3 gateway modules + `apps/skills/skill-task-gtd/` + `apps/agents/agent-task-manager/`. **Focus Shield is AGENT-SAFE, not owner-gated** (§9.5) — it needs a design, not a credential; do not dispatch on §4.1 prose alone. **Top-5 outcomes (Horizons) — DO NOT DISPATCH:** it collides with WS-18; §4 assigns it here, and WS-18's title keeps it struck. **Verify by naming test files — never `pytest tests/unit -k calendar`**: `-k` still collects the whole directory, and whole-directory collection hangs on the Windows box. **Dispatchable today:** §9.1 S1 · the ritual-stamp localStorage residue (§9.1 done-when 4, independently shippable) · §9.6 · §9.7. **OWNER-GATE:** external sync (§9.11 / timeboxing §13 P4) needs Google Calendar and/or Microsoft Graph OAuth client credentials provisioned on the VPS. | | WS-22 | **draw.io** (all 13 tickets open, nothing built) | `drawio_integration.md` | 🟡 owner | Best acceptance structure in the corpus; needs an owner and re-verified anchors (~5 weeks stale). ST-DRW-02 is a decision gate. | --- @@ -112,7 +114,8 @@ gap; calendar P3 was found already shipped (with revised roll-over semantics). ## 3. Decisions recorded (2026-07-31) Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are -**proposed defaults, adopted unless the owner objects**; D9 is an owner call. +**proposed defaults, adopted unless the owner objects**; D9 and D10 are owner +calls, taken and dated. - **D1 — Cost attribution is one workstream.** Stamp every LLM call at the gateway choke points with (run_id, member_email, agent, instance). Per-room @@ -126,6 +129,11 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are number) to unblock WS-14. When WS-8 Phase A lands, those columns become *derived from* `agent_defs` manifests — one store, not two. The agent_architecture manifest is the long-term source of truth. + **Amended 2026-08-03:** WS-14's unblock does **not** wait on WS-8 Phase A. + `config.json`-based instancing already ships via `AgentManifest.instance_key()` + and is live on the blob store and the workspace file manager with no schema + change (`agent_architecture.md` §12.1/§12.5). The `dynamic_agents` columns are + WS-14's own migration; Phase A only changes where they are *derived from*. - **D4 — Orchestrator org-memory: patch now, unify later.** The missing org/agent-scope read on the orchestrator path (`agent_architecture.md` §11.1.2) is fixed as a small standalone defect in WS-15. WS-8's A1 runtime @@ -135,7 +143,9 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are - **D6 — The Workflows app won.** `workflows_app.md` + `docs/workflow-editor/` are authoritative for graphs, compiler, editor, and workflow-as-tool. `multi_agent_orchestration.md` Phases 2–3 and §5.3 are superseded; its - Phases 1/4/5 remain live as WS-12. + **Phase 4 alone remains live as WS-12; Phase 1 was struck to WS-23 and + Phase 5.1 to WS-11 on 2026-08-03** (Phase 5.2 shipped as multiplayer rooms + under WS-10). - **D7 — MCP registry exists, with a MAF-side gap.** `13_mcp_servers.sql` + gateway CRUD + per-run injection are live (the coherence audit missed it by searching for the spec's planned name — R1's disease exactly). @@ -145,7 +155,10 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are native-MAF agents MCP injection is a **silent no-op** (no `MCPStdioTool`/`MCPStreamableHTTPTool` wiring exists). Any manifest `capabilities.mcp_servers` promise (agent_architecture §6) is unimplemented - on MAF until WS-8 closes this; scope it into WS-8 Phase A/B. + on MAF until WS-8 closes this. **Retargeted 2026-08-03:** that instruction is + now carried in the owning spec as the ticket **WS-8c** + (`agent_architecture.md` §12.2, AGENT-SAFE) — dispatch it from there, not from + this decision record. - **D8 — Budgets/caps enforcement lives at the gateway choke points**, never per-app. (Same principle as prompt caching and model tiers: one seam.) - **D9 — "Pomad Centre" — RESOLVED 2026-08-01.** Owner confirmed it is not a @@ -155,6 +168,29 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are security-requirement sites (`agent_platform_hardening` §64's T2 gate now reads "Before multi-tenant (a second org on this platform)"). The name no longer appears anywhere outside this decision record. +- **D10 — Two owner calls taken 2026-08-03.** Recorded here so neither is + re-litigated by a later dispatch. + 1. **Command Center is an internal Fracktal tool.** The team uses it; there + are no external tenants and no third-party agent authors. **Consequence, + already applied in the specs:** WS-3's T2 / full run sandboxing + (`permissions_sandbox_b6.md` §P5-c) is **parked** under a + trusted-colleague threat model — the ladder must hold against colleagues, + not hostile users, and P5-a's credential scoping plus P5-b's ceilings plus + WS-3a/WS-3b address the concrete standing exposures. **Un-parking is + OWNER-GATE and has an explicit condition: a second org on this platform, + or agent authorship from outside Fracktal.** Until then P5-c carries no + acceptance criteria and none should be written; P5-d is blocked behind it. + The same threat model is what makes `ACTION_BROKER_ENFORCE` OFF an + acceptable posture (audit-and-chokepoint rather than per-click approval) + and what bounds the Agent Workshop's value in `agent_architecture.md` §12. + 2. **Loops in the workflow engine are approved**, against `workflows_app.md` + §11's standing anti-n8n rule R1. Real automations iterate; an engine that + cannot iterate pushes makers back to the toil the app exists to remove. + The engine-complexity cost was stated and accepted. **R1 keeps its original + meaning unchanged — it governs the node *catalog* (a node exists only if + the Integration Registry has the integration), not the control-flow + *vocabulary*** — and must not be cited as a blocker on WS-11's 8.3c. + Recorded in `workflows_app.md` §8.3c and §11 R1. ## 4. Single-owner registry (who owns duplicated work) @@ -168,7 +204,11 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are | Budgets | **WS-16** (D2) | multiplayer §4.3/§5.3/Ph4 | | Digest workflows | **WS-15** (also scores workflows G1) | workflows_app §1.2 | | Orchestrator org-memory fix | **WS-15** (D4); structural fix WS-8 A1 | agent_architecture §11.1.2 | -| Workflow engine/editor | **workflows_app.md** (D6) | multi_agent_orchestration Ph2–3/§5.3 | +| Workflow engine/editor | **workflows_app.md** (D6) | multi_agent_orchestration Ph2–3/§5.3 · Ph5.1 (Magentic/GroupChat as graph node types — reassigned to WS-11, 2026-08-03) | +| Isolation ladder (BO-7 / HH-6 / T0–T2) | **`permissions_sandbox_b6.md`** (the Phase-5 build order `P5-a…d`; WS-3) | `agent_platform_hardening_2026-07.md` §1.2 — the ladder *definition* only, and the single T0/T1/T2 table of record · `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO-7 · `competitive_hardening_2026-07.md:119-141` (build log for the 2026-07-27 passes) | +| Context discipline / prompt budget | **WS-23** — `skills_registry.md` + `skills_scope_out.md` | multi_agent_orchestration Ph1 (struck 2026-08-03: 1.1 shipped, 1.2 moot, 1.3 delivered here) | +| Collaborative multi-agent chat (Shape C) | **WS-10** — shipped as multiplayer rooms (`docs/multiplayer/README.md`); the floor-control residue is OWNER-GATE | multi_agent_orchestration Ph5.2/§5.6 (struck 2026-08-03) | +| Calendar / Focus OS | **WS-21** — `calendar_focus_os.md` §5 canonical for `gtd_time_blocks`, §9 canonical for all F2/F3 acceptance; `calendar_timeboxing.md` §13 canonical for P4 external sync | The family has **four** docs, not two: `calendar_ai_review.md` and `calendar_ux_review.md` are **unregistered sub-docs**. `calendar_ai_review.md` is cited by three shipped migration headers (92 / 97 / 100) but by no board row and no spec index entry — *(focus_os §9.13 lists the third as 98; the file that cites it is 100)*. `calendar_ux_review.md` is the **sole** home of the block-reminders/notifications item (focus_os §9.13). **Horizons / Top-5 outcomes is DISPUTED between WS-21 (§4.7) and WS-18 — assigned here, to WS-21**; it is still DO-NOT-DISPATCH until it has acceptance (`calendar_focus_os.md` §9.10). | | Chat HITL model | **generative_ui_2.md §2** (shipped) | chat_ux §12.3 (superseded) | | Multiplayer prior art (`qm`, 2026-08-01) | **`multiplayer_prior_art_qm_2026-08.md` is reference-only** — it owns no work and no status; the specs it links stay authoritative | multiplayer README §4.6/§5.1/§6.4/§6.5 · memory-clearance §3.3/§7 · agent-kinds §9 Q1 · skills_scope_out §6 · WS-10 · WS-23 | | Memory compartments + clearance (incl. `subject:`) | **`docs/multiplayer/memory-clearance.md` §7** (surface design §7.1); dispatched as **WS-10 S1** | memory_architecture §9 `3a′` (link-only since 2026-08-02) · multiplayer README §6.3/§8 Phase 3 (index only) · prior-art §QM-D1 (reference only) | @@ -184,10 +224,25 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are > 2. `note_taker_app.md` §3.13's status-as-blockquote → proper table (cosmetic). > 3. `chat_ux.md` full archival decision (banner + supersession notes are in; > body retained as protocol reference for the still-open §12 VII–XI items). -> 4. `calendar_focus_os.md` "breaks in the packer" may have partially shipped -> (commit 80722e17, lunch-carve-out tests) — verify before dispatching F2. +> 4. ~~`calendar_focus_os.md` "breaks in the packer" may have partially +> shipped — verify before dispatching F2.~~ **CLOSED 2026-08-03.** It +> **shipped 2026-07-23** (`80722e17`, migration 97) as *packer geometry*: a +> widened buffer behind the block that trips `max_focus_run_mins`, plus an +> optional protected lunch window, applied to plan, replan, rollover **and** +> the nightly job. The nuance that keeps F2 alive: **the break is a gap, not +> a row** — nothing renders it, nothing can skip it, nothing counts it. The +> `kind='break'` row is §9.1 S4. > 5. ~~D9 (Pomad Centre) remains an owner call~~ — resolved 2026-08-01, all > 12 sites rewritten as "a second tenant deployment" (see D9). +> 6. **Spec-index and docstring staleness (new, 2026-08-03).** +> `ai-company-brain/AGENTS.md`'s per-feature spec index is missing rows: it +> has **no calendar row at all** (four calendar specs, none listed) and no +> `agent_architecture.md` entry. Separately, +> `ai-company-brain/AGENTS.md:190` and `apps/AGENTS.md:23` still carry the +> struck falsehood that the Action Broker *"ships with zero handlers and is +> not yet wired into the write path"* — untrue since 2026-07-13; see WS-1's +> five registration sites. Both are AGENT-SAFE doc fixes, neither is in this +> change. **Tier 1 — status truth (hours; AGENT-SAFE; do before any dispatch):** 1. `whatsapp_message_manager.md` — header "PLANNING, no code yet" → point at @@ -226,8 +281,11 @@ Resolutions for the cross-doc conflicts the audit surfaced. D1–D8 are addendum; mark §12.3 superseded by generative_ui_2 §2; archive the rest. 13. `note_taker_app.md` — convert the §3.13 blockquote to a status table; reconcile §3.4/D4/D5 with the D3 AssemblyAI decision + deferred Tier-B. -14. `agent_architecture.md` — one status for approve_all (§3.2 vs §11.3 vs - §12 A0); Phases F/G dependency split (3a partly shipped). +14. ~~`agent_architecture.md` — one status for approve_all (§3.2 vs §11.3 vs + §12 A0); Phases F/G dependency split (3a partly shipped).~~ **CLOSED + 2026-08-03 — both halves done:** A0 now carries one status (done + 2026-07-26, remaining scope named as the runtime/entrypoint check), and the + §12 phase table splits F/G onto the still-open half of multiplayer 3a. 15. `email_app_master_plan.md` — refresh §3 at-a-glance to include §3.14; archive `email_feature_review_2026-07.md` per its own §9 instruction. 16. `department_centers.md` — corrections shipped alongside this doc: Phase C @@ -261,8 +319,37 @@ dispatched inline, logged as `.queue.dropped`; see `FOUNDATION_BUILDOUT_CHECKLIST.md` §BO-20.0 (answered: Option A) and its Q1) · **WS-6 observability activation** (Langfuse keys + bringing up `--profile obs` in prod, `OTEL_EXPORTER_OTLP_ENDPOINT` in the deploy env, -`LLM_USAGE_AUDIT=1`, and re-enabling the MAF telemetry kill switch at -`executor.py:114` — it hides a known ContextVar-reset bug) · +`LLM_USAGE_AUDIT=1`, and re-enabling MAF telemetry by setting +`ENABLE_INSTRUMENTATION` — the kill switch is the env read at +**`executor.py:138`**, inside `_disable_agent_telemetry_once` (block +`:113-140`; the long-standing `:114` citation pointed into the comment banner +above it — corrected and re-verified 2026-08-03). It hides a known +ContextVar-reset bug that turns a successful streamed run into a `RUN_ERROR`) · +**`copilot_sandbox_scope`** (`packages/acb_common/acb_common/settings.py:222`, +ships `""` = fully off, in-process everywhere). Putting `code_task` or +`app_builder` in it routes **real Copilot sessions into containers** — a live +execution-path change, not a config tweak. It was registered nowhere until +2026-08-03 · +**`ISOLATION_TIER_ENFORCE`** — the new switch WS-3a introduces +(`permissions_sandbox_b6.md` §P5-a.2). Today every unscoped agent derives T2, +so flipping it **refuses most real runs**; it ships OFF and the refusal must be +behind it · +**WS-12 Phase 4.0's target choice** (minimal-bump vs full-bump, +`multi_agent_orchestration.md` §6 Phase 4.0) — a cost/schedule call, and the +reason WS-12 has **zero** dispatchable PRs. An agent may produce the 4.1 +evidence and must then stop and report · +**WS-12 Phase 4.6's manual soak** of the Copilot streaming path — an agent +cannot simulate or self-certify it, and must not mark 4.6 done without a +recorded human sign-off · +**Calendar external sync (WS-21)** — needs Google Calendar and/or Microsoft +Graph **OAuth client credentials (client id + secret + redirect URI) +provisioned on the VPS** and registered in the Integration Registry; +`calendar_timeboxing.md` §13 P4 clause 1 ("a `calendar_accounts` row created +through a real OAuth connect flow") is unverifiable without them · +**outbound nudge sending — one shared gate for two rows:** WS-21 §9.4's +Waiting-on chase block and WS-18's follow-up nudges both end in a real +outbound message from a real account. Drafting and queueing are AGENT-SAFE; +**sending is not**, and neither row may flip it independently · creating the bot Google account + real-meeting joins · Meta app review · real-account email sends / live-DB one-offs (`merge_ghost_messages --apply`) · **the WS-10 floor-control re-decision** — whether the five `chat_session.floor_mode`s, diff --git a/apps/services/action_broker/action_broker/broker.py b/apps/services/action_broker/action_broker/broker.py index 157b5cdf4..78085e4ad 100644 --- a/apps/services/action_broker/action_broker/broker.py +++ b/apps/services/action_broker/action_broker/broker.py @@ -14,13 +14,16 @@ registry. A real source-of-truth write happens ONLY inside a registered handler, and an action with no handler is REFUSED (never silently applied). -Ships with **zero** handlers registered, so it cannot perform any real write yet -— it is non-breaking and inert until handlers are wired in. Still pending -(needs per-agent authority decisions + a queue table): persisting -``needs_approval`` proposals to a ``pending_actions`` table (mirror -``pending_commit``), the Control Plane approval binding, and routing the existing -ClickUp/email writes through :func:`execute`. See FOUNDATION_BUILDOUT_CHECKLIST -BO-1. +This module itself registers nothing — handlers are wired in by the gateway at +startup / import (five sites as of 2026-08-03: ClickUp task writes, workflow +resume, WhatsApp broadcast, and two app-tool actions). It is therefore **live**, +not inert: ``pending_actions`` persistence, the Control Plane approval binding +(gateway ``routes/actions.py``) and the ClickUp task-write reroute all shipped +2026-07-13. Remaining per FOUNDATION_BUILDOUT_CHECKLIST §BO-1: two gated ClickUp +actions still have no handler (BO-1a), the queued-write ``sync_state`` marker is +ignored (BO-1b), and email writes do not route through here at all (BO-1c). +There is no Zoho write client anywhere in the repo, so the ``"zoho.email"`` +example below is illustrative, not a pointer to real code. """ from __future__ import annotations diff --git a/apps/services/gateway/gateway/routes/actions.py b/apps/services/gateway/gateway/routes/actions.py index 478388e33..5ff99fe22 100644 --- a/apps/services/gateway/gateway/routes/actions.py +++ b/apps/services/gateway/gateway/routes/actions.py @@ -13,10 +13,13 @@ outward-write bodies — CRM/email content — so it is never anonymous-reachable). Approve/reject go through ``action_broker.approve``/``reject``, which fail CLOSED: a missing or non-pending row is never run, and a handler error marks the -row ``failed``, not ``applied``. The broker ships with **zero** handlers, so -until real ones are registered, ``approve`` returns a refusal ("no handler") — -the queue is visible and auditable but still cannot perform any real write -(non-negotiable #4 stays intact). +row ``failed``, not ``applied``. Real handlers ARE registered (five sites as of +2026-08-03 — ClickUp task writes, workflow resume, WhatsApp broadcast, two +app-tool actions), so ``approve`` performs the real write for those actions. +An action with no registered handler still gets a refusal ("no handler") and +the row is marked ``failed`` — see FOUNDATION_BUILDOUT_CHECKLIST §BO-1a, where +two gated ClickUp actions (``delete_task``/``archive_task``) hit exactly that +branch today. """ from __future__ import annotations diff --git a/apps/services/gateway/gateway/routes/tasks/providers.py b/apps/services/gateway/gateway/routes/tasks/providers.py index 9e695d601..0f6d3eb66 100644 --- a/apps/services/gateway/gateway/routes/tasks/providers.py +++ b/apps/services/gateway/gateway/routes/tasks/providers.py @@ -97,9 +97,12 @@ def _broker_enforced(action: str) -> bool: so the broker only audits + chokepoints them. Set the env var to ``1``/``all``/``on`` to queue every write, or to a comma-list of action names to queue specific ones. This is the kill-switch — flip it without a redeploy - (env var + service restart). NOTE: the queue path needs a persistent handler - to execute on approval (a follow-up); until then, enforcing queues a write - but it won't run until that lands. + (env var + service restart). Persistent handlers ARE registered at startup + (``tasks/broker_handlers.py``), so an approved queued write really executes. + ⚠️ NOT for every gated action: this class gates SIX action names but only + four have handlers, so approving a queued ``clickup.delete_task`` or + ``clickup.archive_task`` is refused and the row is marked ``failed``. Do not + turn this on before FOUNDATION_BUILDOUT_CHECKLIST §BO-1a and §BO-1b land. """ import os diff --git a/packages/acb_skills/acb_skills/manifest.py b/packages/acb_skills/acb_skills/manifest.py index 26346fa37..3b238eefe 100644 --- a/packages/acb_skills/acb_skills/manifest.py +++ b/packages/acb_skills/acb_skills/manifest.py @@ -19,7 +19,11 @@ :meth:`isolation_tier`, :meth:`resolve_tool_surface`) are the values the platform should compute from the manifest instead of from hardcoded names. -**Nothing here is wired into the run path yet.** It is deliberately +**Partly wired: only** :meth:`AgentManifest.from_config` **and** +:meth:`AgentManifest.instance_key` **are on the run path** (``executor.py``'s +``_resolve_agent_instance`` and ``gateway/routes/workspace.py``'s +``_agent_instance_for``); every other derived accessor still has zero production +callers — see ``agent_architecture.md`` §12.1. It is deliberately side-effect-free so ``tests/unit/test_agent_manifest.py`` can assert that the derived values match what the platform does today — which is what makes the later flip a provable no-op rather than a hopeful one. From f482db17a8752d540718886b89ede8b11ee5bb30 Mon Sep 17 00:00:00 2001 From: Vijay Raghav Varada Date: Mon, 3 Aug 2026 19:13:54 +0530 Subject: [PATCH 2/2] docs(WS-0): tenancy and visibility become architecture of record MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two audits asked whether the multi-tenant foundation is complete. It is not built, and the owner's answer is that it should not be. This writes that down so the next cycle stops re-deriving it. organization_id sits on 3 of 111 tables and is read by zero authorization decisions; UserContext.organization_id is populated by an extra round-trip and never consulted. A second organization would not fail — it would silently serve the first org's data to the second, and that org's users would be permanently locked out because role seeding and the owner-bootstrap guard are both hardcoded to the default slug. D11: the tenant boundary is the DEPLOYMENT. One deployment per tenant, its own database and its own credentials. Row-level org isolation is explicitly not being built, organization_id stays a label rather than a mechanism, and most of the leak classes are moot by definition rather than by fix. Per-deployment credentials stop being a gap and become correct. A second tenant's real cost is priced honestly so the choice stays reversible on evidence rather than on memory. Three leaks still matter under that decision: org_group is joined on slug alone in three places, including the session-authority intersection. Cheap now, expensive later, and wrong within one org too the moment two Centers share a slug namespace. Written as a lettered ticket whose test must be proven red against a two-org fixture first. D12: the visibility model is private, then Center, then org, plus ad-hoc cross-Center groups by invite for projects that span teams. Each surface must declare its tier rather than inherit one by accident — two doctrines in one codebase is exactly what produced the Notes hole being fixed in the companion PR. Email, Tasks and Notes are private by default. And it answers the semantic that has blocked WS-14 for weeks: a project belongs to a team by an explicit group: grant. Not derived from its assignees, not an owning column. Both alternatives are named with the reason each was rejected. One handed-up claim was wrong and is corrected rather than propagated. I had said the subject vocabulary already generalises — that rooms and app_grants share email | group: | org. They do not. app_grants is email | agent: | agents:* and explicitly rejects the literal org; there is no group: case at all. The rooms docstring claiming the two are identical on purpose is false. Rooms is the only surface honouring group: today, which makes this a real if bounded job rather than a no-op, and the spec carries a per-surface gap table sized accordingly. FOUNDATION_BUILDOUT_CHECKLIST corrections, each re-measured rather than transcribed: BO-10's engine sprawl is 12 sites across 10 modules, not the "three+" documented — it grows by one per app and is the one cost that compounds. BO-13's extractions netted 84 lines against the original, and run_agent_stream is now larger than when its residual was written. BO-14 and BO-15 were reported as closed; both are half closed, and marking either done would have hidden a live gap — BO-14's real residual is that the confirmation path it defers to is called by no tool. BO-23 added: there is no application-level backup. The only dump script is schema-only, 140 migrations replay forward-only on every deploy under ON_ERROR_STOP=1, there are no down-migrations, and the one real safety net is a weekly whole-VPS image whose restore has never been exercised. Scripts and runbook are agent-safe; running them is not. Verdict recorded: yes, go app by app — with three exceptions, none of which is an app. Branch protection (verified absent twice over: protection 404 and rulesets empty), a tested restore path, and the DB engine seam. Also verified and recorded: Centers are currently unreachable by everyone including the owner. The feature tuple has no center.* entries while the frontend gates on exactly those slugs, so the whole nav section is dropped and the routes hit the access gate. Migration 140's own comment claims owners see all Centers via a wildcard baseline; they do not. Documentation only — no .py, .sql, .ts or deploy/ file is touched. Co-Authored-By: Claude Opus 5 --- FOUNDATION_BUILDOUT_CHECKLIST.md | 115 ++++- ai-company-brain/AGENTS.md | 3 +- .../specs/tenancy_and_visibility.md | 411 ++++++++++++++++++ ai-company-brain/work_plan.md | 74 +++- 4 files changed, 584 insertions(+), 19 deletions(-) create mode 100644 ai-company-brain/specs/tenancy_and_visibility.md diff --git a/FOUNDATION_BUILDOUT_CHECKLIST.md b/FOUNDATION_BUILDOUT_CHECKLIST.md index ccc74c699..4d6a45ae6 100644 --- a/FOUNDATION_BUILDOUT_CHECKLIST.md +++ b/FOUNDATION_BUILDOUT_CHECKLIST.md @@ -2,7 +2,8 @@ **Date:** 2026-07-11 · **Deploy status updated:** 2026-07-13 · **Competitive refs added:** 2026-07-13 **§BO‑20 rewritten and verified against code: 2026-08-02** (WS‑4 audit remediation). Verified this pass: the ingestion package contents (no `worker.py`), the ClickUp → `event_hooks.emit_event` → `workflows.triggers.dispatch_event` → `start_run` fan-out, the Gmail/Zoho `TODO` stubs, the repo-wide absence of `xreadgroup`/`xgroup`/`xack`, the four checked-in systemd units, the already-provisioned Redis compose service, `uv.lock`'s lack of any job-queue library, and the gateway lifespan's supervised loops (five wired, **all five** stopped on shutdown, four actually started in the default config — WhatsApp enrichment is flag-gated off). §BO‑20 now carries acceptance criteria, verification commands, gate labels, and one named owner decision (§BO‑20.0). **That decision was answered on 2026-08-02 — `BO‑20 = Option A (in‑process)`** — so nothing in §BO‑20 is blocked on a decision any more. **BO‑20f and BO‑20a are BUILT (both 2026-08-02) and BO‑20b slice 1 (`emit_event` strict mode) 2026-08-03, moving §BO‑20 ☐ → ◑; BO‑20b slice 2 and BO‑20c–e are open and dispatchable.** ⚠️ **Slice 1 is necessary but NOT sufficient** (adversarial review 2026-08-03): the only sink production registers, `workflows.triggers.dispatch_event`, swallows every exception, so `raise_on_error=True` is a no‑op on the real registry — **slice 2's scope grew** to include a matching strict path in `dispatch_event`, and the §BO‑20 non‑goal "Not a change to `dispatch_event`" is struck and qualified. Two claims in the stamp above were made false by BO‑20a and are corrected in place below: `xreadgroup`/`xgroup`/`xack` now exist (in `ingestion/consumer.py` only), and the lifespan's supervised loops are **six** wired / **all six** stopped on shutdown (how many actually *start* is data‑dependent — see §BO‑20 "What is true today" §7, which counts it honestly; the new ingestion consumer joins WhatsApp enrichment as flag‑gated **off**, so it starts nowhere today). `INGESTION_CONSUMER` is registered in `work_plan.md` §6. **Other sections carry no such stamp** — BO‑1/BO‑19 were stamped by the 2026-08-01 doc-truth pass; the rest are as-authored. -**Companion to:** `FOUNDATION_AUDIT_REPORT.md` · handoff details in `FOUNDATION_CONTINUATION.md` (see its "LATEST STATUS" block) · competitive learnings (proven reference implementations from Hermes Agent & OpenClaw) in `ai-company-brain/specs/competitive_hardening_2026-07.md` (`CH-*`) and `COMPETITIVE_COMPARISON.md`. +**§BO‑10 / §BO‑13 / §BO‑14 / §BO‑15 / §BO‑19 re-measured and corrected, §BO‑23 added, and the "can we go app by app?" verdict block added: 2026-08-03** (WS‑0 truth pass, second commit on PR #344). Verified this pass by measuring, not by reading the prior doc: the **12** `create_async_engine` call sites across 10 modules and the fact that the 8 cached singletons are never disposed (BO‑10 said "three+"); `executor.py` at **5,010** lines and `run_agent_stream` at **~1,942** (BO‑13 claimed 4,069 / ~1,600 — both July numbers the file has since grown past); `permission_policy.decide`'s two hard‑veto return paths and `install_dependency`'s `destructive: True` annotation (BO‑14 claimed the gate "can never deny" and the registry was "empty" — **both false**); `model_limits.py` as the retired‑five‑sources context‑window SoT versus `_TIER_DEFAULTS` + three still‑on‑disk config files (BO‑15 is **half** closed, not closed and not untouched); the root `AGENTS.md`'s deleted version table and `infra/AGENTS.md`'s corrected proxy/Langfuse lines (BO‑19 → ✅); `dump_schema.sh`'s `--schema-only`, `apply_migrations.sh`'s 140‑file `ON_ERROR_STOP=1` replay, the absence of any WAL/pgbackrest/wal‑g config, and `deploy/hostinger/README.md:115` (→ **BO‑23**); and, live against GitHub, `branches/main/protection` → 404 with `rulesets` → `[]`. **Two audit claims handed to this pass were wrong and are not transcribed:** BO‑14/BO‑15 were described as fully closed (BO‑14's defects are closed but its *residual* is real and different; BO‑15 is half). Tenancy/visibility architecture from the same day lives in `ai-company-brain/specs/tenancy_and_visibility.md`, not here. +**Companion to:** `FOUNDATION_AUDIT_REPORT.md` · handoff details in `FOUNDATION_CONTINUATION.md` (see its "LATEST STATUS" block) · competitive learnings (proven reference implementations from Hermes Agent & OpenClaw) in `ai-company-brain/specs/competitive_hardening_2026-07.md` (`CH-*`) and `COMPETITIVE_COMPARISON.md` · tenancy + visibility architecture of record in `ai-company-brain/specs/tenancy_and_visibility.md`. > **🚀 Deploy status:** read live deploy state from `gh run list` and `git log origin/main` — not from this doc. Next recommended P0: **BO‑8** (secret rotation + history purge — owner‑gated); BO‑1's approval loop has since shipped (see §BO‑1). > **Update 2026-08-01 (doc-truth pass):** the previous pinned‑commit claim here (`origin/main = ccccdc8`, unpushed `1684e1a`) was a 2026‑07‑13 snapshot and went stale; this doc no longer tracks deploy state. @@ -15,6 +16,32 @@ This is the list of foundational capabilities that are **missing, partially impl --- +## Verdict — can we go app by app? *(2026‑08‑03)* + +**Yes, with three exceptions.** The owner asked whether the foundation is complete +enough to stop doing platform work and start doing app work. It is: auth is +default‑deny and enforced by construction (BO‑2 ✅), the Action Broker is a live +audited chokepoint (BO‑1 ◑ with three lettered tickets), the runtime story is +settled (BO‑12 ✅), the permission gate denies the two things that matter +(BO‑14, corrected below), and the event intake substrate is half built and +sequenced (BO‑20 ◑). None of that blocks the next app. + +**Three items are exceptions.** They are not app work, they do not improve by +being deferred behind app work, and **one of them gets worse with every app +added**: + +| # | Exception | Item | The one‑line reason | +|---|---|---|---| +| 1 | **`main` has no branch protection** | §BO‑17 / `work_plan.md` WS‑5 | Verified live 2026‑08‑03: branch protection → **404**, rulesets → **`[]`**. Every "blocking" gate in the workflow YAMLs is therefore decorative, and every app shipped from here inherits that. **OWNER‑GATE** — a GitHub settings change no agent can make. | +| 2 | **No backup / restore path** | **§BO‑23** (new, below) | Schema‑only dump, no data dump, no restore, no PITR — while 140 migrations replay forward‑only on every deploy under `ON_ERROR_STOP=1` with no down‑migrations. Largest uncovered risk; scales with app count. Scripts + runbook are AGENT‑SAFE, **execution is OWNER‑GATE**. | +| 3 | **DB engine sprawl** | §BO‑10 | **12 `create_async_engine` call sites across 10 modules** (+ one sync engine), 8 of them undisposed process‑lifetime singletons. One arrived per app. **This is the only item whose cost compounds per app** — fix the seam before the next app, not after. | + +Nothing else on this list needs to be closed first. Items 1 and 2 are risk +containment the owner must action; item 3 is the one an agent should fix before +the next app opens engine number 13. + +--- + ## A. Security & trust boundaries ### BO‑1 — Action Broker: real approval‑gated write path *(P0)* ◑ @@ -152,8 +179,23 @@ was stale — corrected 2026-08-02 to match §BO‑20) were anonymous‑reachabl ### BO‑10 — Consolidate DB access to one engine/pool *(P2)* ◑ - **Done (Session 2, 2026‑07‑13):** **every** engine now bounds the CONNECT phase so a slow/unreachable DB can't hang callers — `settings.db_connect_timeout` (default 10s) on `acb_graph.get_engine()` (`ccccdc8`, live in prod), the two gateway asyncpg engines (`1684e1a`), and the four `email_ingestion` async engines (`1ff6c0d`, local, unpushed) via `connect_args={"timeout": …}`. This makes `acb_audit.record()`'s "never block the caller" guarantee real against a hung connect. Test: `tests/unit/test_db_connect_timeout.py`. -- **Missing:** still three+ engines (`acb_graph/db.py`, `routes/tasks/core.py`, `routes/email/core.py`, plus per‑call engines in `email_ingestion/{scheduler,inbound}.py` that also leak — BO‑9), the foundational one otherwise unconfigured; sync `acb_audit.record()` still blocks the async loop (H11) — connect_timeout bounds the hang but the call is still synchronous. -- **Approach:** Provide a single configured async engine in `acb_graph` (sized pool), funnel all callers through it, and make `acb_audit.record()` async (or always call via `to_thread`). +- **Missing — the "three+" above was written in July and is now materially wrong; re‑measured 2026‑08‑03.** It is **12 `create_async_engine(...)` call sites across 10 modules**, plus a 13th **sync** `create_engine` in `acb_graph/db.py:32`: + | Module | Sites | Shape | + |---|---|---| + | `packages/acb_auth/acb_auth/access.py:69` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/admin/_common.py:62` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/apps/_common.py:89` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/email/core.py:404` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/notes/core.py:160` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/tasks/core.py:161` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/whatsapp/core.py:135` | 1 | cached `_ENGINE` singleton | + | `gateway/routes/workflows/core.py:64` | 1 | cached `_ENGINE` singleton | + | `email_ingestion/inbound.py:273` | 1 | per‑call, disposed at `:288` | + | `email_ingestion/scheduler.py:142, 527, 560` | 3 | per‑call, disposed at `:424`/`:540`/`:590` | + | `packages/acb_graph/acb_graph/db.py:32` | (1 sync) | `create_engine`, a different flavour again | + **The eight cached singletons are never disposed** — repo‑wide, the only `engine.dispose()` calls are the four `email_ingestion` per‑call engines cleaning up after themselves, and nothing in the gateway lifespan disposes anything (BO‑9). Also still open: sync `acb_audit.record()` blocks the async loop (H11) — connect_timeout bounds the hang but the call is still synchronous. +- **Why it moved up the list:** the count grew by *one engine per app* — `notes`, `whatsapp`, `workflows` and `apps` all arrived with their own. This is the only foundation item whose cost **compounds per app**, so it is the one to fix before the next app rather than after (see `work_plan.md` §2's "Can we go app by app?" block, exception 3). +- **Approach:** Provide a single configured async engine in `acb_graph` (sized pool), funnel all callers through it, dispose it in the gateway lifespan, and make `acb_audit.record()` async (or always call via `to_thread`). ### BO‑11 — Decide `acb_schemas`: wire in or delete *(P2)* ✅ - **Done:** deleted the package (0 production importers, drifted from the ORM — H10). Removed its 7 `pyproject` dependency declarations + `tool.uv.sources` entry, the smoke‑test import, and the stale "wire/API surface" comment in `acb_graph/models.py`; re‑locked. Bonus: this exposed a latent under‑declared dependency — `orchestrator/triage/schema.py` uses pydantic `EmailStr` (needs `email‑validator`) but only got it transitively via `acb_schemas`; now declared explicitly as `pydantic[email]` on the orchestrator. @@ -174,6 +216,37 @@ was stale — corrected 2026-08-02 to match §BO‑20) were anonymous‑reachabl - **Note:** until this lands, new apps needing search should copy the Workflows stance — deterministic keyword ranking, no private embedding stacks. +### BO‑23 — Backup, restore, and point‑in‑time recovery *(P0)* ☐ *(new — 2026‑08‑03, WS‑0 truth pass; verified against the tree)* + +> **This is the largest uncovered risk on the platform, and it scales with app count.** Every app that ships adds tables whose only copy is one Postgres volume on one VPS. It is filed P0 rather than P2 because unlike every other item here, the failure mode is *unrecoverable* — there is nothing to fix afterwards. + +**What is true today (each claim measured 2026‑08‑03, not inherited):** + +1. **There is no data backup.** The only Postgres dump script in the repo is `scripts/dump_schema.sh`, and it runs `pg_dump --schema-only --no-owner --no-privileges` (`:52`) writing `infra/postgres/schema.generated.sql`. That is **structure with zero rows** — it exists so humans and agents can read the current schema shape in one file, and it is explicitly *not* a backup. +2. **There is no restore path.** Repo‑wide there is no `pg_restore`, no logical data dump, no `--data-only`, and no restore runbook. A grep for `pg_dump|pg_restore|pgbackrest|wal-g|barman` across `*.sh|*.yml|*.yaml|*.py` returns exactly two files: `scripts/apply_migrations.sh` and `scripts/dump_schema.sh`. +3. **There is no PITR.** `archive_mode`, `wal_level` and `archive_command` appear nowhere in `infra/` or `deploy/`; the Postgres compose service (`infra/docker-compose.yml:34-37`) mounts one named volume, `acb-postgres-data`, with default settings. +4. **The only backup that exists is outside the repo and outside our control.** `deploy/hostinger/README.md:115`: *"Hostinger takes weekly backups of the whole VPS automatically (included in plan). For Postgres‑level point‑in‑time recovery later, add `pgbackrest` or a `pg_dump` cron job."* Honest, but it means the worst case is **up to seven days of data loss**, from an image whose restore has never been exercised. +5. **Migrations auto‑apply on every deploy, forward‑only.** `scripts/apply_migrations.sh` replays every numbered file from `02_` upward, in `sort -V` order, through `psql -v ON_ERROR_STOP=1` (`:59-74`), exiting non‑zero on the first failure. **140** files are replayed today (**142** numbered files on disk; `00_`/`01_` are initdb‑only). There is no ledger, no down‑migration, and no dry run. A migration that is idempotent in intent but not in fact takes the deploy down mid‑ladder with the database in a partially‑migrated state — and item 1 means there is nothing to roll back to. +6. **Redis has no persistence configured either.** `infra/` sets no `appendonly`, which `work_plan.md`'s WS‑4 row already records from the other direction: BO‑20b's retry counter *"survives a gateway restart but not a Redis one"*. + +**Done when:** + +1. `scripts/backup_db.sh` exists: a **data‑inclusive** `pg_dump -Fc` of the application database to a timestamped file, with the same `.env`/container‑name resolution shape as `dump_schema.sh` and `apply_migrations.sh` (so the three are operationally consistent), plus a retention sweep and a non‑zero exit on any failure. +2. `scripts/restore_db.sh` exists and is the **documented inverse** — `pg_restore` into a named database, refusing by default to target the live one without an explicit `--force`‑style flag. +3. A runbook (`deploy/hostinger/RESTORE.md` or a section in that README) states, in order: how to take an ad‑hoc backup before a risky deploy, how to restore into a scratch database, how to verify the restore (a row‑count or checksum assertion against a known table), and how to cut over. **A backup nobody has restored is not a backup** — the runbook must contain the verification step, not just the commands. +4. A pre‑migration hook: `apply_migrations.sh` (or the deploy step that calls it) takes a backup **before** replaying the ladder, or refuses to run without one. This is the clause that makes items 1 and 5 above stop compounding. +5. The `deploy/hostinger/README.md:115` "add it later" note is replaced by a pointer to what now exists. +6. Whatever ships is reflected in `infra/AGENTS.md` and `deploy/AGENTS.md` (DOX pass). + +**Gate labels — read this before dispatching:** + +- **AGENT‑SAFE:** writing the two scripts, the runbook, and the doc updates. These are files in the repo. +- **OWNER‑GATE:** *executing* any of it. Running a backup, running a restore, configuring WAL archiving, provisioning off‑box storage, and changing the deploy pipeline all reach the VPS and the production database — `work_plan.md` §6, and the `plan-guard.mjs` hook enforces it independently. An agent must write the tooling, verify it only by reading it, and hand execution to the owner. ⚠️ **Also verify the hook's posture on the paths you intend to write** before promising a PR: `plan-guard` blocks agent *commands* that reach the VPS, and an implementer should confirm rather than assume that authoring a new `scripts/*_db.sh` is permitted. +- **Explicit non‑goal:** do not "test" the restore against the live database. The scratch‑database path in done‑when 3 is the whole point. + +**Dependencies:** none in code. It does *not* wait on BO‑6 (Alembic) — a backup is useful under raw numbered migrations and more useful, not less, because of them. + + --- ## D. Orchestration & runtime @@ -183,7 +256,9 @@ was stale — corrected 2026-08-02 to match §BO‑20) were anonymous‑reachabl - **Competitive ref (CH‑5):** Hermes's multi‑agent layer (orchestrator + isolated sub‑agents exchanging **typed result objects**, resource‑aware concurrency limits, Kanban dispatch) is more built‑out than ours on coordination mechanics — the reference when we finally instantiate the Workflow engine and replace bare‑string sub‑agent handoffs (ties to HH‑7). See `specs/competitive_hardening_2026-07.md`. ### BO‑13 — Break up the executor monolith *(P2)* ◑ -- **Done this pass (behaviour‑preserving extractions, each verified green):** the 5,094‑line file is down to **4,069 lines** via four cohesive‑concern extractions, each re‑exported from `executor` so no importer changed: +> ⚠️ **The line counts below are stale in the optimistic direction — re‑measured 2026‑08‑03 (WS‑0 truth pass).** `executor.py` is **5,010 lines** (`wc -l`), not 4,069, and `run_agent_stream` is **~1,942 lines** (`:2139` to the next top‑level `def` at `:4081`), not ~1,600. The four extractions below did happen and did work; the file has since **grown back past its pre‑extraction size minus 84 lines** because features kept landing in it. Read the July numbers as a record of what the extractions removed, not as the current state. **The honest headline: the extractions netted ‑84 lines against the original 5,094, and the residual function is larger than when the residual was written.** + +- **Done this pass (behaviour‑preserving extractions, each verified green):** the 5,094‑line file was taken down to **4,069 lines** *at the time of that pass* via four cohesive‑concern extractions, each re‑exported from `executor` so no importer changed: - `orchestrator/_todo_tracker.py` — todo‑SQL parsing. - `orchestrator/_copilot_session.py` — Copilot permission handler + infinite‑session policy. - `orchestrator/_tool_injection.py` — platform tool injection + system‑prompt addendum (~630 lines, the biggest cohesive concern). @@ -192,12 +267,15 @@ was stale — corrected 2026-08-02 to match §BO‑20) were anonymous‑reachabl - **Tier‑2 batch:** envelope contract (`RUN_STARTED` first → text streamed → `RUN_FINISHED` terminal), run_id/thread_id propagation, agent‑exception → `RUN_ERROR` (not a crash). - **Tier‑1 native streaming:** a mock agent that yields MAF‑shaped `run(..., stream=True)` updates → asserts the `TEXT_MESSAGE_START/CONTENT/END` lifecycle and `TOOL_CALL_START/ARGS/RESULT` events (via the real event_translator). - **HITL parking (new this pass):** `resolve_user_input` (found / not‑found) and the full `_make_user_input_handler` round‑trip — emits the `user_input_requested` frame to the relay, parks a Future, and returns the answer once `resolve_user_input` fires. Locks the ask_user → prompt → resolve contract. -- **Residual:** the Tier‑1.5 Copilot‑SDK tier and the idle‑timeout / fall‑through control‑flow branches are still not covered (the Copilot/full‑stream branches can't be exercised on the Windows dev box — they hit the same multi‑point infra hang that deselects this file locally — so they need a Linux/CI‑run harness to add safely); and `run_agent_stream` is still one ~1,600‑line function. +- **Residual:** the Tier‑1.5 Copilot‑SDK tier and the idle‑timeout / fall‑through control‑flow branches are still not covered (the Copilot/full‑stream branches can't be exercised on the Windows dev box — they hit the same multi‑point infra hang that deselects this file locally — so they need a Linux/CI‑run harness to add safely); and `run_agent_stream` is still one **~1,942‑line** function (`executor.py:2139-4080`, measured 2026‑08‑03 — the "~1,600" this bullet used to claim was a July number that has since grown). - **Approach for the residual:** (1) extend the harness to the Copilot tier + HITL/idle branches. (2) THEN extract the native / Copilot / batch tiers behind a `Runtime` strategy interface — the `return`‑to‑end vs fall‑through‑to‑batch control flow is the delicate part, so it needs those branches covered first — and move HITL/session‑store/cleanup into collaborators, guarded by this net + the trajectory evals. (3) Ratchet the xenon absolute ceiling down from F. ### BO‑14 — Enforce the permission/risk model *(P1)* ◑ - **Done this pass:** **workspace‑path containment** shipped — `write_artifact`/`save_note`/`recall_notes` routed every caller path through a single `write_artifact.resolve_in_workspace` guard that fails closed on an embedded `..` or an absolute path resolving outside the workspace (previously `write_artifact` could write, and `recall_notes` could READ, arbitrary files). Also fixed a latent bug: `recall_notes` now applies the same `agent-data/` prefixing as `save_note`, so the documented `recall_notes("NOTES.md")` round‑trip actually works. 7 unit tests added. -- **Missing (the enforcement redesign):** the injected‑tool gate still can never deny (M5) and the destructive platform registry is empty. This is deliberately deferred — `decide()` currently *defers* destructive tools (approves, relying on each tool's own `request_confirmation`), so forcing denials risks false‑blocking legitimate tool use across every agent; it needs a product decision on which tools hard‑block + the confirmation UX. +- ~~**Missing:** the injected‑tool gate still can never deny (M5) and the destructive platform registry is empty.~~ **Both halves struck 2026‑08‑03 (WS‑0 truth pass) — verified false against the code:** + - **The gate CAN deny.** `acb_skills/permission_policy.py::decide` (`:127`) returns `False` on two hard vetoes that run **before** the annotation lookup: `("shell_denied", …)` when the command text matches a deny pattern (`:166-169`) and `("write_out_of_workspace", …)` when a write resolves outside the agent workspace (`:171-176`). Both fail closed and a tool's own annotation cannot waive either. + - **The registry is not empty.** `acb_skills/tool_annotations.TOOL_ANNOTATIONS` carries `install_dependency` with `"destructive": True`, with the rationale in place (it installs into the *shared* gateway venv). +- **The real residual, stated accurately.** `decide()` still **approves** annotated‑destructive tools — the `tool_destructive_defer` branch at `:193-197` returns `True` on purpose, deferring to each tool's own `request_confirmation`. The gap is that, per the annotation registry's own comment on `install_dependency`, **no tool in this codebase yet calls `request_confirmation` on its own behalf before running**, so "defer to the tool's confirmation" defers to a card that never fires. That is BO‑14's job. It stays deferred deliberately: forcing denials risks false‑blocking legitimate tool use across every agent, and it needs a product decision on which tools hard‑block plus the confirmation UX. **Do not read "the gate can never deny" anywhere; it is wrong. Read: the gate denies two things and defers the third.** - **Approach for the residual:** annotate the genuinely destructive platform tools (`install_dependency`, outward‑write tools) as `destructive`, pass full call context (not just the name) to `decide`, and make `enforce` mode block destructive/out‑of‑policy calls with a real confirmation card. - **Competitive ref (CH‑1):** Hermes ships an always‑on **hardline blocklist** (`rm -rf /`, fork bombs, `mkfs`, disk‑zeroing `dd`) that no mode can override, plus **fail‑closed timeout→deny** on the approval prompt — both worth adopting as the floor. NVIDIA **NemoClaw**'s key idea for OpenClaw is **out‑of‑process policy enforcement**: evaluate the gate *outside* the agent's own tool surface so a prompt‑injected agent can't route around it. See `specs/competitive_hardening_2026-07.md`. @@ -1528,9 +1606,12 @@ non‑blocking style backlog. ## E. LLM configuration ### BO‑15 — Single source of truth for tier→model + context windows *(P1)* ◑ +> **Split verdict, verified against code 2026‑08‑03 (WS‑0 truth pass).** This item bundled two problems and they are now in different states, so "BO‑15's defects are closed" and "BO‑15 is untouched" are **both wrong**. The **context‑window** half is done; the **tier→model** half is not. The ◑ is honest; what follows says which half is which. + - **Done this pass:** the two hand‑synced tier‑alias maps are collapsed — `v1_compat` now imports `acb_llm.client._TIER_ALIAS_MAP` (the map `context.py` and the tests already use) instead of duplicating it. -- **Missing:** the tier→**model** mapping still has four disagreeing definitions (M3: `client._TIER_DEFAULTS`, `config.yaml`, `tier_overrides.yaml`, `settings.py` comment); `_TIER_CONTEXT_WINDOWS` a stale second copy of what `context.py` computes. -- **Approach:** Make the DB `model_config` table authoritative; delete `tier_overrides.yaml`, `enabled_models.json`, and the proxy directives in `config.yaml` once seeded; have `settings.py` read windows from `context.py`'s dynamic resolver instead of a hardcoded map. +- ~~**Missing:** `_TIER_CONTEXT_WINDOWS` a stale second copy of what `context.py` computes.~~ **✅ CLOSED — struck 2026‑08‑03.** `packages/acb_llm/acb_llm/model_limits.py` is now the single source of truth for "how big is this model?" (`get_limits()`), and its module docstring enumerates the five disagreeing sources it retired — including `_TIER_CONTEXT_WINDOWS` "duplicated verbatim in two packages". `settings.py:1494` is now `_TIER_CONTEXT_WINDOWS = FALLBACK_CONTEXT_WINDOWS` — an alias with **the tier aliases deliberately absent** so the dynamic resolution stands (the comment at `:1485-1493` records the bug this fixed: the stale pin was applied *after* dynamic resolution and overwrote it, under‑reporting the UI's context ring by ~7.6×). +- **Still missing — the tier→model half (M3), re‑verified on disk 2026‑08‑03:** `acb_llm/client.py:37` still defines `_TIER_DEFAULTS` as a hardcoded fallback map, populated at import time from `config.yaml` + `tier_overrides.yaml`; **all three of the files this item wanted deleted still exist** — `infra/litellm/tier_overrides.yaml`, `infra/enabled_models.json`, `infra/provider_models_cache.json` — and the DB `model_config` table is not authoritative over them. This half is unchanged since July. +- **Approach (for the remaining half only):** make the DB `model_config` table authoritative; delete `tier_overrides.yaml`, `enabled_models.json`, `provider_models_cache.json`, and the proxy directives in `config.yaml` once seeded. **Do not** re‑open the context‑window work — route every "how big is this model" question through `model_limits.get_limits()`. ### BO‑16 — Retire the vestigial LiteLLM proxy config *(P3)* ☐ - **Missing:** `infra/litellm/config.yaml` is a full proxy config but no proxy runs; only its tier rows are read (M6). `provider_models_cache.json` is a rotting committed cache. @@ -1552,18 +1633,22 @@ non‑blocking style backlog. ## G. Documentation -### BO‑19 — Doc↔code reconciliation *(P1)* ◑ -- **Missing:** README described LangGraph/Theia/PostgresSaver/escalation_ui and had a garbled layout (**✅ F3** rewrites it); stale "placeholder"/LangGraph docstrings across packages (**✅ F6** sweeps the worst); `AGENTS.md` version pins lag. -- **Done this pass:** `AGENTS.md` Python‑version mismatch fixed — "Python 3.11+" → "3.12+" to match `pyproject` (`>=3.12,<3.14`) and CI/prod (3.12). -- **Residual:** update `AGENTS.md` package versions to the lockfile (`agent-framework-core 1.8.1`) and update `infra/AGENTS.md`'s "no proxy files / no Langfuse" claims to match reality. *(The 3.11/3.12 mismatch is fixed — see "Done this pass" above; duplicate residual entry removed 2026-08-01, doc-truth pass.)* +### BO‑19 — Doc↔code reconciliation *(P1)* ✅ + +> **Closed 2026‑08‑03 (WS‑0 truth pass).** Both residuals were re‑checked against the files, not against the previous doc, and **both are done**. Marking it ✅ does *not* claim the corpus is drift‑free — it claims this item's two named residuals are closed. Ongoing doc truth is `work_plan.md` §5's remediation backlog, which is where new drift belongs. + +- **Missing (historical):** README described LangGraph/Theia/PostgresSaver/escalation_ui and had a garbled layout (**✅ F3** rewrites it); stale "placeholder"/LangGraph docstrings across packages (**✅ F6** sweeps the worst); `AGENTS.md` version pins lag. +- **Done (earlier pass):** `AGENTS.md` Python‑version mismatch fixed — "Python 3.11+" → "3.12+" to match `pyproject` (`>=3.12,<3.14`) and CI/prod (3.12). +- **Residual 1 — `AGENTS.md` package pins → ✅ closed, and better than asked.** The ask was "update the pins to the lockfile". The root `AGENTS.md` **deleted the hand‑copied table instead**, replacing it with *"`uv.lock` is the single source of truth for pinned versions — do not maintain a hand‑copied table here (it drifts: the previous snapshot was stale on 3 of 6 pins)"* plus `uv tree` / `uv pip list` as the check. A table that cannot drift beats a table that is currently accurate. +- **Residual 2 — `infra/AGENTS.md`'s "no proxy files / no Langfuse" claims → ✅ closed.** It now reads *"The legacy proxy files `litellm/config.yaml` + `litellm/tier_overrides.yaml` are **still on disk but vestigial** — only their tier rows are read; retiring them is tracked as BO‑16"* (`:4`) and *"Langfuse container is defined but **opt‑in behind `--profile obs`** and dormant … Distributed tracing is tracked as BO‑5"* (`:18`). Both match the tree: the two YAMLs exist, and Langfuse is a `--profile obs` compose service with the Python package uninstalled. --- ## Suggested sequencing -1. **P0 hardening sprint (do first):** BO‑8 (rotate+purge secrets), BO‑2 (auth enforcement), BO‑1 (Action Broker), BO‑3 (mutation governance). These close the Critical trust‑boundary and governance gaps that everything else sits on. -2. **P1 sprint:** BO‑7 (sandbox), BO‑5 (observability+cost), BO‑6 (migrations), BO‑12/BO‑14 (runtime + permission model), BO‑15 (LLM config SoT), BO‑17/BO‑18 (gates), BO‑19 residual, **BO‑20 (event‑bus consumer + job queue)**. -3. **P2/P3:** BO‑9, BO‑10, BO‑11, BO‑13, BO‑16, **BO‑21 (memory activation)**. +1. **P0 hardening sprint (do first):** **BO‑23 (backup/restore — scripts + runbook are AGENT‑SAFE; it is P0 because it is the only unrecoverable failure mode here)**, BO‑8 (rotate+purge secrets), BO‑2 (auth enforcement — ✅ since), BO‑1 (Action Broker), BO‑3 (mutation governance). These close the Critical trust‑boundary and governance gaps that everything else sits on. +2. **P1 sprint:** BO‑7 (sandbox), BO‑5 (observability+cost), BO‑6 (migrations), BO‑12/BO‑14 (runtime + permission model), BO‑15 (LLM config SoT — **tier→model half only**), BO‑17/BO‑18 (gates), **BO‑20 (event‑bus consumer + job queue)**. *(BO‑19 closed 2026‑08‑03.)* +3. **P2/P3:** BO‑9, **BO‑10 (promoted in practice — it is the one item that compounds per app; see the verdict block at the top)**, BO‑11, BO‑13, BO‑16, **BO‑21 (memory activation)**. **Competitive‑informed items** (proven reference implementations from Hermes Agent / OpenClaw — full mapping in `ai-company-brain/specs/competitive_hardening_2026-07.md`): CH‑1→BO‑7/BO‑14, CH‑2→BO‑1, CH‑3→BO‑20, CH‑4→WBS 3.3, CH‑5→BO‑12, CH‑6→BO‑21, CH‑7→Phase‑5 Annealer, CH‑8→BO‑5. These do not change the sequencing above — they attach a "what good looks like" reference to items we already have, plus the two new items (BO‑20/BO‑21) the comparison surfaced. diff --git a/ai-company-brain/AGENTS.md b/ai-company-brain/AGENTS.md index 4cbe52d20..3dc946d7b 100644 --- a/ai-company-brain/AGENTS.md +++ b/ai-company-brain/AGENTS.md @@ -114,8 +114,9 @@ to it and to `competitive_hardening_2026-07.md` (CH-*) rather than re-describe t | [`task_manager_harness_2026-07.md`](specs/task_manager_harness_2026-07.md) | Task-manager × harness engineering (app-layer sibling) | 🔄 Tier 1 shipped (2026-07-03); Tier 2 planned | | [`llm_caching_memory.md`](specs/llm_caching_memory.md) | Prompt caching (ADR-008) + session memory | 🔄 caching **shipped & wired**; session-memory shipped but **inert by default** (→ BO-21); Phase 7 open | | [`mcp_plugin_integration.md`](specs/mcp_plugin_integration.md) | MCP servers vs Claude plugins vs REST | 🔄 MCP half **built** (`_inject_mcp_servers`); plugin store not started | +| [`tenancy_and_visibility.md`](specs/tenancy_and_visibility.md) | **Tenancy + visibility — the architecture of record for who can see what.** Records two owner calls of 2026-08-03: (a) **the tenant boundary is THE DEPLOYMENT** — one deployment per tenant, row-level org isolation explicitly NOT built, `organization_id` stays a label not a mechanism; (b) the **visibility model** private → Center → org plus ad-hoc cross-Center groups by invite, mapped onto the shipped `email \| group: \| org` subject vocabulary. Also answers "what makes a project a team's project" (an explicit `group:` grant), carries the per-surface gap table for going app by app, and names what is out of scope (row-level tenancy, org switcher, multi-org users). **Read before designing any sharing or scoping on a new surface** | 🟢 architecture of record (owner-answered 2026-08-03) | | [`org_access_control.md`](specs/org_access_control.md) | **Org access control / multi-tenant user management** — members, roles, per-user allow/deny overrides, feature + agent gating, per-member integration credentials, default-deny auth. **§10 is the handoff contract for the multiplayer agent collaboration workstream — read it before designing multiplayer** | 🟢 Phase 1 shipped; remaining scope (modules/teams, session sharing, transcript + memory exposure) **transferred to multiplayer collaboration** | -| [`multi_user_organization_research.md`](specs/multi_user_organization_research.md) | Multi-user / org account research — identity, roles, tenancy, memory + credential scoping, SaaS multi-tenancy | 🔄 §4 identity/roles now implemented via `org_access_control.md`; §5 modules, §7 memory scoping, §8 credential scoping, §9 entity-graph RLS and §17 SaaS remain research | +| [`multi_user_organization_research.md`](specs/multi_user_organization_research.md) | Multi-user / org account research — identity, roles, tenancy, memory + credential scoping, SaaS multi-tenancy | 🔄 §4 identity/roles now implemented via `org_access_control.md`; §5 modules, §7 memory scoping, §8 credential scoping remain research. ⚠️ **§9 entity-graph RLS and §17 SaaS tenancy are SUPERSEDED for planning purposes by `tenancy_and_visibility.md` §1 + §6** (2026-08-03 owner call: one deployment per tenant; row-level org isolation explicitly not built). Read them as research into a path that was considered and declined, not as queued work | | [`chat_agent_framework_review_2026-07.md`](specs/chat_agent_framework_review_2026-07.md) | **Chat + agent framework review** — dual-runtime verdict (MAF framework, Copilot as coding engine), orchestration/memory/artifact/HITL/co-authoring gaps, prioritized plan | 🟢 review complete | | [`single_agent_chat_bug_audit_2026-07.md`](specs/single_agent_chat_bug_audit_2026-07.md) | **Single-agent chat bug audit** — past issues verified fixed; 7 confirmed live bugs (Tier-1 loop-trip crash, Copilot retry duplication, unbounded resumed-session context, relay truncation/ack holes) + fix plan | 🟢 audit complete | | [`generative_ui_2.md`](specs/generative_ui_2.md) | **Generative UI 2.0** — immersive HITL UI: surface(panel)/hitl(blocking) on emit_generative_ui, 11-template library (recipe/flight/train/form/optionPicker…), side-panel genUI tabs, scenario→element map | 🔄 Phase 1 shipped | diff --git a/ai-company-brain/specs/tenancy_and_visibility.md b/ai-company-brain/specs/tenancy_and_visibility.md new file mode 100644 index 000000000..2c8fd61a8 --- /dev/null +++ b/ai-company-brain/specs/tenancy_and_visibility.md @@ -0,0 +1,411 @@ +# Tenancy and visibility — who can see what + +**Status:** Architecture of record · owner-answered 2026-08-03 · **Date:** 2026-08-03 · +**Verified against code:** 2026-08-03 (every claim below re-measured against the tree at +`520476ab`; the measurement is quoted inline so a later reader can re-run it) · +**Owner:** vjvarada + +**Purpose.** Two audits asked whether the multi-tenant foundation is complete. It is +not built, and the owner's answer is that it should not be. This document records +that decision and the visibility model that replaces it, so the next cycle builds +against a written architecture instead of re-deriving one per app. It is the single +owner for "who can see what" (`work_plan.md` §4). + +**Nomenclature (R3).** The owner says "department". The board and the code say +**Center**. Throughout this document: + +> **department = Center = an `org_group` row.** One slug, three namespaces +> (`center.` feature · `/centers/` route · `org_group.slug`), zero +> mapping tables — `department_centers.md` §1. Write "Center". Never introduce a +> `department` table, column, or feature slug. + +--- + +## 1. DECISION — the tenant boundary is the deployment + +> ### `Tenant boundary = THE DEPLOYMENT.` *(owner-answered 2026-08-03)* +> +> One deployment per tenant. If a second organization ever exists it gets its own +> box, its own database, its own credential set. Row-level organization isolation +> is **explicitly not being built.** + +This is the same rule `department_centers.md` already states from the other +direction ("a *separate deployment* is reserved for a separate organization, never +for a department") and the same rule D9 landed when it rewrote twelve "Pomad Centre" +sites as "a second tenant deployment". It is now the architecture of record rather +than an aside in three specs. + +### 1.1 What follows from it + +**`organization_id` stays as a label, not a mechanism.** It exists on **3 of 111 +own tables** — `app_user` (added by `130_org_access_control.sql:56`), `org_role` +(`130:86`) and `org_group` (`138_groups_and_session_participants.sql:42`). Measured: +111 distinct tables are created by the numbered migrations in `infra/postgres/` +(the 152-name count you get from `schema.generated.sql` includes LiteLLM's and +Langfuse's vendored schemas, which are not ours). It is **read by zero +authorization decisions**: `acb_auth.deps` populates `UserContext.organization_id` +from `resolve_identity()` (`deps.py:155-157`, one extra `SELECT` per authenticated +request against `app_user`), and the only Python readers of the value are the +dataclass that stores it (`roles.py:109-111`) and the line that stores it. Every +`WHERE organization_id = :org` in the gateway binds `:org` from `get_org_id(db)`, +which is the hardcoded `slug = 'default'` lookup — **not** from the caller's +identity. + +Under this decision that is **correct, not a bug**. Do not "fix" it by threading +`user.organization_id` into queries: that would be the first 5% of row-level +multi-tenancy, which §6 puts out of scope, and it would create a second scoping +doctrine alongside the one in §3. + +**The leak sites are moot by definition, not by fix.** The 2026-08-03 audit +enumerated the places where a second `organization` row would serve org A's data to +org B. Verified samples, so a reader can judge the class: + +| # | Site | What it does | +|---|---|---| +| 1 | `routes/admin/_common.py:96-112` | `get_org_id()` resolves the org by the literal `DEFAULT_ORG_SLUG = "default"` (`:36`) | +| 2 | `routes/admin/_common.py:115-129` | `get_member()` looks a member up by email with **no** org predicate | +| 3 | `routes/admin/members.py:170-178` | invite is `INSERT … ON CONFLICT (email) DO UPDATE SET organization_id = EXCLUDED.organization_id` — under two orgs this is an account-takeover primitive | +| 4 | `gateway/rooms.py:184-190` | the `in_org` check is `SELECT 1 FROM app_user WHERE email = :email AND status='active'` — no org filter | +| 5 | `gateway/rooms.py:346-356` | `SESSION_VISIBLE_SQL`'s org-visible arm, same shape, same absence | +| 6 | `acb_auth/access.py:338-340` | `_ORG_MEMBER_SQL` is `SELECT email FROM app_user WHERE status = 'active'` — the `org` subject expands to *every* active user on the box | +| 7 | `infra/postgres/130_org_access_control.sql:180` | role seeding does `SELECT id INTO org_id FROM organization WHERE slug = 'default'` (same in `131:` and `133:`) | +| 8 | `acb_auth/access.py:439-458` | `_BOOTSTRAP_OWNER_SQL` hardcodes `slug = 'default'` | +| 9 | `acb_auth/access.py:460-464` | `_HAS_OWNER_SQL` is `SELECT 1 FROM user_role ur JOIN org_role r … r.slug='owner' LIMIT 1` — **no org filter**, so once *any* owner exists anywhere, `ensure_owner_bootstrap()` is a permanent no-op and a second org's users have no inviter | +| 10 | every app-data table | `gtd_items`, `email_*`, `meeting`, `apps`, `workflows`, `chat_session`, `agent_blob`, `mem` carry no org column at all — the bulk of the surface | + +Under one deployment per tenant, **sites 1, 2, 3, 4, 5, 6, 7, 8, 10 cannot fire** — +there is exactly one `organization` row, so "the default org" and "the caller's org" +are the same set. Site 9 is the interesting one: it is not a leak, it is a +**lockout**, and it is the reason a second org on this box would not merely leak but +would be unusable. That is an argument *for* this decision, not a ticket against it. + +The three joins in §2 are the exception: they are wrong **within one org too**, so +they survive this decision. + +**Per-deployment credentials become correct rather than a gap.** Both audits filed +"credentials are deployment singletons" as a multi-tenancy defect. Verified: +`provider_keys` is keyed `provider TEXT PRIMARY KEY` (`08_provider_keys.sql:7`) — +one key per provider for the whole box; `mcp_servers`, `plugins` and `model_config` +have no owner/org column; and integration secrets reach agents by being written into +the **process-global** `os.environ` (`executor.py:4388`, restored at `:4411`). Under +one deployment per tenant, a deployment-wide credential store is exactly the right +shape. The residual is a *within-org* concern — per-member integration credentials +already ship (`org_access_control.md`), and per-run credential scoping is WS-3's +P5-a, not this document's. + +### 1.2 What a second tenant actually costs + +Stated so the choice stays honest rather than becoming a habit. A second tenant +needs, at minimum: + +1. A second VPS (or at least a second isolated Postgres + Redis; `infra/` binds one + `acb-postgres-data` volume and one `acb-redis-data` volume per stack). +2. A second database, migrated from zero — `scripts/apply_migrations.sh` replays + every numbered migration from `02_` upward on every deploy, so a fresh box gets + the full ladder (140 files today; `00_`/`01_` are initdb-only). +3. A second credential set: provider keys, integration OAuth clients, webhook + secrets, `GATEWAY_INTERNAL_TOKEN`, `LITELLM_MASTER_KEY`, encryption key. +4. DNS + TLS + a second systemd unit set (`deploy/hostinger/` carries four units + plus the generated `acb.service`). +5. A second deploy pipeline target, or a parameterised one. + +That is roughly a day of owner-gated work and a permanent second thing to patch. It +is **not** free — but it is bounded, auditable and does not put a `WHERE +organization_id = ?` on 111 tables and every query in the gateway. The trade is +recorded here so a future reader can re-take it deliberately. + +**Constraint this places on new work.** Non-negotiable #3 in the root `AGENTS.md` +already says native-MAF self-mutation must be swapped for a tenant-isolated +mechanism "before any multi-tenant/customer deployment". Under this decision that +condition is satisfied by construction *for the first tenant* and becomes a +provisioning checklist item for any second one. Do not read this decision as +retiring that constraint. + +--- + +## 2. TV-1 — the three leaks that survive this decision *(AGENT-SAFE · 1 small PR)* + +`org_group` is joined on **slug alone** at three places. Slug is unique only +*within* an organization — `UNIQUE (organization_id, slug)` +(`138_groups_and_session_participants.sql:49`) — so a slug-only join is a +cross-organization match by construction. Under §1 there is one org today, so +nothing leaks today. They are on this list for two reasons: they are three +one-line predicates now and an archaeology project later, and **two of the three +sit inside the session-authority intersection**, which is the single most +consequential access computation in the codebase (`groups_sessions_authority.md` +§3 — a shared run acts with the *intersection* of every participant's access). +Getting a wider group than intended there widens, it does not narrow. + +| Anchor | Symbol | The join | +|---|---|---| +| a | `apps/services/gateway/gateway/rooms.py:170-179` | `SELECT g.slug FROM org_group g JOIN org_group_member m … WHERE u.email = :email AND g.slug = ANY(:slugs)` | +| b | `apps/services/gateway/gateway/rooms.py:332-340` | `SESSION_VISIBLE_SQL`: `JOIN org_group g ON g.slug = substring(p.subject from 7)` | +| c | `packages/acb_auth/acb_auth/access.py:330-336` | `_GROUP_MEMBER_SQL`: `WHERE g.slug = :slug AND au.status = 'active'` | + +**Done when:** + +1. All three queries carry an organization predicate resolved from the row being + authorised, not from a literal — e.g. by joining `org_group.organization_id` to + the acting user's `app_user.organization_id`, so the predicate is *derived* and + cannot go stale when §1 is revisited. A hardcoded `slug='default'` join does + **not** satisfy this: it swaps one wrong constant for another. +2. A hermetic test seeds **two** `organization` rows and two identically-slugged + `org_group` rows (one per org) with disjoint members, and asserts that + `resolve_session_access` for a room whose participant subject is + `group:` expands to **only** org A's members. The test must be verified + **red** against the current joins before the fix — a test that passes on today's + code is testing nothing. +3. The same two-org fixture asserts `rooms.py`'s `my_groups` set and + `SESSION_VISIBLE_SQL` do not admit the org-B member. +4. `uv run ruff check ` is clean. Do **not** write + "`uv run ruff check .` clean" — that command reports ~1983 pre-existing errors on + this tree and is not a signal. + +**Non-goals.** Do not add `organization_id` to any other table, do not thread +`UserContext.organization_id` into unrelated queries, and do not touch sites 1–10 in +§1.1. This ticket is three predicates. + +**Related, not duplicated.** Three *live* access defects — Notes readable/deletable +by any colleague, an identity-trust fallback, and a room fail-open — are being fixed +in **PR #346** (`ws-0-live-access-defects`, "fix(access): three live defects — Notes +was readable, deletable and sendable-as by any colleague"). TV-1 is not those, and +neither should absorb the other. + +--- + +## 3. DECISION — the visibility model *(owner-answered 2026-08-03)* + +> **Owner, verbatim:** *"Sensitive services are private, and we should also have +> department-wise privacy so that the sales team cannot see what the finance team is +> doing. At the same time, we can have organizational-level sharing as well."* +> +> *"Ideally we would have department-wise isolation. At the same time we can share +> some things across departments. At the same time, create projects and groups where +> information can be shared between select users of different departments, depending +> on invite or sharing settings."* + +### 3.1 Three tiers, plus invite + +| Tier | Means | Subject that expresses it | +|---|---|---| +| **private** | the owning member only | the member's `email` | +| **Center** | one Center's members — "sales cannot see finance" | `group:` | +| **org** | every active member of the deployment | `org` | +| **ad-hoc cross-Center group** *(by invite)* | a named set spanning Centers, for a project | an `org_group` row that is **not** one of the six Center groups, addressed the same way: `group:` | + +The fourth row is not a fourth mechanism. A project group is an ordinary `org_group` +whose slug does not pair with a `center.*` feature — the code already distinguishes +the two: `routes/admin/groups.py:37` holds the six Center slugs and `:65` exposes an +"is this a Center group" flag precisely so the UI can treat the rest as ordinary +groups. Membership is by invite, which is what the owner asked for, and it is the +same `org_group_member` table. + +### 3.2 The primitive already exists — generalise it, do not reinvent it + +`chat_session_participant.subject` uses exactly this vocabulary today. +`routes/rooms.py::_valid_subject` (`:100-111`) accepts `org` · `group:` · +an email, and `chat_session.visibility` is `CHECK (visibility IN ('private', +'people', 'org'))` (`138_…sql:83`) — the same three tiers, in shipped code, with +group membership resolved at read time (`gateway/rooms.py:163-179`) rather than +denormalised. + +**Correction to the framing that reached this document.** The brief asserted that +`app_grants` uses the same vocabulary. It does not, and the code says so at both +ends: + +- `routes/apps/grants.py::is_valid_subject` (`:68-85`) is `email | agent: | + agents:*` and **explicitly rejects the literal `org`** (`:77`) with the rationale + that `apps.visibility='org'` already means it. It has **no** `group:` case. +- `routes/rooms.py::_valid_subject`'s own docstring claims it is *"Identical to + `routes/apps/grants.is_valid_subject` on purpose."* **That docstring is false** — + the two functions are disjoint on `org`, `group:` and `agent:`. + +So the primitive is **rooms-only** today. Apps have the *tier* vocabulary +(`private|people|org`, `114_custom_apps.sql:30-31`) but a grant subject that cannot +name a Center. The work is to extend the one grant model outward — which is a +smaller job than designing an ACL, and a larger job than "it's already there". + +**Standing rule for reviewers:** a second scoping doctrine in this codebase is what +produced the Notes hole. When a surface needs sharing, it adopts +`email | group: | org` and resolves group membership at read time. It does not +invent `shared_with_department`, a `visible_to` array, or a per-app grant table with +its own subject grammar. + +### 3.3 Which surfaces are private by default + +**Private by default — a grant is required to widen them:** + +- **Email** — `email_accounts.user_id` (`17_email_accounts.sql:16`, "CC user who + owns this connection"). A mailbox is one person's until a shared-mailbox grant + exists (owned by `email_app_master_plan.md`, sequenced by WS-14, per D5). +- **Tasks / GTD** — `gtd_items.user_id TEXT NOT NULL` + (`48_task_manager_gtd.sql:91`), filtered on 27 query sites in `routes/tasks/items.py`. +- **Notes / meetings** — `meeting.owner_email` (`95_note_taker.sql:38`). Nullable + and, until PR #346, not filtered on read. Private is the intended tier; PR #346 is + what makes it true. +- **Memory (personal)** — the `` scope; `prefs:` likewise. + +**Shareable, with the tier stated on the row:** + +- **Chat / rooms** — `visibility ∈ private|people|org` + participant subjects. +- **Custom Apps** — `apps.visibility ∈ private|people|org` + `app_grants`. + +**Org-wide by construction today (a deliberate posture, recorded so it is not +mistaken for an oversight):** + +- **Workflows** — `crud.py:5` states it outright: *"`owner_email` is attribution, + not access."* The list query is `SELECT … FROM workflows w ORDER BY w.updated_at + DESC` (`crud.py:91`) with no owner predicate, and delete is `DELETE FROM workflows + WHERE id = :id` (`:346`). Anyone holding `feature:workflows` sees and can delete + every workflow. That is fine for an internal tool with one org; it is the first + thing that must change if a Center wants a private automation. +- **Memory `org:global`** — org-wide by definition. + +**A new surface must declare its tier.** This is the doctrine that stops each new +app guessing. Concretely, for a reviewer: + +> A PR that adds a persisted user-facing surface names its default tier in the +> migration header and either (a) carries an owner column and filters on it, or +> (b) carries `visibility` + a subject grant table using §3.2's vocabulary, or +> (c) states in the header that it is intentionally org-wide and why. "It inherits +> the app's tier" is not one of the three. + +--- + +## 4. DECISION — what "a project belongs to a team" means + +`DECISION (owner-answered 2026-08-03)` + +This semantic has blocked **WS-14 Centers C** for weeks. It resolves to: + +> **A project belongs to a team when an explicit grant row carries a +> `group:` subject for that project.** Not derived from who is assigned to +> its tasks, and not an owning column on the project row. + +**Why an explicit grant.** + +1. It is the same mechanism as §3.2, so a Center project, a cross-Center project + group, and an org-visible project are one code path with a different subject. +2. It is revocable as a distinct act. Removing a grant is visible in an audit log; + re-assigning tasks to change who can see a project is not. +3. It composes with the intersection rule. `resolve_session_access` already expands + `group:` subjects at read time; a project grant slots into that expansion without + a second resolver. + +**Alternative rejected — derive it from assignees** ("a project belongs to whichever +team its assignees are in"). Rejected because access would then be a side effect of +task assignment: assigning one finance colleague to a sales project would silently +admit all of finance, and *un*assigning the last member of a team would silently +revoke a whole Center's access to a project mid-flight. Access must be an act, not a +consequence. + +**Alternative rejected — an owning `group_id` column on the project row.** Rejected +because it is single-valued: it cannot express the owner's third requirement ("share +between select users of different departments"), so the cross-Center project case +would need a *second* mechanism the day after it shipped. A grant table is +single-valued when it has one row. + +**What WS-14 can now build.** The `dynamic_agents` sharing columns (D3) and the +tasks team slice both depend on this answer, and both now have one. Note the +verified constraint: `dynamic_agents` today has **no** owner, visibility or sharing +column (`15_dynamic_agents.sql:7-20`; grep for sharing/visibility/owner across +`infra/postgres/[0-9]*.sql` returns nothing), so WS-14 owns that migration — at the +**next free number at build time**, never a number written into a doc (R1). + +--- + +## 5. Gap table — the map for going app by app + +Each row verified against code on 2026-08-03. "Honours `group:`" means: a +`group:` subject can be granted on this surface and is expanded at read time. + +| Surface | Storage of record | Current scoping | Honours `group:`? | What it would need | +|---|---|---|---|---| +| **Chat / rooms** | `chat_session`, `chat_session_participant` | `user_id` owner + `visibility ∈ private/people/org` + participant subjects `email\|group:\|org` | **Yes** — the only one | Nothing. This is the reference implementation. Fix TV-1's two joins here. | +| **Tasks / GTD** | `gtd_items` (+ `gtd_projects`, `gtd_spaces`) | `user_id TEXT NOT NULL`, filtered on every read | No | A grant table keyed on the project (per §4) and a read path that unions "mine" with "granted to a group I'm in". The 27 `user_id` predicates in `items.py` are the blast radius. | +| **Email** | `email_accounts` (+ ~20 `email_*` tables hanging off it) | `email_accounts.user_id` | No | Shared mailboxes = a grant on the *account* row, not on messages. Owned by `email_app_master_plan.md` (D5). Per-member provider credentials already exist. | +| **Notes / meetings** | `meeting`, `transcript_segment`, `meeting_note` | `meeting.owner_email` (nullable, `95_note_taker.sql:38`); read filtering is PR #346's | No | Owner filter first (PR #346), then a grant table. Do not add sharing before the owner filter lands — sharing on an unfiltered surface is decoration. | +| **Agents** | `dynamic_agents` + `agent_skill_setting` | No owner/visibility column at all; run rights via the `agents:run:` permission | No | D3's sharing columns (WS-14, next free migration number). Note `agent_blob.instance` already reserves `t:` in its vocabulary (`136_agent_blob_instance.sql:27`) but **nothing writes it** — the partition exists, the team case does not. | +| **Memory** | Mem0 (pgvector), keyed by scope string | Five scope shapes: `` · `prefs:` · `room:` · `agent:` · `org:global` (`routes/memory.py:16-56`) | No — `room:` is the nearest thing | A `group:` scope shape, or the `subject:` compartments already specified as **WS-10 S1** (`docs/multiplayer/memory-clearance.md` §7.1). Do not add a sixth shape independently of that slice. | +| **Workflows** | `workflows`, `workflow_versions`, `workflow_triggers` | `owner_email` is **attribution only**; list query has no owner predicate (`crud.py:91`), delete has none (`:346`) | No | A `visibility` column + grants, if a Center ever wants a private automation. Until then, record the org-wide posture rather than assuming it. | +| **Apps / blobs** | `apps` + `app_grants`; `agent_blob` + `agent_file_history` | `apps.visibility ∈ private/people/org`; `app_grants.subject ∈ email\|agent:\|agents:*` — `org` explicitly rejected (`grants.py:77`); `agent_blob.instance ∈ ''\|u:\|t:` | No | Add `group:` to `is_valid_subject` and expand it at read time, mirroring `rooms.py`. Fix the false "identical to grants.is_valid_subject" docstring at `rooms.py:103` in the same change. | + +**Reading the table.** Rooms is the reference. Apps is the cheapest next +conversion (it already has the tiers; it is missing one subject case). Tasks is the +one §4 unblocks. Notes is sequenced behind PR #346. Workflows is a posture decision +before it is a code change. + +--- + +## 6. Explicitly out of scope + +Named so nobody builds them, and so a future audit does not re-file them as gaps: + +1. **Row-level multi-tenancy.** No `organization_id` on further tables, no RLS + policies, no org predicate threaded through app queries. §1 replaces it. + (`multi_user_organization_research.md` §9's entity-graph RLS and §17's SaaS + tenancy stay research, and are superseded for planning purposes by this + document.) +2. **An org switcher.** No UI, no route, no `X-Organization` header, no + "current org" in session state. One deployment serves one org; there is nothing + to switch to. +3. **Users belonging to multiple orgs.** `app_user.email` is globally unique + (`ON CONFLICT (email)` at `members.py:173` and `access.py:447` both depend on + it). Multi-org membership would require breaking that uniqueness, which would + ripple into every identity lookup. Not being done. +4. **Per-org credentials inside one deployment.** Credentials are per-deployment by + §1.1. Per-*member* integration credentials already ship and are a different + thing. + +If any of these is ever wanted, the correct move is to re-take the §1 decision +first, in this document, with a date and a reason — not to build one of them as a +side effect of an app ticket. + +--- + +## 7. Verification + +Hermetic; no live DB, no prod reach. Each command reproduces a claim above. + +``` +# §1.1 — three tables carry organization_id, out of 111 own tables +grep -rn "organization_id" infra/postgres/*.sql +ls infra/postgres/[0-9]*_*.sql | wc -l # 142 numbered migration files + +# §1.1 — the hardcoded org slug and the ownerless-bootstrap no-op +grep -n "DEFAULT_ORG_SLUG" apps/services/gateway/gateway/routes/admin/_common.py +grep -n "_HAS_OWNER_SQL" -A 5 packages/acb_auth/acb_auth/access.py + +# §2 — the three slug-only joins +grep -n "org_group" apps/services/gateway/gateway/rooms.py \ + packages/acb_auth/acb_auth/access.py + +# §3.2 — the two subject vocabularies, and that they differ +grep -n "def _valid_subject" -A 12 apps/services/gateway/gateway/routes/rooms.py +grep -n "def is_valid_subject" -A 18 apps/services/gateway/gateway/routes/apps/grants.py + +# §5 — the surfaces' owner columns +grep -rn "user_id\|owner_email\|visibility" infra/postgres/48_task_manager_gtd.sql \ + infra/postgres/95_note_taker.sql infra/postgres/114_custom_apps.sql \ + infra/postgres/132_workflows.sql infra/postgres/138_groups_and_session_participants.sql +``` + +Test files that already exercise this area, and are the right place to extend +(**name the file — never run `tests/unit/` as a directory on the Windows box**): +`tests/unit/test_session_authority.py`, `tests/unit/test_rooms.py`, +`tests/unit/test_org_access_control.py`, `tests/unit/test_owner_bootstrap.py` +(⚠️ never against prod). + +--- + +## 8. Open, and deliberately unanswered here + +- **Whether Workflows should stay org-wide.** §5 records the posture; changing it is + a product call, not a defect. No acceptance is written for it. +- **Where a project grant table lives** (`gtd_*` vs a shared `object_grants`). §4 + fixes the *semantic*; the storage shape is WS-14's design call, and the only + binding constraint is §3.2's subject vocabulary. +- **Whether `subject:` memory compartments and a `group:` memory scope are the same + feature.** WS-10 S1 owns the compartment design + (`docs/multiplayer/memory-clearance.md` §7.1); this document only records that + memory has no `group:` scope today. diff --git a/ai-company-brain/work_plan.md b/ai-company-brain/work_plan.md index f73e8a590..f0d154ee0 100644 --- a/ai-company-brain/work_plan.md +++ b/ai-company-brain/work_plan.md @@ -2,7 +2,10 @@ **Status:** Active · **Date:** 2026-08-03 (six-row truth pass: WS-1, WS-3, WS-8, WS-11, WS-12, WS-21 swept to match their rewritten specs; D10 records two owner -calls) · **Owner:** vjvarada +calls. **Second pass the same day:** D11 + D12 record the tenancy boundary and +the visibility model from `specs/tenancy_and_visibility.md`; WS-14 unblocked; +WS-13 gains the verified "Centers are unreachable by anyone" finding; §2 gains +the three app-by-app exceptions) · **Owner:** vjvarada **Purpose:** the single sequencing document from which independent agents are dispatched. Content lives in the owning specs; *this* doc owns ordering, ownership, and the rules that make a spec executable without questions. @@ -66,6 +69,20 @@ Tier 3 annotations are done, verified against code. Residual items listed at the top of §5. Findings folded back into this doc: D7 gained the MAF-side MCP gap; calendar P3 was found already shipped (with revised roll-over semantics). +### Can we go app by app? — yes, with three exceptions *(2026-08-03)* + +The owner asked whether the foundation is complete enough to work app by app. It +is. **Three items are exceptions** — they are not app work, they do not get +better by being deferred behind app work, and one of them gets *worse* with every +app added. Recorded here because §2 is where a reader planning the next app +looks. Full statements live in `FOUNDATION_BUILDOUT_CHECKLIST.md`. + +| # | Exception | Where it lives | Why it can't wait for "after the apps" | +|---|---|---|---| +| 1 | **`main` has no branch protection** | WS-5 · checklist §BO-17 | Re-verified 2026-08-03 against the live repo: `gh api repos/FracktalWorks/CommandCenter/branches/main/protection` → **`404 Branch not protected`** *and* `gh api …/rulesets` → **`[]`**. So there is no protection under either mechanism, and **every CI gate in the YAMLs is decorative** — a push straight to `main` gets zero check-runs and a red PR can merge. Every app shipped from here inherits that. **OWNER-GATE** (a GitHub settings change; an agent cannot make it). | +| 2 | **No backup / restore path** | **new: checklist §BO-23** | The only DB script that dumps anything is `scripts/dump_schema.sh`, which is `pg_dump --schema-only` (`:52`) — **structure, zero rows**. There is no `pg_restore`, no logical data dump, no WAL archiving (`archive_mode`/`wal_level`/`pgbackrest`/`wal-g` appear nowhere in `infra/` or `deploy/`), and no restore runbook. Meanwhile `scripts/apply_migrations.sh` replays **every** numbered migration ≥ `02_` on **every** deploy under `psql -v ON_ERROR_STOP=1` (`:59-74`) with no ledger and no down-migrations — 140 files today, 142 numbered files on disk. `deploy/hostinger/README.md:115` is honest that the only backup is Hostinger's **weekly whole-VPS** image and that PITR is a "later" item. Largest uncovered risk, and it scales with app count. | +| 3 | **DB engine sprawl** | checklist §BO-10 | Measured 2026-08-03: **12 `create_async_engine(...)` call sites across 10 modules** (`acb_auth/access.py:69`; gateway `routes/{admin,apps,email,notes,tasks,whatsapp,workflows}/*core*.py`; `email_ingestion/{inbound,scheduler}.py` ×4), plus a 13th **sync** `create_engine` in `acb_graph/db.py:32`. Eight are module-level cached `_ENGINE` singletons and **none of them is disposed on shutdown** — the only `engine.dispose()` calls in the tree are the four `email_ingestion` per-call engines cleaning up after themselves. **This is the one that compounds: one engine per app, added by each app.** The next app should extend a shared seam, not add engine 13. | + ### Substrate (foundation) | WS | Workstream | Owning spec | State | Next / notes | @@ -93,8 +110,8 @@ gap; calendar P3 was found already shipped (with revised roll-over semantics). | WS | Workstream | State | Next / notes | |---|---|---|---| -| WS-13 | **Centers B — groups become real** (groups admin UI, seed six groups, People directory read view) | 🟡 | Groups admin UI + six-group seed **built 2026-08-01, pending owner review** (`routes/admin/groups.py`, `/settings/groups`, seed migration; see `department_centers.md` Phase B update). People directory read view still open. The unlock for everything below. Single owner: Centers B (groups spec §6 step 5 and org_access Phase 2 are mirrors). | -| WS-14 | **Centers C — scoping deepens** (tasks team slice, shared mailboxes, team-instanced agents, per-Center approvals) | 🟡 WS-13 + D3 | Audit correction: the blob/memory substrate is live but the **`dynamic_agents` sharing columns do not exist** (agent-kinds' "migration 119" was never built) — Centers C includes that migration per D3. | +| WS-13 | **Centers B — groups become real** (groups admin UI, seed six groups, People directory read view) | 🟡 | Groups admin UI + six-group seed **built 2026-08-01, pending owner review** (`routes/admin/groups.py`, `/settings/groups`, seed migration; see `department_centers.md` Phase B update). People directory read view still open. The unlock for everything below. Single owner: Centers B (groups spec §6 step 5 and org_access Phase 2 are mirrors). ⚠️ **NEW FINDING 2026-08-03, verified end-to-end: Centers are unreachable by ANYONE, including the owner.** `/auth/me` returns `"features": list(access.allowed_features())` (`routes/admin/me.py:84`), and `allowed_features()` iterates the **hardcoded Python tuple** `acb_auth.permissions.FEATURES` (`:64-81`) — sixteen slugs, **no `center.*` entry**. The frontend gates on exactly those slugs: `lib/access.ts:66` maps `/centers/` → `c.feature` (= `center.sales`…), `canUseFeature` is `access.features.includes(slug)` (`:118`), and `visibleSections` drops any pane whose feature is absent — **and drops the whole section when it empties** (`lib/nav.ts:229-233`). Net effect: the Centers section renders in neither nav, and typing `/centers/sales` hits `AccessGate`'s "You don't have access to this". Migration `140_center_features.sql` **does** seed six `feature_catalog` rows, but `allowed_features()` never reads that table — so migration 140's own comment ("owners and admins see all Centers via their `feature:*` baseline") is **false as written**: an owner holding `*` still gets an empty set, because the wildcard is only ever evaluated against the sixteen literals. The fix is one line of vocabulary (make `FEATURES` include the Center slugs, or make `allowed_features()` read `feature_catalog`) plus a test that a `feature_catalog` row with no `FEATURES` entry fails loudly. AGENT-SAFE; it belongs to this row because Phase A shipped the surface this hides. | +| WS-14 | **Centers C — scoping deepens** (tasks team slice, shared mailboxes, team-instanced agents, per-Center approvals) | 🟢 **unblocked 2026-08-03 (D12)** | **The blocker is answered.** This row read "blocked on what makes a project a team's project" for weeks; **D12** answers it: **a project belongs to a team when an explicit grant row carries a `group:` subject** — *not* derived from assignees, *not* an owning column. Both alternatives and why they were rejected are recorded in `specs/tenancy_and_visibility.md` §4 (`DECISION (owner-answered 2026-08-03)`); §5's gap table is the app-by-app map, and §3.2 is binding on the mechanism — **extend the existing `email \| group: \| org` subject vocabulary, do not invent a second one.** ⚠️ **The primitive is narrower than previously claimed:** only **rooms** honour `group:` today (`routes/rooms.py::_valid_subject` `:100-111`, expanded at `gateway/rooms.py:163-179`). `app_grants` does **not** — `routes/apps/grants.py::is_valid_subject` (`:68-85`) is `email \| agent: \| agents:*` and explicitly **rejects `org`** (`:77`); the "identical to grants.is_valid_subject" docstring at `rooms.py:103` is false and should be corrected by whichever ticket touches it first. **What it can now build, in order:** (1) the tasks team slice — a project grant table + a read path unioning "mine" with "granted to a group I'm in" (blast radius: 27 `user_id` predicates in `routes/tasks/items.py`); (2) the `dynamic_agents` sharing columns per D3 — re-verified 2026-08-03, `15_dynamic_agents.sql:7-20` has **no** owner/visibility/sharing column and a repo-wide grep finds none, so this migration is genuinely WS-14's, at the **next free number resolved at build time** (R1); (3) `group:` on the Custom-Apps grant subject, the cheapest conversion since `apps.visibility` already carries the three tiers. Shared mailboxes stay `email_app_master_plan.md`'s implementation, sequenced here (D5). **Not blocked on WS-8 Phase A** (D3 amendment) and **not** waiting on WS-13's UI — but note WS-13's new finding: the Center *surfaces* are currently unreachable, so scoping work will need that one-line feature-vocabulary fix to be demonstrable. | | WS-15 | **Centers D — dashboards + Company Center** (Center dashboards, personal dashboard, weekly digest workflows, orchestrator org-memory fix per D4) | 🟡 WS-13 | Digest workflows double as `workflows_app.md` G1 launch metric — one artifact, both scorecards. | | WS-16 | **Centers E — AI budgets** (per-member caps at the LLM choke points; per-room degrade later) | 🟡 WS-6 | Subjects per D2. | @@ -191,6 +208,56 @@ calls, taken and dated. the Integration Registry has the integration), not the control-flow *vocabulary*** — and must not be cited as a blocker on WS-11's 8.3c. Recorded in `workflows_app.md` §8.3c and §11 R1. +- **D11 — The tenant boundary is THE DEPLOYMENT.** *(owner call, 2026-08-03.)* + One deployment per tenant: a second organization gets its own box, its own + database, its own credential set. **Row-level organization isolation is + explicitly NOT being built.** Consequences, each verified against code and + recorded in `specs/tenancy_and_visibility.md` §1: `organization_id` stays a + **label, not a mechanism** — it is on **3 of 111** own tables (`app_user`, + `org_role`, `org_group`) and is read by **zero** authorization decisions + (`UserContext.organization_id` is populated by an extra `SELECT` at + `acb_auth/deps.py:155-157` and never consulted; every `WHERE organization_id` + in the gateway binds from `get_org_id()`'s hardcoded `slug='default'`, not + from the caller). Nine of the ten enumerated leak classes are **moot by + definition** rather than by fix; deployment-singleton credentials + (`provider_keys.provider` is the PK; integration secrets go into the + process-global `os.environ`) become **correct** rather than a gap. The cost of + a second tenant — new box, new DB migrated from zero, new credential set, DNS + + TLS + systemd units — is written down in §1.2 so the choice stays honest. + **Do not** "fix" this by threading `user.organization_id` into queries: that + is the first 5% of row-level multi-tenancy and creates a second scoping + doctrine alongside D12's. The one carve-out is **TV-1** (§2 of that spec): the + three `org_group` joins that match on **slug alone** are wrong *within* one org + too, two of them inside the session-authority intersection — + `gateway/rooms.py:170-179`, `:332-340`, `acb_auth/access.py:330-336`. + AGENT-SAFE, one small PR, with a two-org fixture that must be verified red + first. Owner: the new spec. +- **D12 — Visibility is private → Center → org, plus ad-hoc groups by invite; + and a project belongs to a team by an explicit `group:` grant.** *(owner call, + 2026-08-03; owning spec `specs/tenancy_and_visibility.md` §3–§4.)* The owner's + words were *"department-wise privacy so that the sales team cannot see what the + finance team is doing… at the same time organizational-level sharing… and + projects and groups where information can be shared between select users of + different departments, depending on invite."* **department = Center = + an `org_group` row** (R3; `department_centers.md` §1) — write "Center". + **The primitive exists and must be generalised, not reinvented:** + `routes/rooms.py::_valid_subject` (`:100-111`) already accepts + `email | group: | org` and `chat_session.visibility` is already + `private|people|org`, with group membership expanded at read time + (`gateway/rooms.py:163-179`). **Correction to the claim that reached this + board:** `app_grants` does **not** share that vocabulary — `routes/apps/ + grants.py::is_valid_subject` (`:68-85`) is `email | agent: | agents:*` + and **rejects the literal `org`** (`:77`), with no `group:` case at all; the + docstring at `rooms.py:103` claiming the two are "identical" is **false**. + Rooms is the only surface honouring `group:` today; the gap table in §5 of the + spec is the app-by-app map. **"A project belongs to a team" = an explicit grant + row carrying a `group:` subject** — *not* derived from assignees (access + would become a side effect of task assignment) and *not* an owning column + (single-valued, so it cannot express the cross-Center project the owner asked + for). This is the semantic that has blocked **WS-14** for weeks; it is + answered. **Standing review rule:** a new persisted user-facing surface + declares its tier — it does not inherit one by accident. Two doctrines in one + codebase is what produced the Notes hole (being fixed separately in PR #346). ## 4. Single-owner registry (who owns duplicated work) @@ -212,6 +279,7 @@ calls, taken and dated. | Chat HITL model | **generative_ui_2.md §2** (shipped) | chat_ux §12.3 (superseded) | | Multiplayer prior art (`qm`, 2026-08-01) | **`multiplayer_prior_art_qm_2026-08.md` is reference-only** — it owns no work and no status; the specs it links stay authoritative | multiplayer README §4.6/§5.1/§6.4/§6.5 · memory-clearance §3.3/§7 · agent-kinds §9 Q1 · skills_scope_out §6 · WS-10 · WS-23 | | Memory compartments + clearance (incl. `subject:`) | **`docs/multiplayer/memory-clearance.md` §7** (surface design §7.1); dispatched as **WS-10 S1** | memory_architecture §9 `3a′` (link-only since 2026-08-02) · multiplayer README §6.3/§8 Phase 3 (index only) · prior-art §QM-D1 (reference only) | +| Tenancy boundary + visibility model (who can see what) | **`specs/tenancy_and_visibility.md`** (D11 §1 · D12 §3–§4 · the app-by-app gap table §5 · TV-1 §2) | `department_centers.md` (the "separate deployment is for a separate org, never a department" rule) · `org_access_control.md` §8 Ph2 · `multi_user_organization_research.md` §5/§7/§8/§9/§17 (**research only, and superseded for planning by the new spec**) · `groups_sessions_authority.md` §3 (the intersection rule it constrains) · D9 (the twelve "second tenant deployment" sites) | ## 5. Documentation remediation backlog (WS-0)