From 4d699ba28ded7eb1f6acf861e45c768d51d7a635 Mon Sep 17 00:00:00 2001 From: Vijay Raghav Varada Date: Sat, 8 Aug 2026 03:22:29 +0530 Subject: [PATCH 1/4] =?UTF-8?q?feat(WS-26d-autolead):=20an=20unknown=20inb?= =?UTF-8?q?ound=20sender=20becomes=20a=20lead=20=E2=80=94=20behind=20a=20f?= =?UTF-8?q?lag=20that=20is=20still=20off?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The last demo-path slice. With CRM_AUTO_LEAD off — which is how it ships and how it stays until the owner flips it (work_plan.md §6 (b)) — this branch changes no runtime behaviour at all. That is not a caveat, it is done-when 2: the flag is read at the CALL SITE in process_new_mail, so the OFF state enters no CRM code and issues no CRM query. A gate that lived inside the step would have satisfied a careless test while opening a database session on every sync cycle of every mailbox to discover it had nothing to do. An AST assertion pins the gate's position, because the runtime sentinel cannot see a refactor. THE SEAM IS ALSO REACHED BY DEEP RESYNCS, AND THAT IS THE WHOLE DESIGN. process_new_mail is the one place scheduler, manual sync and webhook all funnel through — which is why the ticket chose it — but resync_account runs a ~1-year all-folder backfill and then fires the same hook, a newly connected mailbox's first sync is deep by the same heuristic, and neither stamps rules_held_back_at. A candidate query of "everything classified" would therefore mint a lead per unknown sender across a year of mail the moment a second mailbox connects, each born zoho_dirty and queued for the LIVE Zoho tenant within one 600s cycle (D-CRM-9), with no confirmation card anywhere on a scheduler hook and no delete tool to take them back. So the step keeps a per-account two-timestamp cursor (migration 157, crm_auto_lead_cursors). activated_at is stamped ONCE on the first ON-state run and never advanced: `received_at > activated_at` is the backfill discriminator, and it says mail that ARRIVED before auto-lead existed for this account mints nothing no matter when a resync gets around to classifying it. processed_watermark is the ordinary incremental cursor. Both apply together. The test that matters seeds a year-old classified backlog and asserts zero leads; deleting the first predicate turns it red. DEDUP IS A SELECT GUARD PLUS IN-BATCH DE-DUPLICATION, NEVER ON CONFLICT. crm_leads has no unique constraint on email — idx_crm_leads_email is a plain index, only zoho_id is UNIQUE — so the shape the ticket originally prescribed could not have fired at all. The cross-invocation race (two concurrent syncs of one account reading the same watermark) is ACCEPTED and recorded: the cost is one visible, hand-deletable duplicate, and the fix that suggests itself is a UNIQUE index on a column where 1,516 imported rows may already carry duplicates — a deploy-blocking constraint of exactly the shape migration 148 had to defuse. The in-batch half is asserted as one unknown-sender probe per ADDRESS rather than "one lead": the fake shares a dict, so "one lead" would stay green with the dedup deleted while production, where those two sessions are genuinely concurrent, minted two. "NEVER A COLLEAGUE" IS TWO GATES BECAUSE IT IS TWO QUESTIONS. sender_scope fails SAFE to "external", which is the wrong direction when the consequence is a lead row for your own CFO in a live tenant, so it is necessary and not sufficient. The second gate is the internal-domain list, normalised through resolve_org_domains — and that is where it earns its place rather than restating the first: sender_scope's own extra-domain arm only lstrip('@')s its input, the exact divergence runner.py documents, so a colleague on a second company domain that somebody typed as an address is external to gate 1 and internal to gate 2. Deleting gate 2 mints him a lead, and a named test says so. The first activity is metadata and never content: type='system' — deliberately outside sync_zoho's `type IN ('note','task')` push predicate, so the subject and sender stay inside the native CRM even though the lead itself is queued for the tenant — with an EMPTY body and no snippet. The candidate SELECT does not read body_text or snippet at all: the projection is the privacy boundary, and nothing downstream can leak a body it was never handed. The exclusion is asserted against push_activities' own statement text, not against the constant beside it. The lead goes through records.create_record, never raw SQL: _resolve_status, the owner_email default, validate_source and mark_dirty_on_insert all live only there, and only the last of the four is visible in the row afterwards. lead_name is left to compute_lead_name over a display name stripped BEFORE it is split. Placement: routes/crm/auto_lead.py, not the email package. What it does is write a CRM record — it owns the cursor table, it uses the CRM's own write path, its flag is a CRM owner gate. It registers no routes and, like broker_handlers.py, is not imported from routes/crm/__init__.py. It IMPORTS the automation package's public identity primitives rather than copying them: D-CRM-4 declined to import another package's private helper, and a third copy of "is this person a colleague?" is the drift that rule exists to prevent. _crm_fakes.py gained one reader — jsonb `@>` containment. Without it the "have we ever emailed them" probe was invisible to the fake, which answered "yes" for every message in Sent regardless of recipient, and the already-known-contact case would have been a test of nothing. Same lesson as the four readers WS-26d-email had to add. Tests: tests/unit/test_crm_auto_lead.py, 52 cases, every done-when named. Seven mutants run red and reverted: the flag check, the activated_at predicate, the watermark advance, the internal-domain gate, type='system', the service write path, and the in-batch dedup. BUILT, NOT FLIPPED, NOT DEPLOYED. R4 sweep in this same change: crm_app.md status header + the WS-26d-autolead ticket + §9.0's D5 row, work_plan.md's WS-26 row and §6 gate (b) — whose "the settings field does not exist yet" note is now retired — plus the gateway and infra AGENTS.md. Co-Authored-By: Claude Fable 5 --- ai-company-brain/specs/crm_app.md | 88 +- ai-company-brain/work_plan.md | 25 +- apps/services/gateway/AGENTS.md | 3 +- .../gateway/gateway/routes/crm/auto_lead.py | 582 +++++++++ .../gateway/routes/email/scheduler_hooks.py | 28 +- infra/AGENTS.md | 2 +- infra/postgres/157_crm_auto_lead_cursor.sql | 82 ++ packages/acb_common/acb_common/settings.py | 11 + tests/unit/_crm_fakes.py | 49 + tests/unit/test_crm_auto_lead.py | 1092 +++++++++++++++++ 10 files changed, 1950 insertions(+), 12 deletions(-) create mode 100644 apps/services/gateway/gateway/routes/crm/auto_lead.py create mode 100644 infra/postgres/157_crm_auto_lead_cursor.sql create mode 100644 tests/unit/test_crm_auto_lead.py diff --git a/ai-company-brain/specs/crm_app.md b/ai-company-brain/specs/crm_app.md index 973731718..0cdd36076 100644 --- a/ai-company-brain/specs/crm_app.md +++ b/ai-company-brain/specs/crm_app.md @@ -50,6 +50,18 @@ > `tests/unit/test_crm_agent_write.py` (76 cases) + `test_crm_agent.py` grown from > 87 to 143. **LIVE: a confirmed agent write is born `zoho_dirty` and reaches the > live tenant within one 600s sync cycle (D-CRM-9).** +> · **WS-26d-autolead: 🟢 BUILT 2026-08-08 (branch `ws-26d-autolead`) — NOT +> FLIPPED, NOT DEPLOYED.** `CRM_AUTO_LEAD` exists in +> `acb_common/settings.py` and ships **False**; the CRM step is +> `routes/crm/auto_lead.py`, called from `process_new_mail` from **inside +> `if auto_lead_enabled():`** so the OFF state enters nothing; the new +> `crm_auto_lead_cursors` table (migration **157**) carries the +> `activated_at` / `processed_watermark` pair the deep-resync discriminator +> needs. **Nothing has been flipped and nothing has been deployed** — the +> flip stays OWNER-GATE (`work_plan.md` §6 (b)), and while the flag is off +> this changes no runtime behaviour at all. Tests: +> `tests/unit/test_crm_auto_lead.py` (52 cases); seven mutants run red and +> reverted. > · **WS-26e: 🟡 SPEC, nothing built.** > **26f** — 🟢 **MERGED + DEPLOYED 2026-08-07 (PR #391), NOT RUN against the tenant.** f1 > `POST /crm/import/zoho/stages` (`routes/crm/stage_metadata.py`, floor @@ -957,7 +969,7 @@ unknown sender becomes a lead on its own. | D2 | **WS-26d-email** | The "this is not a toy" moment. Disjoint files from D1 (`activities.py`/`Timeline.tsx` vs. importer/admin/settings). | ∥ with D1 | | D3 | **WS-26g** — ✅ **BUILT 2026-08-07** (branch `ws-26g-reports`, no migration) | The forecast number. **After D1** — f2 and the reports tab both extend the `page.tsx`/`urlState.ts` tab grammar, and two parallel PRs there is a needless conflict. | after D1 | | D4 | **WS-26d-write** ✅ **BUILT 2026-08-08** | The AI-creates-a-lead demo beat. Lives in `apps/agents/agent-crm/` — collides with nothing above. No migration. | ∥ with any | -| D5 | **WS-26d-autolead** | Build whenever; the **flip is OWNER-GATE** and pushes real leads into Zoho (D-CRM-9) — demo it only if the owner wants that story told live. | ∥ with any | +| D5 | **WS-26d-autolead** — ✅ **BUILT 2026-08-08, flag OFF, NOT flipped, NOT deployed** (migration 157) | Built whenever; the **flip is OWNER-GATE** and pushes real leads into Zoho (D-CRM-9) — demo it only if the owner wants that story told live. | ∥ with any | **Deferred until after the demo, deliberately — not demoted:** WS-26h (discipline), WS-26i (data management), WS-26e (cutover). No demo viewer sees them; they lose nothing @@ -1404,9 +1416,81 @@ in a later slice. **Tests:** `tests/unit/test_crm_email_timeline.py` (B7), reusing `tests/unit/_crm_fakes.py`. Frontend: extend the existing CRM vitest for the third `kind`. -### WS-26d-autolead — `CRM_AUTO_LEAD` · 🟡 AGENT-SAFE to build · 🔴 OWNER-GATE to flip +### WS-26d-autolead — `CRM_AUTO_LEAD` · ✅ **BUILT 2026-08-08** · 🔴 **OWNER-GATE to flip — NOT FLIPPED, NOT DEPLOYED** *(Closes B4.)* +> **As built** (branch `ws-26d-autolead`, migration **157** +> `157_crm_auto_lead_cursor.sql` — the number taken from the directory at +> build time per R1, and `test_crm_auto_lead.py` finds the file by CONTENT, +> so a renumber in review breaks nothing). **The flag is `False` everywhere and nothing has been +> deployed**: with `CRM_AUTO_LEAD` off this branch changes no runtime +> behaviour, which is the whole point of done-when 2. +> +> * `crm_auto_lead: bool = False` in `packages/acb_common/acb_common/settings.py`, +> beside `crm_zoho_sync` (its precedent shape). **`.env.example` deliberately +> untouched** — plan-guard territory; the flag is documented here and in +> `settings.py` only, and a test pins its absence from that file. +> * `apps/services/gateway/gateway/routes/crm/auto_lead.py` — the whole step. +> **It lives in `routes/crm` and not in the email package** because what it +> does is write a CRM record; it registers no routes and, like +> `broker_handlers.py`, is deliberately absent from +> `routes/crm/__init__.py`. It **imports** the automation package's PUBLIC +> identity primitives (`sender_scope` / `resolve_org_domains` / +> `normalize_domain`) rather than copying them — D-CRM-4 declined to import +> another package's *private* helper, and a third copy of "is this person a +> colleague?" is the drift that rule exists to prevent. +> * The call site in `routes/email/scheduler_hooks.py::process_new_mail` is +> `if auto_lead_enabled(): await create_leads_from_new_mail(account_id)` +> inside the sibling `try/except` shape, logging `sync.auto_lead_failed`. +> **It runs LAST, after auto-archive**, so the step considers what is still +> in the INBOX once the account's own automation has finished — mail the +> user's own rules archived never becomes a lead. +> * `tests/unit/test_crm_auto_lead.py` — **52 cases**, each done-when named in +> a test. `tests/unit/_crm_fakes.py` gained ONE reader (`@>` jsonb +> containment) because without it the "have we ever emailed them" probe was +> invisible to the fake, which answered "yes" for every Sent message and +> would have made the already-known-contact case a test of nothing. +> * **Seven mutants run red and were reverted**: the flag check, the +> `received_at > activated_at` predicate, the watermark advance, the +> internal-domain second gate, `type='system'`, the service write path, and +> the in-batch dedup. +> +> **Five decisions the ticket did not record, each with its reason:** +> 1. **`activated_at` and `processed_watermark` are stamped to the SAME +> instant on activation**, so the activating cycle mints nothing. That is +> the deep-resync case stated positively: on the day the flag flips, every +> mailbox's entire history predates activation. +> 2. **The two colleague gates answer different questions, and gate 1 is asked +> WITHOUT the configured extra domains.** `sender_scope`'s own extra-domain +> arm only `lstrip('@')`s its input while `resolve_org_domains` runs +> `normalize_domain` — the exact divergence `runner.py:1635-1641` documents. +> Routing the configured list through gate 2 alone means there is ONE +> normalisation of it here rather than two that can disagree; it is also +> what makes gate 2 load-bearing rather than a restatement of gate 1, and a +> named test (a colleague on an org domain somebody typed as an address) +> goes red when it is deleted. +> 3. **The watermark advances over messages that minted nothing, and over +> messages that raised.** This is a best-effort enrichment step, not a +> queue: a poison message holding the cursor would re-fail on every cycle +> for the life of the mailbox. Errors are COUNTED in the log line instead, +> and each candidate is wrapped in `core.savepoint` so one statement error +> cannot abort the batch's transaction (the WS-26b lesson). +> 4. **The lead and its first activity are two transactions.** +> `create_record` opens and commits its own session — it is the same +> function `POST /crm/leads` calls — so a failure between the two leaves a +> lead with an empty timeline, logged and counted. The alternative was a +> second, divergent write path for the record itself, which is what +> done-when 3 forbids. +> 5. **An account whose `user_id` is blank is skipped before the cursor is +> even activated.** `actor()` would attribute the lead to `"anonymous"`, +> and a lead that is nobody's follow-up and that the `owner` filter cannot +> match is worse than no lead. +> +> **What an owner still has to do, in order:** merge → deploy (migration 157 +> applies automatically) → flip `CRM_AUTO_LEAD` (§6 (b)). The first ON-state +> run on each mailbox activates the cursor and mints nothing; leads start +> appearing from mail that arrives after that moment. + **The hook is `process_new_mail(account_id)` — `routes/email/scheduler_hooks.py:57`.** It is the shared new-mail pipeline (rules → sweep → categorize senders → classify threads → auto-archive) and it is the single entry point *however mail arrived*: the background diff --git a/ai-company-brain/work_plan.md b/ai-company-brain/work_plan.md index dea849d8c..e4980a0d7 100644 --- a/ai-company-brain/work_plan.md +++ b/ai-company-brain/work_plan.md @@ -147,7 +147,7 @@ looks. Full statements live in `FOUNDATION_BUILDOUT_CHECKLIST.md`. | WS-20 | **WhatsApp activation + remainder** (search UI 🟢 AGENT-SAFE; OCR needs a vision-tier decision; Odoo/Zoho-bound items blocked) | `whatsapp_message_manager.md` §11 (header fixed 2026-08-01) | 🟡 owner | **OWNER-GATE:** Meta env/app review, enrichment cost flags. | | WS-21 | **Calendar F2/F3** (`gtd_time_blocks`, email windows, mobile timeline, external sync) | `calendar_focus_os.md` **§9** (canonical for all F2/F3 acceptance; **§5** canonical for `gtd_time_blocks`) + `calendar_timeboxing.md` **§13** (canonical for P4) — both rewritten 2026-08-03 | 🟡 partial | **Re-audited 2026-08-03 → GO-NARROWED.** P3 roll-over was already shipped (released-to-unscheduled, mig 78 + `start_auto_rollover`). ~~"ideal week"~~ **struck — substantially shipped** (mig 98 + settings round-trip + editor + grid render + packer honouring + 2 unit tests); only the unused-focus-window / template-adherence gap remains (§9.6). **Breaks-in-the-packer SHIPPED 2026-07-23** (`80722e17`, mig **97**) as *packer geometry* — a widened buffer plus lunch protection, **a gap, not a `kind='break'` row**, which is exactly why F2 survives (§5 residual 4, now closed). **The 2026-08-01 acceptance was satisfiable by doing nothing** — 2 of its 3 `gtd_time_blocks` clauses were already green against shipped code; they are deleted and replaced with four that all fail today. **`gtd_time_blocks` is 4 slices, not 1 PR** (§9.1 S1–S4): the "non-breaking `TimeBlock[]` swap" claim was **FALSE** — the measured blast radius is 17 TS files + 3 gateway modules + `apps/skills/skill-task-gtd/` + `apps/agents/agent-task-manager/`. **Focus Shield is AGENT-SAFE, not owner-gated** (§9.5) — it needs a design, not a credential; do not dispatch on §4.1 prose alone. **Top-5 outcomes (Horizons) — DO NOT DISPATCH:** it collides with WS-18; §4 assigns it here, and WS-18's title keeps it struck. **Verify by naming test files — never `pytest tests/unit -k calendar`**: `-k` still collects the whole directory, and whole-directory collection hangs on the Windows box. **Dispatchable today:** §9.1 S1 · the ritual-stamp localStorage residue (§9.1 done-when 4, independently shippable) · §9.6 · §9.7. **OWNER-GATE:** external sync (§9.11 / timeboxing §13 P4) needs Google Calendar and/or Microsoft Graph OAuth client credentials provisioned on the VPS. | | WS-22 | **draw.io** (all 13 tickets open, nothing built) | `drawio_integration.md` | 🟡 owner | Best acceptance structure in the corpus; needs an owner and re-verified anchors (~5 weeks stale). ST-DRW-02 is a decision gate. | -| **WS-26** | **CRM app — native CRM + Zoho retirement** *(minted 2026-08-05)* | `specs/crm_app.md` | ✅ **a + b + c BUILT + DEPLOYED** · ✅ **d read half BUILT + DEPLOYED** · ✅ **D2 = d-email BUILT 2026-08-07** (branch `ws-26d-email-timeline`, merged) · ✅ **D4 = d-write MERGED + DEPLOYED 2026-08-08 (PR #400, no migration; deploy 31217978773 log-verified)** · 🟢 **d-autolead dispatchable** · ✅ **D1 = f BUILT 2026-08-07 (branch `ws-26f-pipeline-truth`, NOT run against prod)** · ✅ **D3 = g BUILT 2026-08-07 (branch `ws-26g-reports`, no migration)** · 🟢 **DEMO CRITICAL PATH (owner-directed 2026-08-07, spec §9.0): ~~D1 f~~ (∥ D2 d-email) → ~~D3 g~~ → ~~D4 d-write~~ → D5 d-autolead** · 🟡 **h/i/e deferred past the demo; i spec-thin** | Research pass 2026-08-05: `frappe/crm` (AGPL — **concepts only, no code**), `trycompai/crm` (MIT), full-tree Zoho sweep. **Zoho today is a read-only nightly mirror** into the Phase-0 graph tables (`person`/`customer`/`deal`) with no UI, no write path, and **no Leads pull** — so leaving Zoho is import-and-retire, not a live cutover. Spine: Frappe's lead→convert→deal+contact+organization with **statuses-as-data** (color/position/type/probability); trycompai's single activity-spine table + `source` provenance + `last_activity_at` discipline. **BO-10 contribution: WS-26a adds the shared engine seam (`gateway/db.py::get_engine()`, tasks converted as proof) instead of engine 13.** Tickets: **a** schema + feature registration + core API — **BUILT 2026-08-05** (mig `144_crm.sql`, `feature:crm`, `gateway/db.py` seam + tasks converted, `routes/crm/`; **migration 144 applied on prod and `/crm` live as of 2026-08-06**) · **b** **Zoho two-way sync — BUILT 2026-08-05** (branch `ws-26b-zoho-sync`: `list_leads` + `list_deleted` on the read client, the single write client `ingestion/sources/zoho/writer.py` with one grep-asserted caller, mig `145_crm_zoho_sync.sql` (dirty columns + `crm_zoho_tombstones` + `crm_sync_cursors`), `routes/crm/{import_zoho,sync_zoho,broker_handlers}.py`, `crm.zoho_*` broker handlers registered from `main.py`, 80 new hermetic tests). *(Re-scoped 2026-08-05, owner-directed D-CRM-7: "faithful two way sync until we do away with Zoho entirely" — coexistence is bidirectional, not import-once.)* **Measured 2026-08-06: mig 145 is applied on prod and the BACKFILL HAS RUN — 737 orgs / 1,189 contacts / 1,516 leads / 551 deals / 1,909 notes, zero dirty rows, zero unmatched owners; the §7.1 pre-flip curl confirmed the tenant honors RFC-1123 `If-Modified-Since` (304). The PUSH direction has still never run: `CRM_ZOHO_SYNC` ships OFF, nothing has ever written the live Zoho tenant, and enabling the flag or hand-running a push cycle against prod stays OWNER-GATE §6.** WS-1's "no Zoho write path anywhere" clause was corrected in the same change (done-when 6) · **c** UI + the API addendum — **BUILT 2026-08-05** on branch `ws-26c-crm-ui` atop 26a and **merged with b into `ws-26-crm-app` 2026-08-06** (`/crm` app + BFF proxy; the three frontend registration points with `CenterApp` re-typed so `live ⇒ href` is a compile error; `routes/crm/deal_contacts.py` with one-primary-per-deal enforced on the shared `core.link_deal_contact` seam the convert path now also uses — 26b's importer is the one excepted writer and computes `is_primary` in-statement so a backfill can never demote a hand-set primary; `organization_name` on the deal list + board via a derived-table LEFT JOIN; the three review residuals — `?status_id` on a pipeline-less entity → 422, explicit `null` on a defaulted NOT NULL column → 422 not a driver 500, and a hand-edited `lead_name` surviving a name-field PATCH. **Deployed:** migrations 144 and 145 are applied on prod as of 2026-08-06 and `/crm` is live, so live rendering, drag persistence and deep links are owner-verifiable now) · **d** integrations — **audited 2026-08-06 GO-NARROWED and the narrowed slice is BUILT** (branch `ws-26d-agent-crm`): `apps/agents/agent-crm/` (`crm-assistant`, MAF, four READ tools over the existing `/crm` routes carrying the caller's `X-User-Email`, read-only enforced at the transport by a GET-only method allowlist) registered in `_KNOWN_AGENTS` + `_AGENT_REGISTRY` + `agent_registry.json`, plus `"crm"` added to the WhatsApp `_KNOWN_SYSTEMS` allowlist **parse-only** (nothing writes `wa_contacts.entity_ref`, the `crm` context block stays `None`, both pinned by test). **The three held-back items are now DISPATCHABLE — their doc blockers (B3/B4/B5/B7) were closed 2026-08-06 in `crm_app.md` §9.1-§9.3, every anchor read off `origin/main` rather than recalled:** **WS-26d-email** (the timeline join is CALLER-scoped, never record-scoped — it reuses the email app's `_account_scope` predicate, copied into `routes/crm/` rather than imported per D-CRM-4, joins by thread not message, inbound `from_address` only, and needs a new address index at the next free migration number) · **WS-26d-autolead** (hook = `routes/email/scheduler_hooks.py::process_new_mail`, the one seam scheduler+manual+webhook all funnel through; the per-message rules loop was considered and REJECTED because a classifier outage there double-fires and history backfills never reach it; unknown-sender test mirrors `_maybe_block_cold`, colleague suppression via `is_own_mail`) · **WS-26d-write — BUILT 2026-08-08** (branch `ws-26d-write`, **no migration**: every route the four tools call already existed). `request_confirmation` awaited at the top of each tool before any mutating request is built, fail-closed, and the `non_interactive_default` keyword is asserted ABSENT from the whole module rather than asserted != "approve" — pinning the argument rather than the value means a mutant does not get to pick a spelling the fence has not heard of. `_ALLOWED_METHODS` **widened, never deleted**: `{GET, POST, PATCH}`, still checked inside `_request`, with `DELETE`/`PUT` and any `_delete`/`_put` helper still absent, so the check that used to enforce "read-only" now enforces "never destroys". Path fence extended past `ast.JoinedStr` to `.format`/`%`/`+` (the re-review's P2) and — the part that makes it maintainable — **tested against synthetic sources one per idiom**, so "the fence went blind" is a red test rather than a silent gap. Two supervisor rulings landed as built: `update_deal_status` resolves the stage BY NAME inside the tool against `GET /crm/statuses/deal` (no UUID on the LLM surface; an unknown name returns the real lane names), and a lost-type target requires a `lost_reason` resolved the same way against `GET /crm/lost-reasons` — pre-empting the 422 the "close this as lost" demo beat would otherwise hit — with the vocabulary **only ever read, never created**. `create_lead` takes **no `owner_email` argument at all** (the route derives it from the acting user), deleting an LLM-filled identity field from the surface entirely. ⚠️ **One recorded departure from done-when 1**: the invariant asserted is *no mutation before consent*, not *no HTTP before consent* — two tools must read to describe honestly what they are about to do, and every pre-card call being a GET is itself pinned; the two tools that owe nothing to a pre-read are still held to literally zero calls. `@_annotate_risk` is the shared annotation convention and is NOT enforcement; the Action Broker covers the Zoho push, not the native write — the two fail in OPPOSITE directions and are not interchangeable. 76 new hermetic cases + `test_crm_agent.py` 87 → 143; ten mutants run red and reverted. **Built, not deployed.** The push-queue question is CLOSED — **D-CRM-9 (owner, 2026-08-06): agent-originated writes queue for Zoho exactly like human ones** (🟡 remainder; the flip stays OWNER-GATE) · **e** cutover + retirement inventory + **Zoho refresh-token revoke, which executes part of WS-2's standing P0** (🔴 OWNER-GATE end-to-end). Data-visibility departure recorded: org-visible to `feature:crm` holders in v1, `owner_email` is assignment not ACL (D-CRM-3; workflows v1 is the precedent) — revisit at WS-14 `group:` grants / colleague #1. **Pipeline blueprint added 2026-08-07 (spec §5.1) after the owner's first live board session found lanes out of order and imported stages at 0% probability — root cause: the importer appends unseen Zoho stages past the seeds at probability 0, and 144's seeds renamed Zoho's defaults so name-match missed them; the admin API that could fix it is headless.** New tickets: **f** pipeline truth + settings UI — **BUILT 2026-08-07** (branch `ws-26f-pipeline-truth`, **no migration**: 144 already carried `position`, `probability`, `type`, `closed_at` and `expected_close_date`). `routes/crm/stage_metadata.py` = `POST /crm/import/zoho/stages`, floor `admin:access:manage`, **dry-run by default** and `?apply=true` to write; **>1 pipeline STOPS before the DB is opened** (D-CRM-11); f4's `closed_at` proxy is a direct UPDATE that bypasses `mark_dirty_on_update`, asserted statically against the statement text AND the module's call graph because the shared fake writes only what a SET clause names. Two settings readers on the Zoho read client with `ZohoScopeError`/`ZohoApiVersionError` so **no-scope, no-data and no-such-endpoint are three different reported outcomes** — and no-scope is the *expected* first answer, since the tenant's refresh token was never minted with `ZohoCRM.settings.*`. D-CRM-10's clamp landed in `admin.py::_validate_status` reading the ROW rather than the payload (a PATCH naming only `type`, or only `probability`, contradicts the rule only in combination with what is stored). `?tab=settings` is the headless-API fix and needs no Zoho token at all. **Nothing has been run against the tenant — dry run included.** · **g** forecast & funnel reports off `crm_status_changes` — **BUILT 2026-08-07** (branch `ws-26g-reports`, **no migration**: 144's `crm_status_changes` already carried every column). `routes/crm/reports.py` = four read-only endpoints (`/crm/reports/{pipeline,funnel,win-loss,owners}`) on the shared gated router, plus `?tab=reports`. **`WEIGHTED_SQL` lifted into `core.py`** (pipeline re-exports it) so the forecast formula has ONE definition, and `core.status_wire` absorbed two duplicate status projections instead of gaining a third. **The cross-language parity mechanism is minted here, not inherited** — the `priority.ts ⟷ priority.py` "precedent" is two hand-kept mirrors joined by a comment with no shared fixture anywhere: `tests/fixtures/crm_weighted_parity.json` (new directory) is read by BOTH pytest (through the emitted SQL, whose expression `_crm_fakes._WEIGHTED_SUM_RE` parses out of the statement text) and vitest (through `board.ts::weightedDeal`/`weightedRows`). ⚠️ **The funnel is defined against what the log RECORDS, not what its name suggests**: `crm_status_changes` logs transitions only, so all 551 imported deals have zero rows and "entered" is a VISITED-SET union (`from_status`, `to_status`, and the deal's current stage) — a `to_status` count reports an empty funnel for the whole live board; dwell is grouped by **`from_status`** (the stage being LEFT, the opposite key from the naive reading); the log stores NAMES, so a renamed lane orphans its history and orphans are REPORTED in `unmatched`, never dropped; and NULL `closed_at` — every imported closed deal until f4's owner-gated backfill runs — falls outside the trailing window, with the count reported so a 0% win rate is explicable rather than mysterious. The lost-reason breakdown carries a NAMED unattributed bucket: the earlier "complete by construction" claim is FALSE, since the importer bypasses both gates and `lost_reason_id` is `ON DELETE SET NULL`. **No `GROUP BY` is emitted** — the ticket asked for the choice to be stated: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would stop BEING the expression the fixture and the fake read. 47 hermetic cases + 20 vitest + the 14-row shared fixture on both sides, 5 mutants red. **Two findings:** the `entity_type='deal'` mutant initially SURVIVED (the funnel keys through deal ids, so a lead row is excluded twice over) — closed with a row stamped `lead` against a DEAL's id, which is realistic precisely because `entity_id` has **no FK**; and `_crm_fakes`' `lower(col) = :param` reader matched a NULL column, which SQL never does, making the unassigned-owner bucket count rows its own aggregate never summed · **h** stage entry-requirements + rot badges (after f2) · **i** merge/bulk/CSV/saved-views (🟡 spec-thin, audit-narrow first). **Demo path (2026-08-07, spec §9.0): full chain + all gates intact, tickets re-sequenced not thinned; f gained f4 — imported won/lost deals have no `closed_at` (importer never stamps it), backfilled from Zoho `Closing_Date` as a labeled proxy via a direct UPDATE that MUST bypass dirty-marking or ~500 no-op pushes queue for the live tenant.** | +| **WS-26** | **CRM app — native CRM + Zoho retirement** *(minted 2026-08-05)* | `specs/crm_app.md` | ✅ **a + b + c BUILT + DEPLOYED** · ✅ **d read half BUILT + DEPLOYED** · ✅ **D2 = d-email BUILT 2026-08-07** (branch `ws-26d-email-timeline`, merged) · ✅ **D4 = d-write MERGED + DEPLOYED 2026-08-08 (PR #400, no migration; deploy 31217978773 log-verified)** · ✅ **D5 = d-autolead BUILT 2026-08-08 (branch `ws-26d-autolead`, migration 157; flag OFF, NOT flipped, NOT deployed)** · ✅ **D1 = f BUILT 2026-08-07 (branch `ws-26f-pipeline-truth`, NOT run against prod)** · ✅ **D3 = g BUILT 2026-08-07 (branch `ws-26g-reports`, no migration)** · 🟢 **DEMO CRITICAL PATH COMPLETE (owner-directed 2026-08-07, spec §9.0): ~~D1 f~~ (∥ D2 d-email) → ~~D3 g~~ → ~~D4 d-write~~ → ~~D5 d-autolead~~** · 🟡 **h/i/e deferred past the demo; i spec-thin** | Research pass 2026-08-05: `frappe/crm` (AGPL — **concepts only, no code**), `trycompai/crm` (MIT), full-tree Zoho sweep. **Zoho today is a read-only nightly mirror** into the Phase-0 graph tables (`person`/`customer`/`deal`) with no UI, no write path, and **no Leads pull** — so leaving Zoho is import-and-retire, not a live cutover. Spine: Frappe's lead→convert→deal+contact+organization with **statuses-as-data** (color/position/type/probability); trycompai's single activity-spine table + `source` provenance + `last_activity_at` discipline. **BO-10 contribution: WS-26a adds the shared engine seam (`gateway/db.py::get_engine()`, tasks converted as proof) instead of engine 13.** Tickets: **a** schema + feature registration + core API — **BUILT 2026-08-05** (mig `144_crm.sql`, `feature:crm`, `gateway/db.py` seam + tasks converted, `routes/crm/`; **migration 144 applied on prod and `/crm` live as of 2026-08-06**) · **b** **Zoho two-way sync — BUILT 2026-08-05** (branch `ws-26b-zoho-sync`: `list_leads` + `list_deleted` on the read client, the single write client `ingestion/sources/zoho/writer.py` with one grep-asserted caller, mig `145_crm_zoho_sync.sql` (dirty columns + `crm_zoho_tombstones` + `crm_sync_cursors`), `routes/crm/{import_zoho,sync_zoho,broker_handlers}.py`, `crm.zoho_*` broker handlers registered from `main.py`, 80 new hermetic tests). *(Re-scoped 2026-08-05, owner-directed D-CRM-7: "faithful two way sync until we do away with Zoho entirely" — coexistence is bidirectional, not import-once.)* **Measured 2026-08-06: mig 145 is applied on prod and the BACKFILL HAS RUN — 737 orgs / 1,189 contacts / 1,516 leads / 551 deals / 1,909 notes, zero dirty rows, zero unmatched owners; the §7.1 pre-flip curl confirmed the tenant honors RFC-1123 `If-Modified-Since` (304). The PUSH direction has still never run: `CRM_ZOHO_SYNC` ships OFF, nothing has ever written the live Zoho tenant, and enabling the flag or hand-running a push cycle against prod stays OWNER-GATE §6.** WS-1's "no Zoho write path anywhere" clause was corrected in the same change (done-when 6) · **c** UI + the API addendum — **BUILT 2026-08-05** on branch `ws-26c-crm-ui` atop 26a and **merged with b into `ws-26-crm-app` 2026-08-06** (`/crm` app + BFF proxy; the three frontend registration points with `CenterApp` re-typed so `live ⇒ href` is a compile error; `routes/crm/deal_contacts.py` with one-primary-per-deal enforced on the shared `core.link_deal_contact` seam the convert path now also uses — 26b's importer is the one excepted writer and computes `is_primary` in-statement so a backfill can never demote a hand-set primary; `organization_name` on the deal list + board via a derived-table LEFT JOIN; the three review residuals — `?status_id` on a pipeline-less entity → 422, explicit `null` on a defaulted NOT NULL column → 422 not a driver 500, and a hand-edited `lead_name` surviving a name-field PATCH. **Deployed:** migrations 144 and 145 are applied on prod as of 2026-08-06 and `/crm` is live, so live rendering, drag persistence and deep links are owner-verifiable now) · **d** integrations — **audited 2026-08-06 GO-NARROWED and the narrowed slice is BUILT** (branch `ws-26d-agent-crm`): `apps/agents/agent-crm/` (`crm-assistant`, MAF, four READ tools over the existing `/crm` routes carrying the caller's `X-User-Email`, read-only enforced at the transport by a GET-only method allowlist) registered in `_KNOWN_AGENTS` + `_AGENT_REGISTRY` + `agent_registry.json`, plus `"crm"` added to the WhatsApp `_KNOWN_SYSTEMS` allowlist **parse-only** (nothing writes `wa_contacts.entity_ref`, the `crm` context block stays `None`, both pinned by test). **The three held-back items are now DISPATCHABLE — their doc blockers (B3/B4/B5/B7) were closed 2026-08-06 in `crm_app.md` §9.1-§9.3, every anchor read off `origin/main` rather than recalled:** **WS-26d-email** (the timeline join is CALLER-scoped, never record-scoped — it reuses the email app's `_account_scope` predicate, copied into `routes/crm/` rather than imported per D-CRM-4, joins by thread not message, inbound `from_address` only, and needs a new address index at the next free migration number) · **WS-26d-autolead — BUILT 2026-08-08** (branch `ws-26d-autolead`, migration **157** `crm_auto_lead_cursors`). Hook = `routes/email/scheduler_hooks.py::process_new_mail`, the one seam scheduler+manual+webhook all funnel through; the per-message rules loop was considered and REJECTED because a classifier outage there double-fires and history backfills never reach it. ⚠️ **That same seam is ALSO reached by deep resyncs** (2026-08-08 audit blocker G1, closed by PR #402 before the build), which is why the step keeps a per-account TWO-timestamp cursor: `activated_at`, stamped once and never advanced, makes `received_at > activated_at` the backfill discriminator, and `processed_watermark` is the incremental one. Without the first, connecting a second mailbox mints a lead per unknown sender across a year of mail, each queued for the live tenant. Unknown-sender check mirrors `_maybe_block_cold`'s two steps and adds a third (no `crm_contacts`/`crm_leads` row with that `lower(email)`) — **the ticket's original `ON CONFLICT DO NOTHING` could not have fired** (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so dedup is that SELECT guard plus in-batch de-duplication, with the cross-invocation race accepted and the UNIQUE index refused. Colleague suppression is TWO gates: `sender_scope` (which fails SAFE to "external", the wrong direction here) and the normalised internal-domain list. The originating message is logged `type='system'` — outside `sync_zoho`'s `type IN ('note','task')` push predicate — with subject + sender in `meta` and **`body` empty**, so the mail's content never leaves the native CRM even though the lead does. The lead goes through `records.create_record`, never raw SQL. **Built, flag `False`, NOT flipped, NOT deployed; the flip stays §6 (b).** 52 hermetic cases; 7 mutants red and reverted · **WS-26d-write — BUILT 2026-08-08** (branch `ws-26d-write`, **no migration**: every route the four tools call already existed). `request_confirmation` awaited at the top of each tool before any mutating request is built, fail-closed, and the `non_interactive_default` keyword is asserted ABSENT from the whole module rather than asserted != "approve" — pinning the argument rather than the value means a mutant does not get to pick a spelling the fence has not heard of. `_ALLOWED_METHODS` **widened, never deleted**: `{GET, POST, PATCH}`, still checked inside `_request`, with `DELETE`/`PUT` and any `_delete`/`_put` helper still absent, so the check that used to enforce "read-only" now enforces "never destroys". Path fence extended past `ast.JoinedStr` to `.format`/`%`/`+` (the re-review's P2) and — the part that makes it maintainable — **tested against synthetic sources one per idiom**, so "the fence went blind" is a red test rather than a silent gap. Two supervisor rulings landed as built: `update_deal_status` resolves the stage BY NAME inside the tool against `GET /crm/statuses/deal` (no UUID on the LLM surface; an unknown name returns the real lane names), and a lost-type target requires a `lost_reason` resolved the same way against `GET /crm/lost-reasons` — pre-empting the 422 the "close this as lost" demo beat would otherwise hit — with the vocabulary **only ever read, never created**. `create_lead` takes **no `owner_email` argument at all** (the route derives it from the acting user), deleting an LLM-filled identity field from the surface entirely. ⚠️ **One recorded departure from done-when 1**: the invariant asserted is *no mutation before consent*, not *no HTTP before consent* — two tools must read to describe honestly what they are about to do, and every pre-card call being a GET is itself pinned; the two tools that owe nothing to a pre-read are still held to literally zero calls. `@_annotate_risk` is the shared annotation convention and is NOT enforcement; the Action Broker covers the Zoho push, not the native write — the two fail in OPPOSITE directions and are not interchangeable. 76 new hermetic cases + `test_crm_agent.py` 87 → 143; ten mutants run red and reverted. **Built, not deployed.** The push-queue question is CLOSED — **D-CRM-9 (owner, 2026-08-06): agent-originated writes queue for Zoho exactly like human ones** (🟡 remainder; the flip stays OWNER-GATE) · **e** cutover + retirement inventory + **Zoho refresh-token revoke, which executes part of WS-2's standing P0** (🔴 OWNER-GATE end-to-end). Data-visibility departure recorded: org-visible to `feature:crm` holders in v1, `owner_email` is assignment not ACL (D-CRM-3; workflows v1 is the precedent) — revisit at WS-14 `group:` grants / colleague #1. **Pipeline blueprint added 2026-08-07 (spec §5.1) after the owner's first live board session found lanes out of order and imported stages at 0% probability — root cause: the importer appends unseen Zoho stages past the seeds at probability 0, and 144's seeds renamed Zoho's defaults so name-match missed them; the admin API that could fix it is headless.** New tickets: **f** pipeline truth + settings UI — **BUILT 2026-08-07** (branch `ws-26f-pipeline-truth`, **no migration**: 144 already carried `position`, `probability`, `type`, `closed_at` and `expected_close_date`). `routes/crm/stage_metadata.py` = `POST /crm/import/zoho/stages`, floor `admin:access:manage`, **dry-run by default** and `?apply=true` to write; **>1 pipeline STOPS before the DB is opened** (D-CRM-11); f4's `closed_at` proxy is a direct UPDATE that bypasses `mark_dirty_on_update`, asserted statically against the statement text AND the module's call graph because the shared fake writes only what a SET clause names. Two settings readers on the Zoho read client with `ZohoScopeError`/`ZohoApiVersionError` so **no-scope, no-data and no-such-endpoint are three different reported outcomes** — and no-scope is the *expected* first answer, since the tenant's refresh token was never minted with `ZohoCRM.settings.*`. D-CRM-10's clamp landed in `admin.py::_validate_status` reading the ROW rather than the payload (a PATCH naming only `type`, or only `probability`, contradicts the rule only in combination with what is stored). `?tab=settings` is the headless-API fix and needs no Zoho token at all. **Nothing has been run against the tenant — dry run included.** · **g** forecast & funnel reports off `crm_status_changes` — **BUILT 2026-08-07** (branch `ws-26g-reports`, **no migration**: 144's `crm_status_changes` already carried every column). `routes/crm/reports.py` = four read-only endpoints (`/crm/reports/{pipeline,funnel,win-loss,owners}`) on the shared gated router, plus `?tab=reports`. **`WEIGHTED_SQL` lifted into `core.py`** (pipeline re-exports it) so the forecast formula has ONE definition, and `core.status_wire` absorbed two duplicate status projections instead of gaining a third. **The cross-language parity mechanism is minted here, not inherited** — the `priority.ts ⟷ priority.py` "precedent" is two hand-kept mirrors joined by a comment with no shared fixture anywhere: `tests/fixtures/crm_weighted_parity.json` (new directory) is read by BOTH pytest (through the emitted SQL, whose expression `_crm_fakes._WEIGHTED_SUM_RE` parses out of the statement text) and vitest (through `board.ts::weightedDeal`/`weightedRows`). ⚠️ **The funnel is defined against what the log RECORDS, not what its name suggests**: `crm_status_changes` logs transitions only, so all 551 imported deals have zero rows and "entered" is a VISITED-SET union (`from_status`, `to_status`, and the deal's current stage) — a `to_status` count reports an empty funnel for the whole live board; dwell is grouped by **`from_status`** (the stage being LEFT, the opposite key from the naive reading); the log stores NAMES, so a renamed lane orphans its history and orphans are REPORTED in `unmatched`, never dropped; and NULL `closed_at` — every imported closed deal until f4's owner-gated backfill runs — falls outside the trailing window, with the count reported so a 0% win rate is explicable rather than mysterious. The lost-reason breakdown carries a NAMED unattributed bucket: the earlier "complete by construction" claim is FALSE, since the importer bypasses both gates and `lost_reason_id` is `ON DELETE SET NULL`. **No `GROUP BY` is emitted** — the ticket asked for the choice to be stated: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would stop BEING the expression the fixture and the fake read. 47 hermetic cases + 20 vitest + the 14-row shared fixture on both sides, 5 mutants red. **Two findings:** the `entity_type='deal'` mutant initially SURVIVED (the funnel keys through deal ids, so a lead row is excluded twice over) — closed with a row stamped `lead` against a DEAL's id, which is realistic precisely because `entity_id` has **no FK**; and `_crm_fakes`' `lower(col) = :param` reader matched a NULL column, which SQL never does, making the unassigned-owner bucket count rows its own aggregate never summed · **h** stage entry-requirements + rot badges (after f2) · **i** merge/bulk/CSV/saved-views (🟡 spec-thin, audit-narrow first). **Demo path (2026-08-07, spec §9.0): full chain + all gates intact, tickets re-sequenced not thinned; f gained f4 — imported won/lost deals have no `closed_at` (importer never stamps it), backfilled from Zoho `Closing_Date` as a labeled proxy via a direct UPDATE that MUST bypass dirty-marking or ~500 no-op pushes queue for the live tenant.** | | **WS-27** | **Projects app — native project management + ClickUp retirement** *(minted 2026-08-05)* | `specs/project_management_app.md` | ✅ **a + b + d + e + i BUILT 2026-08-06 · j + k + l + m + n BUILT 2026-08-07** · 🟢 f dispatchable · 🟡 c gated · 🟡 h sequenced | Research pass 2026-08-05: `Paca-AI/paca` v0.11.0 (Apache-2.0 — **patterns adopted, no code translated**; findings + the adopt/adapt/refuse table live in `specs/paca_pm_research_2026-08.md`, reference-only), plus a full-tree ClickUp sweep. **ClickUp today is TWO independent systems** — the Phase-0 graph mirror (read-only, shallow) *and* the per-user Tasks-app connector with a **live broker-gated write path** — so leaving ClickUp is coexistence-sync-then-invert, **not** WS-26's import-and-retire; the constraint-8 inversion is staged and recorded in spec §7. Spine: Paca's two-self-FK hierarchy (departments→projects→subprojects→tasks→subtasks as `pm_projects` + `pm_tasks`, types-as-data with the Epic-root rule), statuses-as-data with a semantic `category` (D-CRM-2 convergence), per-view fractional ordering (`pm_view_task_positions` — what lets People-Center and Center-slice boards order the same task differently), and a single activity spine. **First data-scoped app:** `pm_project_grants` on the shipped `email\|group:\|org` vocabulary (D12; sibling of C1's D13, which is unchanged), 404-not-403, and the full-portfolio view gives D14's zero-consumer `data:org:read` its **first consumer**. **Three owner answers recorded 2026-08-06 as D-PM-8/9/10, and two of them changed the build:** **D-PM-8** no portfolio/program layer — grants are the only grouping axis, a cross-department project simply carries several (a `pm_programs` table stays purely additive if wanted later); **D-PM-9** agent edits to ClickUp-linked tasks are treated **exactly like human edits** (*the agent proposed queueing agent-originated pushes for approval and was overruled*) — so during coexistence a mistaken agent edit reaches the live workspace with no human in between while `ACTION_BROKER_ENFORCE` is off; bounded by attribution (`agent:`), timeline-reversibility, and the fact that the enforce flip converts the whole class to queue-on-approval. Read D-PM-9's Cost paragraph before building WS-27f; **D-PM-10** ClickUp Spaces map to Centers **explicitly**, from agent-proposed suggestions (assignee-overlap → name match → EVAL-LOCKED content classification), owner-confirmed, applied as `group:` grants — and an **unmapped Space still imports in full with no group grant**, staying reachable in `/projects` for `data:org:read` holders and its assignees. This supersedes the "pilot vs all Spaces" framing: scope is now a per-Space decision the plan step surfaces, so a pilot and a full import are one code path. Tickets: **a** schema + `feature:projects` both sides + core API on the `gateway/db.py` seam — **BUILT 2026-08-06** (mig `146_projects.sql`, `routes/projects/` with zero `create_async_engine` calls, 115 hermetic cases + 5 mutants measured red; **not deployed — the migration has not been applied anywhere**) · **b** ClickUp org importer **+ the Space→Center mapping plan** — **BUILT 2026-08-06** (`routes/projects/mapping.py` + `import_clickup.py`; `POST /projects/import/clickup/plan` proposes and writes nothing, `POST /projects/import/clickup` applies the confirmed mapping; 25 hermetic cases, 4 mutants red incl. *applying the suggestion instead of the confirmed mapping*; **neither endpoint has been run — prod execution stays OWNER-GATE, §6**) · **c** two-way coexistence sync — three-way field merge, conflicts logged to the timeline (🟡 **blocked on WS-1's BO-1a + BO-1b, named prerequisites**; enabling push is OWNER-GATE) · **d** UI + Center (app + scope) projections, no forks — **BUILT 2026-08-06** (`src/app/projects/` tree + board + list + task panel + timeline, BFF proxy, nav/access registration, all six Centers linking at the SAME `/projects` path and differing only by `?center=`; 34 vitest cases incl. a registration fence, 6 mutants red) · **authoring landed 2026-08-06** — d shipped a UI that could read and drag but never **create**, so a member could only work with rows a ClickUp import had put there: new department / subproject (from the node, where the parent already is), new task (status not sent — the API picks the project's default), subtask from the panel, and **assignees as chips where an agent and a person share one field** (`lib/assignees.ts`, 17 vitest cases, 7 mutants red). That last one is where D-PM-4 stops being a schema note and is the precondition for WS-27f's dispatch being reachable at all · **e** the personal lens — **BUILT 2026-08-06** and **its shape changed**: `D-PM-6` was revised (owner-directed — *"the personal task manager should be a proper extension … a cohesive whole"*) from a mirror into **one store**. `pm_tasks` is THE task table; private work is a personal project (`pm_projects.personal_owner`); the GTD overlay is **per-member** (`pm_task_personal`, mig `147_projects_personal.sql`) so two assignees can hold different dispositions. Assignment is no longer a sync — the inbox row IS the project row, and completing it there moves the shared status. 31 hermetic cases, 6 mutants red. **Its surface landed the same day** — "My work" above the project tree in the SAME app (`components/MyWork.tsx` + `lib/mywork.ts`, 17 vitest cases, 7 mutants red): capture-first, four work lanes that render even when empty, untriaged counted in the header (the Weekly Review's question, answerable only because dispositions are derived not stored), and a completion checkbox that moves the **shared** status. e had shipped API-only, so the cohesion the revision bought was true in the schema and invisible to a member. One repair it forced: `TaskPanel` read the *selected* project's statuses, wrong for a task opened from My work — now resolved from the task's own root project. **Cost accepted: `gtd_items` becomes legacy and WS-27h retires it** · **f** automation + agent dispatch — **BUILT 2026-08-06** (`routes/projects/automation.py` + `agent_dispatch.py`, the `pm_task` node type in `workflows/engine/`, `PM_EVENT_TOPICS` served by the catalog; 34 hermetic cases, 10 mutants red). Both halves of `workflows_app.md` §13 — **U1** the task-mutation node and **U7** dispatch. The engine imports a transport-free SERVICE, not a route, so an automation's edit is indistinguishable in validation from a human PATCH and lands the same timeline row; **status is named, never keyed** (a graph pinned to one project's status UUID could only automate that project); a `pm_task` node is deliberately **not** write-class (the approval gate is for outward writes — now pinned by a test rather than true by accident); and "already in target state" is asserted to issue **no UPDATE at all**, because `update_row` stamps `updated_at` and a redundant write is invisible in a diff while making a task look freshly touched. Assignment dispatches from an event SINK beside the workflows dispatcher, so a broken agent cannot fail the act of assigning somebody a task, and the handoff activity is committed BEFORE the run starts. **Engine defect found and fixed:** `resolve_value` keeps an unresolvable `{{ref}}` as-is at run time by design and `{{trigger.missing}}` passes the publish gate because its *root* is legal — the literal would have reached Postgres as a would-be uuid and returned "Task not found", pointing the maker at the wrong thing · **i** attachments — **BUILT 2026-08-06** (mig `150_projects_attachments.sql`; 25 cases, 10 mutants red): one file store (`gtd_attachments` reused, upload rules imported) with a thin `pm_task_attachments` join that carries the ACCESS decision, so a file is readable by whoever can see the task rather than only its uploader; no attach-by-id endpoint exists, because naming an arbitrary attachment id would let a caller attach somebody else's private capture to their own task and read it back. **Caught before shipping:** the projects BFF proxy re-serialised every POST as JSON, so a multipart upload would have reached the gateway with NO FILE while still answering 201 · **j–r** the rest of the ClickUp-parity gap, measured against the built tree and sequenced in spec §11 (attachments, notifications/@mentions, filters+saved views, custom fields, tags, bulk edit, recurring, dependency UI, calendar, search) — all 🟢, and **n (bulk edit) gates g — and n is now BUILT (2026-08-07), so that dependency is SATISFIED**: an import that cannot be re-triaged in bulk is one somebody abandons halfway, leaving two live systems, which is the state the retirement exists to end. g itself stays 🔴 OWNER-GATE for its own reasons (§6); this changes the prerequisite, not the gate · **g** cutover + retirement inventory (both ClickUp systems) + token revoke + the root-`AGENTS.md` constraint-8 amendment (🔴 OWNER-GATE end-to-end) · **h** `gtd_items` retirement — the cost D-PM-6's revision accepted: union read, row migration into `pm_tasks` + `pm_task_personal`, then `items.py`'s 27 owner-scoped predicates retire with the table they scope (🟡 after e; the data move is 🔴 OWNER-GATE — it rewrites the owner's live task store).. **j BUILT 2026-08-07** (mig `152_projects_notifications.sql`, `routes/projects/notifications.py`, the header bell + mention picker; 39 hermetic + 27 vitest cases, 10 mutants red): closes §11.2's second gap — *"assignment is silent"*. **Three rules decide who hears**, each the whole reason for a rule: never the actor (a bell that pings you about your own click gets muted, and a muted bell notifies nobody about anything); never an agent (they are handed work by the WS-27f dispatch sink, so a row addressed to one sits unread forever inflating a badge nobody can clear — enforced in Python AND by a CHECK); and **never somebody who cannot open it**, which is the security property: the notification carries the task's TITLE, so delivering one outside the grant closure leaks it and lands them on a 404. That third rule needed `resolve_visibility_for`, which answers for a THIRD PARTY — `resolve_visibility` reads a `UserContext` and the recipient of a mention has no request in flight — by reading the tables `/auth/me` reads and handing them to the **real** `build_access`, so wildcards and allow/deny overrides resolve identically on both paths. Notifications are written **inside the transaction**, not emitted on the bus: `emit` swallows failures by construction so a broken workflow can never fail a task edit, which is right for agent dispatch and wrong here. A mention is an **address, not a name**, because 148 dropped `UNIQUE(name)` — `@Priya` has no answer. **Two bugs found on the way in, both shipping at the time:** every project-task file upload was answering **422** (`ACTIVITY_TYPES` never learned 150's `attachment`; all 25 attachment tests passed because they monkeypatch `record_activity` — the seam under test was mocked out), fixed with two tests that READ the migrations; and `/projects?task=` did nothing, though the People Center has linked there since WS-28b. **WS-27b's UI BUILT 2026-08-07** (`components/ImportClickUp.tsx`, `lib/importPlan.ts`; 18 vitest cases, 3 mutants red): the importer shipped with WS-27b and was **unreachable from the product** — the empty state named "import a ClickUp workspace" and no control anywhere did it, so a new install stayed empty and the only route to real data was curl. Three steps, only the last of which writes: Preview (`/plan`, reads the tenant) → Dry run (`dry_run:true`, exercises the flattening) → Import. **The mapping stays the owner's act (D-PM-10)** and the UI is built so it stays one: the suggestion is pre-filled and shown beside its confidence IN WORDS ("a guess — check it", not `0.45`, because a bare number invites acceptance without looking), and a CONFIRMED mapping always beats a fresh suggestion so a re-run never silently re-maps a Space somebody already ruled on. Unmapped Spaces are a notice rather than a blocker, matching the importer. ⚠️ The gate is unchanged and is now exactly one click: building this was agent-safe, pressing Import is the owner's act, and no agent has run either endpoint against production. **The Tasks-app mirror path BUILT 2026-08-07** (`routes/projects/import_tasks.py`, `POST /import/from-tasks`; 43 hermetic cases, 7 mutants red) — owner-directed: *"just show up all the data that is there in the Tasks app inside the Projects app"*. A SECOND importer rather than a flag: the ClickUp one needs a live token, spends LLM budget and demands a Center mapping BEFORE anything is written, which is backwards for "show me my work today". This reads `gtd_projects`/`gtd_items` — the mirror already on the box — so no API call, no token, no model spend, and it works when the connector is stale. One named department, with the real ClickUp shape beneath it — **Space → Folder → List** rebuilt as projects, each carrying its own `clickup_id`/`clickup_kind`, so promoting a Space node is how the department split happens later. **Verified against a real Postgres** (full migration set + seeded mirror), which found THREE defects the hermetic suite could not: `gtd_projects.space_id` is LOCAL-only so the Space was always NULL; `pm_projects` has no `clickup_snapshot` column, so the first real click would have 500'd (this shipped in #393 and was fixed before anybody pressed it); and the preview under-counted 4 vs 7 because container nodes were only tallied on the write path. The root IS org-granted (same act as `create_node`, narrower than bulk-granting a tenant). Pinned: nothing outside `pm_*` is written; only `source <> 'LOCAL'` rows are read (a personal capture must not be published to a shared board); the provider's own status names are kept; an orphaned task is COUNTED, not dropped. **Mutation found a real gap**: deleting `dry_run` from the write guards left every test green, because on a FIRST dry run `root_id is None` blocks the write anyway — on a SECOND one the department exists and `dry_run` is the only protection. That is the realistic case (preview → import → preview again) and it now has its own test. **k BUILT 2026-08-07** (`routes/projects/filters.py`, `lib/grouping.ts`, `components/FilterBar.tsx`; 34 hermetic + 24 vitest cases, 13 mutants red, and 23 checks against a REAL Postgres) — closes §11.2's third gap, the one whose name was the sentence *"my open bugs in Ops, grouped by assignee"*. **ONE filter builder serves both the list endpoint and saved views**, because a saved view is nothing but a stored set of these filters and two implementations would drift — a *saved* view showing a different set than the same filters typed by hand is the one thing it may not do, so a test compares the two outputs directly. **Every filter is a WHERE clause**: paging happens in SQL, so a filter applied in Python after `LIMIT` returns short pages, and *"page 2 is empty but there are 40 more"* is a bug people work around for months instead of reporting. **`overdue` means past due AND still open** — a finished task with a past due date is done, and permanent red is how a board teaches people to ignore red. **An unknown category is a 422 naming the five real ones**, not an empty board a client reads as "this project is empty". Unknown config **keys** are DROPPED while a bad **value** falls back, because those are different failures: a view is a preference written by an older client, so refusing one over an unrecognised key would make every deploy a migration of everybody's saved views, whereas rendering still has to produce something. On the board, **a task with two assignees appears in BOTH columns** (it is both people's work; picking one hides it from the other, so the header counts tasks not group sizes), **empty status lanes are kept** while every other grouping drops empties (a missing "In progress" column reads as "no such state", not "nothing in progress"), and **dragging is offered only when the columns are statuses** — a drop writes the field the columns represent, and status is the one that is a plain PATCH, so a card that can be dragged into a column which cannot accept it and snaps back is worse than an honestly static column. `toConfig` is deliberately NOT `toQuery`: a query string carries only text so `toQuery` writes `"true"`, and `fromConfig` refuses a string where a toggle belongs, so a view built from query shape would come back with every toggle silently cleared. The project's **order-bearing board is withheld from the chips** — `tree.py` seeds one `board` view per project and it owns every `pm_view_task_positions` row, so offering its ✕ would offer to delete every hand-arranged position; saved views sit at position 300 above the seeded pair and `orderBearingView` is one function used by both the drag handler and the delete guard. **A FIFTH live bug, found the same way as the previous four:** `due_before` was `CAST(:due_before AS timestamptz)` with the raw query-string value — asyncpg infers the parameter's type FROM that cast and then refuses to encode a `str`, so the query never reached Postgres and **`?due_before=…` answered 500** while the hermetic fake, which agrees with whatever SQL it is handed, stayed green. `parse_when` parses on this side and binds a real `datetime`; garbage is a 422 that says what was expected; a naive value is read as UTC rather than inheriting the connection's TimeZone. Two tests — one on the bound value's TYPE, one refusing any `CAST(:param AS timestamp…)` anywhere in the builder — so the next `after=` filter written the obvious way fails in CI instead of in front of a member. **l BUILT 2026-08-07** (mig `155_projects_custom_fields.sql`, `routes/projects/custom_fields.py`, `lib/customFields.ts` + the panel block and the Fields dialog; 47 hermetic + 36 vitest cases, 23 mutants red, 35 checks against a REAL Postgres) — ClickUp's signature feature, and the shape §5's non-goals already recorded as the additive path: **definitions in a table, values denormalised onto the task as JSONB keyed by `field_key`**. NOT a row per (task, field): that is the textbook EAV answer and costs a join per field on every board paint — five fields across two hundred imported tasks is a thousand rows to gather and re-pivot, per render — whereas the JSONB column arrives with the task for free. **The cost is stated rather than discovered**: a value is not referentially tied to its definition, so the DATABASE cannot stop a key no definition owns from being written, and that guarantee moves into Python. Hence the validation IS the feature: an unknown key is a **422 not a silent drop** (a typo that no-ops looks exactly like a save); a patch **MERGES** (a client that knows three of five fields must not wipe the other two — and an older client, or an automation written before a field existed, is precisely that client); an explicit **null CLEARS the key** rather than storing a null, because it is the only way to express "unset this" and a stored null makes "never filled in" and "deliberately emptied" one value in every filter; and **`true` is not the number 1** — `isinstance(True, int)` is True in Python, so the coercers are one-per-type in a dispatch dict specifically so the boolean check can never drift below the number one. **The deliberate departure from Paca:** its research notes record "deleting a definition does not clean task data" as an accepted cost; it is NOT accepted here, because a key left in the JSONB is invisible — no definition means no column, no form row, no filter — until somebody recreates the name and every old value resurfaces carrying the new meaning. The cleared count is reported (R7/R8). Two things a definition may not change once values exist, both because the stored values would stop meaning what they say: **`field_key` is never editable** (it is the identity every value is filed under) and **`field_type` is a 409 naming the count** (text→select cannot re-interpret what is already written); dropping a select option some task holds is refused the same way, adding one is free, and the UI shows the derived key while the name is still being typed since that is the last moment anybody can change it. **Custom fields are REVERTIBLE, which is what makes them first-class:** `patch_task` folds a custom edit into the SAME `field_change` activity under `custom.` rather than inventing an activity type — `record_activity` refuses a type the CHECK does not list, the trap that made every attachment upload answer 422 — and revert restores by **merging onto what the task holds NOW**, never writing back the whole object, since another field may have been edited since and replacing the blob would silently undo that too. **A bug the ticket's own tests caught before it shipped:** `changedValues` compared a form's boolean against a `null` baseline, so a task with an unanswered checkbox sent `open:false` on EVERY save and posted a timeline entry for an edit nobody made — a checkbox has no "unset" state to render, so `false` is its baseline. **And a fence that was quietly a subset check:** `test_projects_routes` asserted a LIST of mounted paths, which catches the module somebody remembered to add a path for; it now also reads the package directory and asserts every module declaring a `@router` route is imported by `__init__.py` — the C1 trap where a missing import mounts nothing while every direct-call test still passes. Verified by deleting the import and watching it fail. **m BUILT 2026-08-07** (mig `156_projects_tags.sql`, `routes/projects/tags.py`, `lib/tags.ts` + the panel picker, the filter row, a `tag` board axis and the Tags dialog; 31 hermetic + 37 vitest cases, 16 mutants red, 30 checks against a REAL Postgres) — the row the research notes left open ON PURPOSE: `paca_pm_research_2026-08.md` row 13 REFUSED Paca's model (*"a bare jsonb string array on tasks. No registry, no colors, no rename/merge — the weakest part of Paca's model"*) and §5 shipped `pm_tasks.tags TEXT[]` in its place with a registry named as additive later. **The array STAYS** — a join table would add a row per tag per task and a join to every board paint, to buy referential integrity this app enforces in one place, whereas the array arrives with the task and its GIN index (146) already answers "tagged X". What the registry buys: **one spelling per tag** (identity is case-INSENSITIVE via a unique index over `lower(name)` so two racing requests cannot create both, display is case-PRESERVING, and the task's array stores the REGISTRY's form — which is what makes "filter by bug finds all of it" true rather than aspirational, and what lets a rename be one statement); **rename**; **merge**; and a **colour**. **Applying an unregistered tag REGISTERS it** — refusing would make tagging a two-step errand (leave the task, create the tag, come back), which is how tagging gets abandoned, and an abandoned tag set is worse than a messy one. **The cost is stated: every typo becomes a tag** — which is exactly why merge is here and is not optional, and why the picker SHOWS the moment of creation rather than minting silently. **A rename onto an existing name is a 409, not a silent merge**: different operations, different outcomes, and one of them destroys a tag — quietly doing the destructive one because the names collided is what stops people using a rename button. **A task carrying BOTH tags ends a merge with the target ONCE** — the case that is easy to get wrong, and getting it wrong leaves a duplicate that renders twice and survives the next merge too; `merged_tags` is pure so that case is asserted directly, and the rewrite runs over affected rows in Python rather than as an `array_replace`, which would leave the duplicate. **TWO tag filters** because both questions get asked and neither answers the other: `tags` is ANY (`&&`) and `tags_all` is ALL (`@>`); collapsing them would silently pick a meaning, and with three tags the answers differ by almost everything. On the board a task with three tags appears in three columns, the same honesty as two assignees appearing in both theirs. **The migration backfills AND rewrites data, deliberately and narrowly:** `tags` has been on `pm_tasks` since 146 and the import path writes them, so an empty registry beside a tagged corpus means the first rename finds nothing; the winning display form is **the spelling people actually use** (most frequent, ties broken deterministically — `min()` alone would canonicalise 400 "Bug" to a single stray "BUG"), and task arrays are then made to agree, only ever swapping one CASING of a tag for another, with the count reported in a NOTICE. **A bug the live run caught in that block:** the canonicalisation used the implicit-comma `FROM pm_tasks t, unnest(t.tags) ... LEFT JOIN` form, where the LEFT JOIN binds only to `unnest(...)` and `t` is not in scope for its ON clause — the migration aborted with "invalid reference to FROM-clause entry for table t". Rewritten as `CROSS JOIN LATERAL … WITH ORDINALITY`, which also fixed a second problem the first version would have shipped: `array_agg(DISTINCT ...)` sorts by its own expression, so every task's tag list would have come back alphabetised. **n BUILT 2026-08-07** (`routes/projects/bulk.py` → `POST /projects/tasks/bulk`, `lib/selection.ts`, `components/BulkBar.tsx` + checkboxes on board and list; 35 hermetic + 32 vitest cases, 16 mutants red, 34 checks against a REAL Postgres; **no migration**) — **the ticket §11.3 names as gating g, so that prerequisite is now satisfied.** It **reuses `automation.apply_task_patch` rather than growing a second writer**: that service exists because WS-27f needed an edit indistinguishable in validation from a human PATCH, and a bulk endpoint with its own field handling would be a third opinion about what a task edit is. Tags go through the same registry the panel uses, because the registry cannot be true if bulk is a second door into the array. **Status is named, never keyed — load-bearing here rather than stylistic**: a selection spans projects and a status id belongs to one root, so `status_id` for fifty tasks across three projects puts two thirds of them in a lane that is not theirs; it is a 422 with its OWN message, because it is the mistake somebody makes by copying a single-task PATCH body and "unknown field" would not explain why the thing that works on one task is refused on fifty. **Assignees and tags are ADD/REMOVE, never SET** — "assign these to Priya" means ALSO Priya, and a replace wipes every individual assignment the fifty tasks already carried; the destructive spelling is absent rather than discouraged. **Shape validated once, outcomes per task**, because those are different failures: an unsettable field is the same mistake for all fifty (422 before any write), while a status name present in one project and absent from another is a fact about THAT task — failing the batch for it makes a mixed selection unusable, which is precisely the selection somebody makes after an import. **An invisible task is SKIPPED, not an error** (R5: per-id reporting says exactly what a per-id 404 says, and aborting would let a caller probe for existence). **One transaction** — a re-triage that half happened is harder to recover from than one that did not, because nobody can tell which half. **Re-asserting a value is not a change** (`moved_people` is pure so the claim is asserted directly): fifty tasks already Priya's would each gain a timeline entry saying nothing, and a count nobody can trust is worse than no count. **ONE notification per person per batch, not one per task** — being handed fifty tasks should ring once and say fifty; fifty bells is a bell people turn off, which is WS-27j's own argument applied to the case that would have broken it. In the browser the selection is **pruned whenever the filter changes** (select forty, narrow to three, press Done must not act on thirty-seven nobody can see), a shift-click ranges over the board's ON-SCREEN order, a two-assignee card drawn in two columns counts once, and the outcome line names every category including the boring ones. **AND IT UNCOVERED A SHIPPED WS-27j BUG** (spec §11.12): `notifications.deliverable` probed only `project_clause('t.root_project_id')` while `core.task_visibility_clause` — whose docstring warns a second implementation "would drift the moment one is edited alone" — carries TWO ways in. So (1) anybody assigned work in a project they hold no grant on was judged undeliverable: they could OPEN the task, the assignment notified nobody, and the response told the assigner they could not see it — the silent assignment WS-27j exists to end, still open for the most common case in a grant-scoped app; and (2) scoping to `root_project_id` ALSO missed a grant made on a SUBPROJECT — the old test asserted that scoping on the argument that "probing `project_id` would miss a grant made on an ancestor", which is BACKWARDS, as a real-Postgres run showed: the closure is recursive and expands DOWNWARD. Fixed to use the shared clause; the test that encoded the wrong reasoning now records why it was wrong. **A fake collision the fix exposed:** `FakeDB` matched the rule-3 probe by substring, and the new clause embeds both `pm_task_assignees` and the closure's `UNION` — exactly the AUDIENCE branch's fingerprint — so `deliverable` got a list of assignees where it expected a visibility answer and four tests failed for a reason unrelated to the code under test. A fake that dispatches on substrings needs fingerprints that are SPECIFIC, not merely present | | **WS-28** | **People Center — directory, org chart, and the assignment seam** *(minted 2026-08-06)* | `specs/people_center_app.md` | ✅ **a + b BUILT 2026-08-06 · b-write BUILT 2026-08-07** · 🟢 c–e dispatchable · 🔴 f owner-gate | Scope owner-set 2026-08-06: **directory, skills, org chart, capacity, seats/roles — exactly what assignment and planning need**; leave/onboarding/hiring are named as later phases so their absence is a decision. **The fact this spec exists to settle:** there are TWO people stores and that is deliberate — `app_user` answers *can they sign in and what may they see*, `gtd_people` answers *who are they and what can they do*, and the directory must include people with **no login** (contractors), which is why the Projects app's assignee is a plain string. They join on lowercased email, and **P-1 fixes that join before it is relied on**: migration 49 made `name` UNIQUE and left `email` unconstrained, so today two rows may share an address and an email→person join is ambiguous. Surfaces: directory (honouring WS-24 N4's HR projection, with a *restricted* empty state distinct from *none*) · person page · org chart from `manager_id` with a Center overlay that **shows** department/group mismatches rather than smoothing them · capability search over stated skills → résumé evidence → the existing `capability_embedding`, which **suggests and never assigns** · seats & roles matrix (read + propose; applying a membership change stays owner-gated per §6 (d)). Closes WS-13's outstanding *People directory read view* item. Tickets **a** key-shape fix (🟢) · **b** directory + person page (🟢) · **c** org chart (🟢) · **d** capability search (🟢, ranking EVAL-LOCKED) · **e** the Projects seams — directory-backed assignee picker listing agents and directory-only people, capacity derived from open assigned tasks (🟢) · **f** seats & roles writes (🔴 OWNER-GATE).. **a BUILT 2026-08-06** (mig `148_people_key_shape.sql` + `scripts/import_hr_people.py`; 22 cases, 11 mutants red, 1 equivalent): `UNIQUE(name)` dropped, partial unique on `lower(email)`, status CHECK. **P-1 did not name its own consequence** — the HR importer upserts `ON CONFLICT (name)` and would have failed outright, so a `source_key` (`:`) carries the upsert instead, backfilled BEFORE the constraint is dropped while `name` is still distinct. **Neither new constraint may block a deploy** (main was bitten twice this month): a duplicate address is quarantined into a new `email_conflict` column with a deterministic winner rather than failing `CREATE UNIQUE INDEX`, and the status CHECK is added `NOT VALID` then validated in a guarded block, so an unanticipated legacy value leaves a NOTICE instead of stopping the deploy. ⚠️ `schema.generated.sql` NOT refreshed — needs a live DB; regenerate on the first deploy that applies 148. **b BUILT 2026-08-06** (mig `149_people.sql`, `routes/people/`, `src/app/people/`; 32 hermetic + 28 vitest cases, 11 mutants red): `/people` directory, person page with all four panels, and the People Center's "Directory & org chart" sub-app flipped live — closing WS-13's outstanding read view. **Its own feature slug**, not `feature:tasks`: a manager who needs the org chart should not be handed the personal GTD task manager to get it. The gate is new but the HR **projection is imported** from `tasks.core` and a test asserts the function's *identity*, since two answers to "may this caller see skills" are two answers waiting to drift. Three filters (the `q` skills clause, `skill`, `has_capacity`) are dropped without `admin:members:read` so search cannot become an oracle for the hidden field — and the response carries `hr_visible` so the UI says "restricted" rather than leaving a blank strip to read as "nobody filled it in". Load is **computed from open assigned tasks** and carries `unestimated`, because a bar built from the estimate sum alone shows somebody holding thirty un-estimated tasks as completely free. The work panel is scoped by the **viewer's** grants and answers `available:false` without `feature:projects`. **Registration is FIVE places, not four** — the fifth is `test_org_access_enforcement.GATED_ROUTERS`, hand-maintained, where an absent router is unchecked rather than passing; also added the named `test_projects_is_registered_on_both_sides` that WS-27a never wrote. **b-write BUILT 2026-08-07** (`people/components/PersonEditor.tsx`, `people/lib/form.ts`, `people/lib/write.ts`; 26 hermetic + 23 vitest cases, 7 mutants red): closes the regression the 2026-08-06 scope narrowing opened — deleting the tasks app's People view took `PersonEditor` with it, and with it the only UI for creating a person, editing skills and uploading a résumé. The GET-only `/api/people` proxy is **unchanged**: the writes go to `/api/tasks/people`, where they have always lived. Controls are absent rather than disabled, driven by a new **`can_manage`** flag on the reads — hide-rather-than-disable is impossible unless the read tells the UI, and the alternative is drawing the button and letting the click find a 403. **Restoring it turned up three ways migration 148 had already broken the write routes**, each a 500 in front of an admin rather than a test failure: the status vocabulary moved (49's `inactive`/`on_leave` vs 148's CHECK) and is now ONE tuple shared by filter, facets, select and validation; `create_person` still refused a duplicate NAME, preserving precisely the behaviour 148 dropped `UNIQUE(name)` to remove; and nothing checked the address 148 made unique, so a duplicate was an `IntegrityError` instead of a 409 naming the other row. **The lesson recorded in spec §10:** a migration that changes a table's shape has to be walked against every route that WRITES it — the read routes were built after 148 and were correct by construction, the write routes predated it and were never revisited | @@ -632,13 +632,24 @@ push into the live Zoho tenant on the next sync cycle (which `POST /crm/sync/zoho` runs with or without `CRM_ZOHO_SYNC`)**. Ruled D-CRM-9 (owner, 2026-08-06): this is intended behaviour — agent- and auto-originated writes enter the push queue exactly like human ones. So the flip is both a live -change to email-app behaviour and, transitively, a write path into Zoho. ⚠️ The -settings field **does not exist yet** and was deliberately not added by WS-26d's -read half. **The hook it was missing is now named (2026-08-06, `crm_app.md` §9.2):** +change to email-app behaviour and, transitively, a write path into Zoho. +**The settings field now EXISTS — `crm_auto_lead: bool = False` in +`packages/acb_common/acb_common/settings.py`, beside `crm_zoho_sync`** (WS-26d-autolead, +BUILT 2026-08-08, branch `ws-26d-autolead`; the ⚠️ "does not exist yet" note is +retired). **Built, NOT flipped, NOT deployed**, and while it is off the branch +changes no runtime behaviour at all. The hook is `routes/email/scheduler_hooks.py::process_new_mail` — the one seam the scheduler, -the manual-sync route and the webhook all funnel through. The field lands with -WS-26d-autolead, shipping OFF, with a regression proving the OFF state makes no -CRM call at all · +the manual-sync route and the webhook all funnel through — and the flag is read +at that CALL SITE, before the CRM step is entered, so the OFF state issues no CRM +query; both the runtime regression and an AST assertion that the gate is +lexically outside the step live in `tests/unit/test_crm_auto_lead.py`. ⚠️ Two +things an owner should know before flipping: the first ON-state run per mailbox +only ACTIVATES the cursor (`crm_auto_lead_cursors`, migration 157) and mints +nothing — mail that arrived before that instant is history by construction, which +is what stops a deep resync minting a year of leads — and the accepted residual is +that two concurrent syncs of one account can double-mint one visible, +hand-deletable duplicate (a UNIQUE index on `crm_leads.email` is refused: 1,516 +imported rows may already carry duplicates, the migration-148 shape) · **(c) the WS-26e cutover + retirement** — the final import + parity check, repointing the graph-mirror consumers (`sales_views.py`, `reconciler.py`), retiring `ingestion/sources/zoho/` + cron + webhook + config (spec §7.4, which diff --git a/apps/services/gateway/AGENTS.md b/apps/services/gateway/AGENTS.md index 4d2cd0577..b34e9cd6a 100644 --- a/apps/services/gateway/AGENTS.md +++ b/apps/services/gateway/AGENTS.md @@ -24,7 +24,7 @@ webhook receivers, OAuth callbacks, and the Control Plane API. 5b. routes/agent_skills.py — WS-23 S2 per-agent skill toggles: GET/PUT /agent/{name}/skills over the `agent_skill_setting` table (org-access override shape — reason/set_by provenance; replace-wholesale PUT gated `admin:access:manage`). Rules enforced at the API: family must exist in SKILL_FAMILIES; `core` and `apps` are un-toggleable → 422 (core = floor, apps = managed via app_grants). Enforcement of stored rows lives in orchestrator/_tool_injection.py::_resolve_injected_scope (intersection only; no rows ⇒ byte-identical to pre-S2) 6. routes/memory.py -- Memory search and management endpoints. The path parameter is a SCOPE KEY (`acb_memory.scope_key`: an email, `agent:`, or `org:global`), not a user id, and `_authorize_scope` answers one rule per shape — your own email only (not admins': administering members is not reading their private context), agent memory on the same `agents:run:` the run path enforces, org memory on `memory:read_org`/`memory:write_org`, `room:` on room membership (its audience IS the room), `prefs:` on being that person, unknown shapes refused so a growing vocabulary fails closed. `require_internal_auth` alone used to be the whole gate, which was the wrong gate: it proves the caller IS the platform, and the Next BFF is the platform on behalf of every signed-in user. A service principal reaches shared scopes but must assert `X-User-Email` to touch a person's — omitting an identity must fail closed, since omission is how the original hole happened. `delete` also verifies the memory is IN the scope, or naming your own scope with someone else's memory id deletes theirs. 7. routes/settings.py -- LLM settings, model config -8. routes/email.py -- Email account CRUD, message listing/search, send, sync, AI chat, OAuth flow for Gmail/Microsoft/IMAP. Background sync scheduler hooks (refresh/remove) on account PATCH/DELETE. **`transport/oauth.py` — the connect flow**: the authorize leg is GATED and stays gated (the browser reaches it through the Next BFF; see "Authentication posture" below), and its `user_email` query parameter is a **fallback only** — `user.email` wins, or a crafted parameter still decides whose account the mailbox attaches to. ⚠️ **Known defect, filed not fixed** (`ai-company-brain/specs/email_app_master_plan.md` §7 Tier 1 item 1): `_oauth_states` is a module-level in-process dict. Every deploy restarts the gateway, so any OAuth flow in flight when a deploy lands loses its state and the callback fails `state not in _oauth_states` → the user is bounced to `/email/oauth/callback?error=invalid_state`. It is also not shared across workers, and entries are never expired so abandoned flows leak forever. Fix is Redis + TTL alongside signing the state; do not paper over it by widening the state check. transport/contacts.py serves GET /email/contacts/card — the people card behind a sender's name/avatar (identity, correspondence stats, last N messages, plus phone/title/company parsed out of the sender's own signature). Read side is derived entirely from mail already in the caller's own accounts (every query goes through core._account_scope); no directory lookup, no provider call. Write side: each open upserts what the parse learned into `email_contacts` (mig 119) via `_remember_contact`, so the mailbox accumulates a people directory for the planned Contacts view — see ai-company-brain/specs/email_app_master_plan.md §3.14 before extending it. Invariants that must not be weakened: `manual_fields[]` columns are never overwritten by a parse; an empty parse never blanks a stored value; derived facts (counts, last-seen) are never stored; the domain→company guess is display-only and applied AFTER the write. +8. routes/email.py -- Email account CRUD, message listing/search, send, sync, AI chat, OAuth flow for Gmail/Microsoft/IMAP. Background sync scheduler hooks (refresh/remove) on account PATCH/DELETE. ⚠️ **`scheduler_hooks.py::process_new_mail` is the shared new-mail pipeline and the one seam the scheduler, the manual-sync route and the Graph webhook all funnel through** — rules → sweep → categorize senders → classify threads → auto-archive → **CRM auto-lead**. The last step is NOT this package's: it lives in `routes/crm/auto_lead.py` (WS-26d-autolead), runs after auto-archive so it only ever sees what is still in the inbox once the account's own automation has finished, and is called from inside `if auto_lead_enabled():` so the OFF state — which is the shipped state — enters no CRM code at all. Every step stays try/except-isolated on `sync.*_failed`: a failure in any of them, the CRM one included, must never break mail sync. ⚠️ This hook is **also reached by deep resyncs and first syncs**, which classify up to a year of mail without stamping `rules_held_back_at`; anything added here that acts per-message needs its own history discriminator (the CRM step's is a two-timestamp cursor). **`transport/oauth.py` — the connect flow**: the authorize leg is GATED and stays gated (the browser reaches it through the Next BFF; see "Authentication posture" below), and its `user_email` query parameter is a **fallback only** — `user.email` wins, or a crafted parameter still decides whose account the mailbox attaches to. ⚠️ **Known defect, filed not fixed** (`ai-company-brain/specs/email_app_master_plan.md` §7 Tier 1 item 1): `_oauth_states` is a module-level in-process dict. Every deploy restarts the gateway, so any OAuth flow in flight when a deploy lands loses its state and the callback fails `state not in _oauth_states` → the user is bounced to `/email/oauth/callback?error=invalid_state`. It is also not shared across workers, and entries are never expired so abandoned flows leak forever. Fix is Redis + TTL alongside signing the state; do not paper over it by widening the state check. transport/contacts.py serves GET /email/contacts/card — the people card behind a sender's name/avatar (identity, correspondence stats, last N messages, plus phone/title/company parsed out of the sender's own signature). Read side is derived entirely from mail already in the caller's own accounts (every query goes through core._account_scope); no directory lookup, no provider call. Write side: each open upserts what the parse learned into `email_contacts` (mig 119) via `_remember_contact`, so the mailbox accumulates a people directory for the planned Contacts view — see ai-company-brain/specs/email_app_master_plan.md §3.14 before extending it. Invariants that must not be weakened: `manual_fields[]` columns are never overwritten by a parse; an empty parse never blanks a stored value; derived facts (counts, last-seen) are never stored; the domain→company guess is display-only and applied AFTER the write. 9. routes/v1_compat.py -- OpenAI-compatible /v1/chat/completions endpoint (used by Copilot SDK BYOK provider and MAF OpenAIChatCompletionClient). Includes message sanitization for providers with strict validation (e.g. DeepSeek rejects assistant messages with neither content nor tool_calls). 10. routes/debug.py -- E2 post-hoc diagnostics over the agent_run trace store (GET /debug/runs, /debug/runs/{id}, POST .../flag). EXECUTIVE/AGENT-gated. 11. routes/observability.py -- E2 LIVE observability over the global activity bus (cc:activity): GET /observability/activity/recent (backfill), /observability/activity/stream (SSE, agent+model activations across chat and ALL apps), /observability/active (runs in flight), /observability/roster (all agents + working/idle status for the office view), /observability/cost (daily LLM $ rollup by model/app). EXECUTIVE/AGENT-gated. Publish side: acb_common.activity + the executor run boundary + acb_llm._emit_usage (which also prices each call via litellm). App attribution is automatic — acb_llm.context._infer_app_source() reads the caller's gateway.routes. module, so any new app is observable with zero wiring. @@ -54,6 +54,7 @@ webhook receivers, OAuth callbacks, and the Control Plane API. - ⚠️ **`reports.py` (WS-26g) — read-only, and the funnel is defined against what the log RECORDS rather than what its name suggests.** `GET /crm/reports/{pipeline,funnel,win-loss,owners}`; no write, no Zoho call, no flag, no migration. `WEIGHTED_SQL` moved into `core.py` beside `WEIGHTED_TYPES` when this became its second consumer (`pipeline.py` re-exports it, so no caller moved) — a second copy would defeat `_crm_fakes._WEIGHTED_SUM_RE`, which reads the expression OUT of the statement text precisely so a drifted formula changes the tests' answer; `core.status_wire` absorbed `admin`'s and `pipeline`'s duplicate status projections for the same reason. ⚠️ The owner leaderboard's bucket key (`.strip().lower()` in Python) and its aggregate predicate (`lower(trim(owner_email))` in SQL) are one normalisation written twice and must stay byte-consistent: while the SQL lacked `trim()`, a padded address split a bucket the tally had already merged, so the leaderboard under-reported an owner, dropped a deal into no bucket at all, and still said `omitted: 0`. Four properties are load-bearing. **(1)** `crm_status_changes` logs TRANSITIONS only — `create_record` writes no row and the importer writes none — so all 551 imported deals have zero rows and a deal's first stage is never a `to_status`; "entered" is therefore a VISITED-SET union (`from_status`, `to_status`, and the deal's CURRENT stage), and dropping that last term reports an empty funnel for the whole live board. **(2)** Dwell is grouped by **`from_status`**, the stage being LEFT — `to_status` would label every measurement one lane too far on, plausibly. **(3)** The log stores NAMES, not ids, so a lane rename orphans its history; orphans are tallied into `unmatched` and never dropped. `entity_type = 'deal'` filters every such read — and note it is defence-in-depth, not the sole guard, since the funnel also keys through deal ids: the test that makes it load-bearing seeds a row stamped `lead` against a DEAL's id, which is realistic because `entity_id` has **no foreign key** (the log outlives the row on purpose). **(4)** The trailing window is bounded at BOTH ends. NULL `closed_at` — every imported closed deal until WS-26f f4's owner-gated backfill runs — falls outside it (zeros, never "closed today"), with the count reported so a 0% win rate is explicable; and so does a FUTURE `closed_at`, because f4's proxy is Zoho's `Closing_Date`, a forecast date that imported deals routinely carry ahead of today — with a lower bound only, running the repair would have started counting next quarter's deals as closed this quarter and inflating the cycle average by their forward span. The lost-reason breakdown carries a NAMED unattributed bucket (the importer bypasses both gates; `lost_reason_id` is `ON DELETE SET NULL`). **No `GROUP BY` is emitted, deliberately**: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would have to reach it through a join and would stop being the expression the fixture and the fake both read. - **A hand-edited `lead_name` survives a PATCH that moves its inputs** (`core.lead_name_is_derived`): the name is re-derived only while the stored value still equals what the fallback chain would produce. Answered by recomputing rather than by a `lead_name_is_custom` column — a flag has to be maintained by every writer (importer, sync engine, agent tools) and the one that forgets it silently reverts a typed name. - ⚠️ **The timeline's THIRD source is email, and it is the ONE place in this package scoped to the CALLER rather than to the org** (WS-26d-email, spec §9). Everything else here follows D-CRM-3 — org-visible to every `feature:crm` holder, no owner predicate. Email cannot: the CRM is org-visible while a mailbox belongs to one person, so an unscoped join publishes one member's inbox to the whole company. `activities._timeline(entity, record_id, limit, user)` therefore **requires the caller** and all four routes pass it; a route that drops `user` again would compile, return a timeline, and have no identity left to scope by. The predicate is `activities._email_account_scope`, a **verbatim copy** of `routes/email/core.py::_account_scope` (D-CRM-4, the same call `broker_handlers.broker_gate` made — importing another route package's private helper is the coupling this package declined once already). ⚠️ **Two copies is a coincidence; a THIRD copy anywhere means promote it to a shared module instead.** The fragment hardcodes the alias `em`, so the query aliases `email_messages` as `em` and it drops in unchanged (`email/automation/analytics.py` had to `.replace()` it). Other invariants: the unit is the **thread** (`DISTINCT ON (account_id, COALESCE(thread_id, id::text))` — a row-per-message timeline double-counts every conversation, and grouping on a raw nullable `thread_id` folds every un-threaded message in an account into one entry); addresses resolve **once per record**, not per source, because a deal's set already contains its originating lead's and a per-source pass would return every inherited thread twice; `crm_deals` has no `email` column so a deal joins through `lead_id → crm_leads.email` **and** `crm_deal_contacts → crm_contacts.email`, unioned, with the lead's threads labelled `origin="lead"`; **no addresses means no query at all** (an empty `IN ()` is a syntax error, and the failure a fallback would produce is the whole mailbox on a record that names nobody); inbound `from_address` only in v1, and organizations deliberately do **not** join by domain (an `@fracktal.in` match would attach the entire company mailbox to our own org record). Index: `(account_id, LOWER(from_address->>'email'))` on `email_messages` — the two FTS GINs bury the address inside a `to_tsvector` and are usable only via `@@`. ⚠️ **`tests/unit/test_crm_email_timeline.py` carries a MUTATION FENCE**: deleting the `_email_account_scope(…)` call must turn `test_a_holder_with_no_mailbox_sees_no_email` and `test_two_holders_each_see_only_their_own_account` RED. `_crm_fakes.py` grew four readers so it can see that (a scope subquery, a lowercased JSONB address comparison, a composite LEFT JOIN, and `DISTINCT ON` grouping) — before them the fake did not merely ignore the scope, `_PLAIN_EQ` MISREAD the subquery's own `user_id = :uid`. Do not simplify the SQL to suit the fake; extend the fake. + - ⚠️ **`auto_lead.py` (WS-26d-autolead) — the package's second UNATTENDED writer, and the only one reached from another app's hook.** It registers **no routes** and is therefore NOT imported from `__init__.py` (same reason as `broker_handlers.py`); its one entry point `create_leads_from_new_mail(account_id)` is called from `routes/email/scheduler_hooks.py::process_new_mail`. **It lives here rather than in the email package because what it does is write a CRM record** — it owns `crm_auto_lead_cursors`, it goes through `records.create_record`, and its flag is a CRM owner gate. Unlike the timeline join above it **imports** the automation package's PUBLIC identity primitives (`sender_scope` / `resolve_org_domains` / `normalize_domain`) instead of copying them: D-CRM-4 declined to import another package's *private* helper, and a third copy of "is this person a colleague?" is exactly the drift that rule prevents. Six properties are load-bearing. **(1) The flag is read at the CALL SITE, before the step is entered** — `if auto_lead_enabled(): await create_leads_from_new_mail(...)` — so with `CRM_AUTO_LEAD` off no CRM code runs and no CRM query is issued on the mail path; `auto_lead_enabled` is the flag's ONE definition and a gate moved *inside* the step is pinned red by an AST assertion, not only by a runtime sentinel. **(2) TWO cursor predicates, together.** `process_new_mail` is also reached by ~1-year deep resyncs and by a newly connected mailbox's first sync, and neither stamps `rules_held_back_at`, so "everything classified" would mint a lead per unknown sender across a year of mail — each born `zoho_dirty` and queued for the LIVE tenant within one 600s cycle (D-CRM-9), with no confirmation card on a scheduler hook and no delete tool. `received_at > activated_at` (stamped once, never advanced) is the backfill discriminator; `rules_processed_at > processed_watermark` is the incremental cursor. **(3) Dedup is a SELECT guard plus in-batch de-duplication, never `ON CONFLICT`** — `crm_leads` has no unique constraint on email (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so the ticket's original upsert arm could not have fired. The cross-invocation race is ACCEPTED and recorded; **do not "fix" it with a unique index** (1,516 imported rows, the migration-148 shape). **(4) "External" is necessary, not sufficient** — `sender_scope` fails SAFE to `"external"`, which is the wrong direction when the consequence is a lead row for your own CFO in a live Zoho tenant, so the normalised internal-domain list is a second, independent gate. **(5) The lead goes through `records.create_record`, never raw SQL** (`_resolve_status`, the `owner_email` default, `validate_source` and `mark_dirty_on_insert` all live only there, and only the last is visible in the row afterwards), and `lead_name` is left to `compute_lead_name` over a display name STRIPPED before it is split. **(6) The first activity is `type='system'` — outside `sync_zoho.push_activities`' `type IN ('note','task')` predicate — carrying the subject and the sender in `meta` and an EMPTY body.** The step never selects `body_text` or `snippet`: the projection is the privacy boundary (D-CRM-12 applied to what a machine writes). `tests/unit/test_crm_auto_lead.py` (52 cases) carries a seven-mutant fence over exactly those properties, and `_crm_fakes.py` gained a `@>` containment reader for it — without one the "have we ever emailed them" probe was invisible and the fake answered "yes" for every Sent message. 14. routes/admin/ -- Org access control `/admin` API + `/auth/me` (spec: ai-company-brain/specs/org_access_control.md, Phase 1): member roster and lifecycle (invite/suspend/remove — soft, because ~every user-scoped table keys people by email — **plus a separate hard delete**, below), role assignment, custom role CRUD, per-user allow/deny overrides, and the feature catalog the admin UI renders from. `GET /auth/me` is deliberately NOT admin-gated — every signed-in member calls it to resolve their own feature/agent access, and it returns resolved OUTCOMES (allowed feature slugs, runnable agent names) rather than raw permission patterns, so the matching rule has exactly one implementation. `GET /admin/members/{email}/access` returns each decision WITH its provenance (which role granted it, which override took it away) — the admin UI shows that verbatim rather than re-deriving it. Invariants enforced in `_common.py`: the org always keeps an owner, nobody assigns a role above their own rank, system roles are immutable, and **nobody locks themselves out**. That fourth one (`assert_not_self_lockout`, `colleague_onboarding.md` §2 Step 5 / N7+N8) is called by `update_member` (PATCH), `remove_member` (DELETE) **and `purge_member` (DELETE …/purge)** — three doors reach the same `is_active = False`, and while the check lived inside DELETE alone the PATCH had none: `PATCH {"status": "suspended"}` on your own row was refused only by `assert_owner_survives` firing coincidentally in a one-owner org, so a second owner opened it. ⚠️ The rule is **"any status that is not `active`"**, never a list of destructive ones — `EffectiveAccess.is_active` is `status == "active"` exactly, so `invited` is a lockout too, and an enumeration would have to remember it. Comparison is case-insensitive and empty-safe on both sides (an IdP that re-cases a UPN must not switch the guard off; a caller with no identity is not everybody). ⚠️ **It and `assert_owner_survives` both answer 409** — a test that asserts the bare status code cannot tell which fired, and for self-suspension the one that fires today on `main` is the wrong one; discriminate on the detail text and on what was written (`tests/unit/test_admin_member_offboarding.py`). **`purge_member` — `DELETE /admin/members/{email}/purge` (N8)** is the hard delete: a SEPARATE route on the same `admin:members:manage`, never a flag on Remove (which would put the irreversible path one typo from the reversible one). Its decision is **purge the person, keep their work** — the `app_user` row, every access grant (`user_role`, `user_permission_override`, `org_group_member`, `chat_session_participant`, `app_grants`, `app_tool_grants`), every credential (`email_accounts`, `wa_accounts`, `task_accounts`), their PRIVATE `chat_session` rows and their `access_request` row go; what they authored and **the audit trail stay** (an audit trail that disappears with the person is not one — `app_audit` already carries a FK-less `app_id` commented "audit survives hard delete"). ⚠️ **Nothing is anonymised, on purpose**: the address is the join key across ~50 tables, so scrubbing `owner_email` would orphan the apps rather than hide the person. ⚠️ **The three credential rows cascade, and the map is `members._CREDENTIAL_CASCADES`** — `email_accounts` takes the whole mirrored mailbox (**17 direct children, 20 with transitives**), `wa_accounts` the whole WhatsApp mirror (**14 / 16**; `wa_media` hangs off `wa_messages`, NOT off the account), `task_accounts` the SYNCED half of `gtd_items` **and `gtd_projects`**; the credential is `NOT NULL` on the row, so it cannot go without it. That map is hand-maintained and says so, and is pinned against `infra/postgres/` by a test that re-derives it — the first version named 15 of the 20 email tables, which on a route whose safety argument is "the admin is told the blast radius before clicking" is the wrong direction of error. ⚠️ **THREE tables are split across both lists, and each predicate is load-bearing:** `chat_session` by `visibility` (private deleted, shared kept — a room cascades `chat_message` and one person's off-boarding must not take a shared transcript), and `gtd_items` + `gtd_projects` by `account_id` (`IS NOT NULL` = the SYNCED mirror, counted and deleted explicitly; `IS NULL` = the LOCAL rows they authored here, kept). ⚠️ **A KEEP clause must exclude everything the delete side CASCADES away, not merely everything it names.** The `tasks` keep clause originally had no `account_id` predicate, so a member with 847 synced tasks was answered `kept: {"tasks": 847}` while all 847 went with `task_accounts` — the response reported a destruction as a survival. `_PURGE_DELETES`/`_PURGE_KEEPS` derive `count_sql` and `delete_sql` from ONE `where` clause so the count and the delete cannot differ, but that is a within-row-spec guarantee and says nothing about a third statement three entries up; one transaction, one commit, and `record_admin_change` fires BEFORE it (`acb_audit` has its own session, so the record of a destruction survives a rollback of it — though `acb_audit/log.py:49` swallows every exception, so a *completed* purge is NOT guaranteed to leave an audit row). Pinned by `tests/unit/test_admin_member_purge.py` — including the structural assertion that no audit table appears on the delete side at all, the exact permission slug on the route (deleting it leaves the `admin:members:read` floor, which `manager` holds), and the cross-table cascade fences built on `tests/unit/_schema_cascade.py`, which derives the FK graph from the numbered migrations. ⚠️ **`_admin_fakes._FakeDB` models no foreign keys and therefore no cascades**, so every cross-table claim here has to be structural; no behavioural case over a seeded fake can make one. Every write calls `invalidate_access` so a change lands immediately instead of after the resolver's 60s TTL. Tables: infra/postgres/130_org_access_control.sql. Same `_common.py`-is-the-leaf layout as routes/apps and routes/tasks — and here the leaf rule is strict: feature modules import from `_common`, **never from each other**. ⚠️ **The `/admin` auth floor is PER-ROUTE, not a package property.** `_common.py` creates the router with **no** `dependencies=`; every route declares `Depends(require_admin_user)` in its own signature. A route added without it inherits no floor at all and is reachable by any authenticated member — the easiest hole to ship in this package. `access_requests.py` — **sign-in requests** (`colleague_onboarding.md` §6 / N6a, migration 143): `/admin` was push-only, so somebody arriving at the front door produced a journald warning nobody read back (53 of them for one address over 18 hours on 2026-08-03/04, and the owner learned out of band). `acb_auth.access.resolve_access` now upserts an `access_request` row when — and ONLY when — `record_request=True`, which exactly one caller passes; `GET /admin/members/requests` + `POST .../{email}/approve|deny` let the owner answer it, both writes on the EXISTING `admin:members:invite` (no new slug — a new slug is nobody's grant until an admin creates it). **Approve provisions AND activates in one action** (`status='active'`, not `'invited'`) because an approval IS the decision to let somebody in and they are already at the door; leaving them `invited` would re-create the two-click trap §2 Step 1b documents. Both provisioning callers go through `_common.provision_member` — ONE path, so invariants 1 and 2 apply to approvals too (it calls `assert_owner_survives` itself, because `set_roles` REPLACES assignments and provisioning the last owner with the default `member` role would otherwise delete the org's only owner grant). ⚠️ **Both writes hold `admin:members:invite`, which is WEAKER than the `admin:members:manage` that suspends or off-boards, so every path by which the weaker one could reverse the stronger is a cross-gate escalation.** Two independent locks, and each is load-bearing for a different sequence: (1) `_load_request(db, email, *, allowed_statuses=…)` — keyword-only, no default — refuses an already-DECIDED row, because decided rows are kept on purpose (dw9) and the tab renders only `pending`, so a decided row is invisible *and* still addressable; approve takes `("pending",)`, deny takes `("pending", "denied")` since re-denying grants nothing, and denying an *approved* request is refused because it could only make the queue contradict the roster. (2) `_common._PROVISION_MEMBER_SQL`'s `ON CONFLICT` arms **name the statuses they rewrite and never negate**: `invited` → the caller's status (the one door to `active`), `removed` → the caller's status **only when it is not `active`** (so invite still returns an off-boarded person as `invited`, byte-for-byte as before, while approve cannot reinstate them — `removed → active` stays `PATCH /admin/members/{email}`), and `active`/`suspended` are never touched. ⚠️ A `<>`/`NOT IN` test against `app_user.status` is the mutation to watch for: it reads as tidier and silently rewrites rows set under a stronger permission. `tests/unit/test_signin_requests.py` pins the SQL **structurally** (`test_provisioning_only_ever_rewrites_a_status_it_names`) — its fake DB re-implements the `ON CONFLICT` arms in Python and a mirror can only agree with itself, so the behavioural cases there cannot see the statement being widened and must not be trusted to. ⚠️ **Lock (2) declines SILENTLY — it just does not rewrite the row — so it is only half an answer, and the other half is `APPROVE_MATRIX`.** Approve used to run its `_decide(…, "approved")` after that quiet decline: HTTP 200, request marked `approved`, `set_roles` re-granting `['member']` to an off-boarded member, and the person gone for good from a tab that renders only `pending` (the resolver's upsert never rewrites `status`). **`access_requests.APPROVE_MATRIX`, read by `_disposition_for` BEFORE anything is written, is the contract:** absent → provision; `invited` → activate + assign the roles; `active` → do nothing, leave their roles alone, resolve the request as `approved` and say so in `ApproveResult.detail`; `suspended`/`removed` → **409, request stays `pending`** so the person stays visible; anything else → refuse (fail closed). The invariant: **approve never rewrites the roles of a member who already exists in a state other than `invited`** — `provision_member` ends in `set_roles`, which REPLACES assignments, and roles are otherwise `admin:members:manage` territory. The matrix is pinned against `members.VALID_STATUSES`, so a fifth member status cannot ship without somebody deciding what approving one means. `_DECIDE_SQL` also binds the read's own status filter into the UPDATE (`AND status = ANY(:allowed) … RETURNING id`) and 409s on zero rows **before `db.commit()`**, so a lost race discards its own provisioning instead of half-applying it; each route must pass `_decide` the same tuple it passed `_load_request` (a test asserts that from the source). 15. routes/workflows/ -- Workflows app `/workflows` API (spec: ai-company-brain/specs/workflows_app.md; RFC: docs/workflow-editor/README.md): workflow CRUD over the React-Flow-native edit-model (`workflows.graph` jsonb, persisted verbatim), publish → compile to an immutable `workflow_versions.serialized` run-model (edit-model ≠ run-model; runs pin versions), run start/history/detail + a per-run SSE event stream (in-process hub in service.py; runs are supervised asyncio tasks — durable queueing is BO‑20), the served node catalog (agents from the live registry, integrations from acb_skills with availability probe, workflow tool registry, ready modules — the palette is never hard-coded, spec D7), Module Studio (workflow_modules CRUD + conversational generate on acb_llm tier routing + AST validate + subprocess test/run), the inbound webhook trigger `POST /workflows/hooks/{hook_token}` (public by token — in PUBLIC_ROUTES + the router's exempt list; optional HMAC `X-CC-Signature`; rate-limited; fires only published workflows with an enabled webhook trigger), and the cron schedule scanner (scheduler.py — apscheduler CronTrigger parsing inside a supervised asyncio loop with CAS claims on `last_fired_at`; started/stopped from main.py lifespan). The engine subpackage (engine/: templating, graph compile/validate, node handlers over injected NodeServices, MAF WorkflowBuilder runner, module AST validator + restricted subprocess runner) is transport-free — no FastAPI/DB imports — so it is unit-testable alone and movable into the orchestrator if isolation later demands. Agent nodes call `orchestrator.executor.run_agent` (source="workflow", MAF batch path — constraint #9); write-class tool nodes dispatch through `action_broker.propose/submit` (fail closed, constraint #4); module code is import-free/pure-transform only (real sandbox is BO‑7). Capability search (search.py): **keyword-only by explicit owner decision** — deterministic token/substring ranking over the live registries (no index table, no embeddings; an embedding-backed variant was built and deliberately removed in favour of BO‑22, the platform-wide semantic-search service, whose ranking backend will swap in behind the same API shape) — `GET /workflows/catalog/search` serves the palette's search box AND the copilot's shortlist from the same ranking. Workflow Copilot (copilot.py): `POST /workflows/{id}/copilot` — chat-to-build; the LLM emits `{reply, graph, new_modules}`; **missing modules are auto-created** (Module Studio AST validation, saved `ready` with `auto_created` provenance, name→id rewired), the graph is validated with one named-issue repair round against the same validators as publish, and the result is returned for CLIENT-side apply — the copilot never writes the workflow row. `_call_copilot` is the stubbing seam for tests. Tables: infra/postgres/132_workflows.sql. Slice 2: **approval node** — an `approval` node pauses the run (engine returns status `paused`; downstream marked `pending`), `service._hold_for_approval` files a `workflow.resume_run` proposal into the EXISTING Action Broker inbox (`pending_actions` → /approvals UI) with everything a resume needs in the `workflow_run_pauses.snapshot`; approving fires `broker_handlers._resume_run_handler` which replays the run with completed nodes' stored outputs (`precomputed` — no repeated side effects) and the gate resolved; a rejected proposal is reconciled lazily on run read (run → `cancelled`). **Event triggers** — `triggers.dispatch_event` starts runs for published workflows whose `kind='event'` binding matches `(source, event_type)` (empty type = all); fed by BOTH `/agent/webhook/{source}` (routes/agent.py calls it after agent routing; response carries `workflow_runs`) and the native ClickUp receiver via `ingestion.event_hooks` (a `post_sync.py`-style sink registry — ingestion never imports upward; main.py registers the dispatcher at startup). Same core-is-the-leaf layout as routes/tasks; ⚠️ `__init__.py` import order is load-bearing (static paths before crud's `/{workflow_id}`; a regression test pins it). Startup: main.py lifespan calls `service.reconcile_orphaned_runs()` BEFORE starting the scheduler — rows still `running` belong to a dead process and are swept to `failed` ("interrupted by a platform restart"); `paused` rows are deliberately untouched (resume rebuilds everything from the pause snapshot), and `runs.py` keeps the per-read lazy patch for reads that race the sweep. Run-history drill-in (spec F9): clicking a history row in the editor's RunConsole fetches the run detail and paints its recorded `node_results` onto the canvas (cleared when a live test run starts). Engine semantics are locked by a CI-blocking golden trajectory eval — `evals/trajectories/test_workflow_engine_trajectory.py`; `skill-eval.yml` triggers on `routes/workflows/**` so engine edits re-run the gate. **Publish authority** (spec Q3, migration 133): `POST /{id}/publish`, `/versions/{v}/rollback`, and `/disable` require the `workflows:publish` capability on top of the router's `feature:workflows` gate — they are the acts that ARM triggers to run unattended. Drafting, validate, Test runs, duplicate, and the copilot stay open to the feature (a draft fires no triggers and its writes are still broker-held). `/auth/me` returns a resolved `capabilities` list so the editor can grey out Publish with a reason instead of a bare 403 — the browser must never re-derive wildcard matching (`permissions` holds raw patterns; an owner has `*`). **Wait node** (F3 logic vocabulary): `{"seconds": N}`, ≤`WAIT_INLINE_MAX_SECONDS` (60) sleeps inline inside the run; longer pauses the run exactly like an approval but with `reason='wait'` + a `resume_at` deadline in the pause snapshot and NO broker proposal (nobody decides anything) — `scheduler.scan_due_waits()` runs in the same loop as cron triggers and hands matured pauses to the SAME `service.resume_run`, which routes by pause reason (`elapsed_waits` vs `resolved_approvals`, so an elapsed wait can never clear an approval downstream). A resumed wait must never sleep again: the handler only sleeps when the duration is inline-short. Lifecycle extras: `POST /{id}/duplicate` (crud.py — copies graph/variables/triggers into a fresh DRAFT; the hook token is ALWAYS regenerated, it is a credential) and `POST /{id}/versions/{v}/rollback` (publish.py — republishes version v's immutable snapshot as a NEW version; deliberately does not re-validate as a gate since rollback is incident response — catalog drift comes back as non-blocking `warnings`, and the draft edit-model is never clobbered). **Automation health** (spec R2, migration 134): every terminal run calls `service.evaluate_automation_health()`, which disables a published workflow after `AUTO_DISABLE_AFTER` (5) consecutive failures **from `UNATTENDED_TRIGGERS` only** (`schedule`/`webhook`/`event` — a maker's Test runs and agent `api` calls must never disable production). The streak is derived from `workflow_runs`, never a counter column, and is scoped to runs after `workflows.health_since`, which publish/rollback/enable each re-stamp — without that window a re-enabled workflow would re-disable on its next failure, since the failures that tripped the policy are still the newest rows. The disable is a CAS on `status='published'` so concurrent failing runs produce exactly one disable; `disabled_reason`/`disabled_at` are written the same way for the human Disable path, so the gallery answers "why is this off?" identically. Notification is in-product (persisted reason → gallery badge + editor banner, `workflows.auto_disabled` log, activity-feed `disabled` event); outward notification would be an outward write and belongs on the broker path. `POST /{id}/enable` (publish.py, same `workflows:publish` gate) is the way back: it re-arms the EXISTING live version rather than minting one, 409s if the workflow was never published, and is idempotent when already live. `_execute_run`'s `trigger_kind` is a REQUIRED keyword — a dropped kwarg would make the whole policy silently inert. **Trigger durability** (spec §3.3a): schedules are DB rows, not OS cron and not an APScheduler process — `CronTrigger` is a parser only. ⚠️ `compute_due_fire` only looks FORWARD, so a trigger with `last_fired_at IS NULL` yields no tick; `_claim_baseline` arms it on first sight instead of firing (a cron says *when*, not *how far back*). Without that step a new schedule never fires **at all** — it produced no tick, so it never got a baseline, so it produced no tick. `config.timezone` is an IANA wall clock (default UTC) validated at save with the cron, so a 9am job stays 9am across DST; the zone is passed to `CronTrigger.from_crontab`, and instants stay UTC-aware throughout. `update_workflow` rewrites trigger rows wholesale but CARRIES `last_fired_at` across for unchanged schedules (`_trigger_identity` = kind + cron@timezone) — otherwise every canvas save re-armed the cron and lost the already-fired-this-tick guarantee. Because the CAS claim commits BEFORE `start_run`, a claimed tick can never be re-offered: every path out of that block calls `service.record_skipped_run()`, which writes a terminal `cancelled` run row (cancelled, not failed — being busy must not feed the R2 auto-disable policy). **Hook URL**: `core.hook_url()` builds it from `settings.public_api_base_url` and `get_workflow` returns `hook_url`/`hook_path`; the browser must NEVER assemble one from `window.location`, because the control-plane `/api` proxy re-serializes JSON (breaking sender HMAC) and drops non-JSON bodies. The Next route `api/workflows/hooks/[token]/route.ts` is a raw-bytes passthrough that attaches no internal bearer. **Typed tool arguments** (`engine/tool_args.py` — n8n's typed-node-parameters pattern, Sim's `subBlocks`): a tool's `args_schema` value is a mini-language `type[?][|description]` over the closed set `{string,number,boolean,object,array}`; an unknown type degrades to `string` rather than raising (one bad declaration must not take the whole catalog down). It is parsed in ONE place and consumed in three — the catalog serves `args[]` (parsed) so the browser never re-implements the grammar, `validate_graph(tool_schemas=…)` blocks publish on a missing/unknown/mistyped argument (`tool_args` issue code), and `execute_tool` re-checks at run time because a draft Test, a copilot graph, or an older published version can all reach a handler that publish never saw. `{{refs}}` satisfy required checks and are exempt from type checks on both sides — they resolve at run time. Type checking is deliberately lenient (only container-vs-scalar category errors) so the messages that fire are worth reading. `tests/unit/test_workflows_tool_contract.py` holds each declaration to its handler by AST-scanning for `args.get("x")`/`args["x"]` — the drift it hunts is a handler growing an input the schema never declares (`_broker_write`'s `target_field` is dynamic, so it has its own explicit test). **Golden workflow fixtures** (`evals/trajectories/workflows/*.json` + `test_workflow_fixtures.py`): whole workflows paired with an expected outcome, one generic runner; `expect.publishable: false` fixtures pin the publish gates. Tool schemas and destructive actions come from the REAL registry so fixtures break when the shipped catalog changes; fixtures assert which seams were crossed (`agent_calls`/`tool_calls`/`tool_args`), because "succeeded" while silently never calling the integration is the failure mode they exist to catch. 16. agents.json -- Dynamic agent registry (persisted alongside pyproject.toml) diff --git a/apps/services/gateway/gateway/routes/crm/auto_lead.py b/apps/services/gateway/gateway/routes/crm/auto_lead.py new file mode 100644 index 000000000..6d7e58d35 --- /dev/null +++ b/apps/services/gateway/gateway/routes/crm/auto_lead.py @@ -0,0 +1,582 @@ +"""CRM · auto_lead — an unknown inbound sender becomes a lead (WS-26d-autolead). + +Spec: ``ai-company-brain/specs/crm_app.md`` §9 ``WS-26d-autolead`` · D-CRM-9 · +D-CRM-12. Migration: ``crm_auto_lead_cursors``. + +**This module registers no routes.** Like ``broker_handlers``, it is deliberately +absent from ``routes/crm/__init__.py``: its one entry point, +:func:`create_leads_from_new_mail`, is called from the email app's shared +new-mail hook (``routes/email/scheduler_hooks.py::process_new_mail``) — the one +seam the background scheduler, the manual-sync route and the Graph webhook all +funnel through, so mail is considered identically however it arrived. + +**It lives in ``routes/crm`` and not in the email package** because what it +does is write a CRM record: it owns a CRM cursor table, it goes through the +CRM's own service write path, and it is gated by a CRM owner gate. It borrows +the email automation package's PUBLIC identity primitives +(``sender_scope`` / ``resolve_org_domains`` / ``normalize_domain``) rather than +copying them — D-CRM-4 declined to import another route package's *private* +helper, and "is this person a colleague?" already has one public answer across +the automation package. A third copy of it here would be the drift the rule +exists to prevent. + +Four properties are load-bearing, and each one is a way this feature can do +real damage rather than merely be wrong: + +1. **The flag is read BEFORE the step is entered.** ``CRM_AUTO_LEAD`` ships OFF + and :func:`auto_lead_enabled` is its single definition, but the *call site* + is what is guarded — with the flag off nothing here runs and no CRM query is + issued on the mail path at all. A short-circuit *inside* this module would + satisfy a careless test and still open a database session on every sync + cycle of every mailbox. + +2. **Two cursor predicates, together.** ``process_new_mail`` is also reached by + deep resyncs and by the first-ever sync of a newly connected mailbox, and + neither marks the mail it classifies as history. ``received_at > + activated_at`` is therefore the backfill discriminator (mail that ARRIVED + before auto-lead was first active on the account mints nothing, whenever it + is classified), and ``rules_processed_at > processed_watermark`` is the + incremental cursor. Without the first, connecting a second mailbox mints a + lead per unknown sender in a year of mail — each born ``zoho_dirty`` and + queued for the live Zoho tenant within one 600s cycle. + +3. **"Unknown" is three questions, and "external" is only necessary.** + :func:`_is_unknown_sender` mirrors ``senders._maybe_block_cold``'s two steps + (the cold-sender memo, then "have we ever emailed them") and adds a third + (no ``crm_contacts`` / ``crm_leads`` row already carries the address). + Separately, :func:`_is_external_sender` runs TWO gates, because + ``sender_scope`` fails SAFE to ``"external"`` — the wrong direction here. A + lead row for your own CFO, pushed into the live Zoho tenant, is what the + second gate exists to prevent. + +4. **The first activity is metadata, never content** (D-CRM-12 applied to what + a machine writes). ``type='system'`` — deliberately outside the Zoho push + predicate, so the activity never leaves the native CRM — subject and sender + in ``meta``, and ``body`` empty. Sender + subject is the proportionate + disclosure for a cold inbound inquiry on an org-visible record; the mail + body is not, and no snippet of it is either. This module never reads + ``body_text`` or ``snippet``. + +**One race is accepted and recorded** (spec §9): two concurrent +``process_new_mail`` invocations for one account can read the same watermark +and double-mint. The cost is one visible, hand-deletable duplicate lead. The +fix that suggests itself — a UNIQUE index on ``crm_leads.email`` — is a +deploy-blocking constraint on a column where 1,516 imported rows may already +carry duplicates, which is exactly the shape migration 148 had to defuse. Do +not add it. +""" + +from __future__ import annotations + +import json +from typing import Any + +from acb_auth import UserContext, UserRole +from acb_common import get_settings +from gateway.routes.crm.core import ( + LEADS, + LeadIn, + _get_db, + _log, + bump_last_activity, + insert_row, + now, + savepoint, +) +from gateway.routes.crm.records import create_record +from gateway.routes.email.automation.identity import ( + normalize_domain, + resolve_org_domains, + sender_scope, +) +from sqlalchemy import text + +#: How many candidate messages one cycle may consider. A cap rather than a +#: full drain because this runs inside the mail-sync hook: an unbounded batch +#: after a busy weekend would hold the hook (and, transitively, the account's +#: next sync) for as long as it took. The remainder is NOT dropped — the +#: watermark advances only over what was considered, so the next cycle picks it +#: up — and the overflow is COUNTED in the log line, because silent truncation +#: reads as "covered everything". +MAX_CANDIDATES_PER_CYCLE = 200 + +#: The activity ``type`` the originating message is logged as. **Not 'note'.** +#: ``sync_zoho.push_activities`` pushes ``type IN ('note', 'task')`` only, so +#: 'system' is how this row stays inside the native CRM — the mail's subject +#: and sender never reach the Zoho tenant even though the lead itself does +#: (D-CRM-9). ``tests/unit/test_crm_auto_lead.py`` pins the exclusion against +#: that statement's own text rather than trusting this comment. +ACTIVITY_TYPE = "system" + +#: ``crm_leads.source`` — the vocabulary is CHECK-constrained by migration 144 +#: and validated at the boundary by ``core.validate_source``. +LEAD_SOURCE = "email" + +#: The folder a candidate must be in, and the folder the "have we ever emailed +#: them" probe reads. Bound as parameters rather than written as SQL literals +#: so both are one lowercase comparison against a value the caller can see. +INBOX_FOLDER = "inbox" +SENT_FOLDER = "sent" + +#: The candidate predicate, written ONCE and shared by the fetch and the +#: overflow count. Two copies would let the count answer a different question +#: from the batch — and the count is the number an operator reads to decide +#: whether the cap is hurting. +_CANDIDATE_WHERE = ( + "account_id = :account_id " + "AND LOWER(folder) = :folder " + "AND rules_processed_at IS NOT NULL " + "AND rules_held_back_at IS NULL " + "AND received_at > :activated_at " + "AND rules_processed_at > :watermark" +) + +#: ⚠️ The projection is the privacy boundary: ``body_text`` and ``snippet`` are +#: absent on purpose and must stay absent. Nothing downstream can leak a body +#: it was never handed. +_CANDIDATE_SQL = ( + "SELECT id, subject, from_address, received_at, thread_id, " + "internet_message_id, rules_processed_at " + f"FROM email_messages WHERE {_CANDIDATE_WHERE} " + "ORDER BY rules_processed_at LIMIT :limit" +) + +_CANDIDATE_COUNT_SQL = f"SELECT COUNT(*) FROM email_messages WHERE {_CANDIDATE_WHERE}" + + +def auto_lead_enabled() -> bool: + """``CRM_AUTO_LEAD`` — the single definition of what the flag means. + + Ships OFF. ON means unknown inbound senders become CRM leads unattended, + each born ``zoho_dirty`` and therefore queued for the LIVE Zoho tenant + (D-CRM-9) with no confirmation card anywhere on a scheduler hook and no + delete tool. Flipping it is an OWNER-GATE act (``work_plan.md`` §6 (b)). + + Read by ``process_new_mail`` BEFORE it enters the step, never inside it — + see property 1 in the module docstring. + """ + return bool(getattr(get_settings(), "crm_auto_lead", False)) + + +def _new_stats() -> dict[str, int]: + """The counters the log line carries. Declared in one place so a counter + added later cannot be missing from the line that reports it.""" + return { + "candidates": 0, + "overflow": 0, + "created": 0, + "skipped_internal": 0, + "skipped_known": 0, + "skipped_unusable": 0, + "deduped_in_batch": 0, + "errors": 0, + } + + +async def create_leads_from_new_mail(account_id: str) -> dict[str, int]: + """Turn this account's newly-classified inbound mail into leads. + + The entry point ``process_new_mail`` calls, and the only public write path + in this module. Returns its counters so a caller (and a test) can see what + a cycle considered rather than only what it created — "created 0" and + "considered 0" are different facts and one of them is a bug report. + """ + stats = _new_stats() + db = await _get_db() + try: + account = await _load_account(db, account_id) + if account is None: + return stats + cursor = await _load_or_activate_cursor(db, account_id) + if cursor is None: # pragma: no cover — the row was just written + return stats + candidates = await _load_candidates(db, account_id, cursor) + stats["candidates"] = len(candidates) + if not candidates: + _emit(account_id, stats) + return stats + stats["overflow"] = await _count_overflow(db, account_id, cursor, + len(candidates)) + domains = await _internal_domains(db, account_id, account.email_address) + seen: set[str] = set() + for message in candidates: + await _consider(db, account, message, domains, seen, stats) + # Advanced over everything CONSIDERED, including the messages that + # minted nothing and the ones that raised: this is a best-effort + # enrichment step, not a queue, and a poison message that held the + # cursor still would re-fail on every cycle forever. The errors are + # counted in the line below instead. + await _advance_watermark( + db, account_id, + max(message.rules_processed_at for message in candidates), + ) + await db.commit() + _emit(account_id, stats) + return stats + finally: + await db.close() + + +# ── The account, and the cursor that decides what is history ──────────────── + +async def _load_account(db: Any, account_id: str) -> Any | None: + """The mailbox's own address and its owner. + + ``email_accounts.user_id`` holds the owner's EMAIL (the column is TEXT and + every reader compares it to one — see ``routes/email/core._account_scope``), + which is what makes a created lead somebody's follow-up rather than + 'anonymous'. An account with no owner recorded is skipped: attributing a + lead to nobody is worse than not creating it. + """ + row = (await db.execute(text( + "SELECT id, user_id, email_address FROM email_accounts WHERE id = :id" + ), {"id": account_id})).fetchone() + if row is None or not str(getattr(row, "user_id", "") or "").strip(): + return None + return row + + +async def _read_cursor(db: Any, account_id: str) -> Any | None: + return (await db.execute(text( + "SELECT account_id, activated_at, processed_watermark " + "FROM crm_auto_lead_cursors WHERE account_id = :account_id" + ), {"account_id": account_id})).fetchone() + + +async def _load_or_activate_cursor(db: Any, account_id: str) -> Any | None: + """Read the account's cursor, creating it on the first ON-state run. + + Activation stamps ``activated_at`` and ``processed_watermark`` to the SAME + instant, so the activating cycle itself mints nothing: everything already + in the mailbox arrived before auto-lead existed for this account, which is + precisely the deep-resync case. ``ON CONFLICT DO NOTHING`` + a re-read + means a concurrent activation is one row and one activation instant, not a + primary-key error on the mail path. + """ + row = await _read_cursor(db, account_id) + if row is not None: + return row + at = now() + await db.execute(text( + "INSERT INTO crm_auto_lead_cursors " + "(account_id, activated_at, processed_watermark) " + "VALUES (:account_id, :activated_at, :processed_watermark) " + "ON CONFLICT (account_id) DO NOTHING" + ), {"account_id": account_id, "activated_at": at, "processed_watermark": at}) + await db.commit() + _log.info("crm.auto_lead_activated", account_id=account_id) + return await _read_cursor(db, account_id) + + +def _cursor_params(account_id: str, cursor: Any) -> dict[str, Any]: + return { + "account_id": account_id, + "folder": INBOX_FOLDER, + "activated_at": cursor.activated_at, + "watermark": cursor.processed_watermark, + } + + +async def _load_candidates(db: Any, account_id: str, cursor: Any) -> list[Any]: + """Classified inbox mail this account has not been considered for yet. + + Both cursor predicates apply. Ordered by ``rules_processed_at`` ascending + so the batch's maximum IS the new watermark — ordering by ``received_at`` + would advance the cursor past messages the cap left behind. + """ + return (await db.execute( + text(_CANDIDATE_SQL), + {**_cursor_params(account_id, cursor), "limit": MAX_CANDIDATES_PER_CYCLE}, + )).fetchall() + + +async def _count_overflow( + db: Any, account_id: str, cursor: Any, taken: int, +) -> int: + """How many candidates the cap left for the next cycle. + + Only asked when the batch came back full — a short batch is the whole + remainder by definition, and a COUNT on every quiet cycle is a scan + nobody reads. + """ + if taken < MAX_CANDIDATES_PER_CYCLE: + return 0 + total = (await db.execute( + text(_CANDIDATE_COUNT_SQL), _cursor_params(account_id, cursor), + )).scalar() + return max(0, int(total or 0) - taken) + + +async def _advance_watermark(db: Any, account_id: str, watermark: Any) -> None: + await db.execute(text( + "UPDATE crm_auto_lead_cursors " + "SET processed_watermark = :processed_watermark, updated_at = now() " + "WHERE account_id = :account_id" + ), {"account_id": account_id, "processed_watermark": watermark}) + + +# ── Who the sender is ─────────────────────────────────────────────────────── + +async def _internal_domains( + db: Any, account_id: str, account_address: str | None, +) -> frozenset[str]: + """The account's own domain plus every configured org domain. + + Composed from the automation package's two PUBLIC helpers rather than + imported from ``cleanup._internal_domains`` (private) or restated here: + ``resolve_org_domains`` is the one place the ``org_domains`` setting is + normalised, and reading that column raw is a documented defect — a user who + typed ``ops@acme.com`` or ``@acme.com`` has their org recognised by + ``normalize_domain`` and NOT by an ad-hoc ``lstrip('@')``. + """ + domains = {normalize_domain(account_address or "")} + domains |= set(await resolve_org_domains(db, account_id)) + return frozenset(domain for domain in domains if domain) + + +def _is_external_sender( + address: str, account_address: str | None, internal_domains: frozenset[str], +) -> bool: + """Two gates, answering two different questions. Both must pass. + + **Gate 1** is ``sender_scope`` — the automation package's own answer to "is + this me, or my own domain?", asked from the account's address alone. + + **Gate 2** is the configured internal-domain list. It is not a restatement + of gate 1: ``sender_scope`` fails SAFE to ``"external"`` on an unparseable + address (the wrong direction when the consequence is a lead row in a live + Zoho tenant), and the extra ``org_domains`` reach it through a different + normalisation than ``resolve_org_domains`` applies — so a colleague on the + company's SECOND domain, configured by somebody who pasted an address + rather than a bare host, is external to gate 1 and internal to gate 2. + Routing the configured list through gate 2 only means there is exactly one + normalisation of it here rather than two that can disagree. + + An address with no domain at all fails gate 2: we do not mint a lead from + something we could not parse. + """ + if sender_scope(address, account_address or "") != "external": + return False + domain = normalize_domain(address) + return bool(domain) and domain not in internal_domains + + +async def _is_unknown_sender(db: Any, account_id: str, address: str) -> bool: + """Three steps — the two ``_maybe_block_cold`` already answers, plus ours. + + 1. the cold-sender memo: this account has already decided something about + this address (flagged cold, or whitelisted). Either way it is not a + stranger who just wrote in. + 2. have we ever emailed them. ⚠️ ``@>`` is exact, as Postgres is, so a + recipient stored with different casing escapes this probe — which is + part of why it is not the only step. + 3. ours: no CRM row already carries the address. A contact or a lead means + the company already knows this person, and a second lead for them is + the duplicate a human then has to merge. + """ + memo = (await db.execute(text( + "SELECT status FROM email_cold_senders " + "WHERE account_id = :account_id AND from_email = :address" + ), {"account_id": account_id, "address": address})).fetchone() + if memo: + return False + replied = (await db.execute(text( + "SELECT 1 FROM email_messages " + "WHERE account_id = :account_id AND LOWER(folder) = :folder " + "AND to_addresses @> :recipient LIMIT 1" + ), { + "account_id": account_id, "folder": SENT_FOLDER, + "recipient": json.dumps([{"email": address}]), + })).fetchone() + if replied: + return False + for table in ("crm_contacts", "crm_leads"): + known = (await db.execute(text( + f"SELECT 1 FROM {table} WHERE lower(email) = :address LIMIT 1" + ), {"address": address})).fetchone() + if known: + return False + return True + + +# ── One candidate ─────────────────────────────────────────────────────────── + +def _sender(message: Any) -> tuple[str, str]: + """``(address, display name)`` off the message's ``from_address`` JSONB. + + The driver hands back a dict or the raw text depending on how the row was + read; both shapes appear in this codebase, so both are handled here rather + than at four call sites. + """ + raw = getattr(message, "from_address", None) + if not isinstance(raw, dict): + try: + raw = json.loads(raw or "{}") + except (TypeError, ValueError): + raw = {} + if not isinstance(raw, dict): + raw = {} + return ( + str(raw.get("email") or "").strip().lower(), + str(raw.get("name") or "").strip(), + ) + + +def _split_display_name(raw: str) -> tuple[str | None, str | None]: + """The mail's display name → ``(first_name, last_name)``. + + **Stripped before it is split.** Providers pad and quote this field, and + splitting first turns ``'"Asha" '`` into a first name of ``'"Asha"'`` and a + surname of ``''``. What lands here is then run through §3.3's fallback + chain by ``create_record`` — this function must never build the display + name itself, because a name hand-built here would be a SECOND answer to + "what is this lead called" and the wrong one is the one that gets stored + (the "Asha Asha" trap, §8 B-series). + + A display name that is an address — which is what every provider puts there + for a sender who set none — yields ``(None, None)``, so the chain falls + through to the email local part rather than minting a lead named + ``noreply@example.com``. + """ + name = (raw or "").strip().strip('"').strip("'").strip() + if not name or "@" in name: + return None, None + parts = name.split() + if len(parts) == 1: + return parts[0], None + return parts[0], " ".join(parts[1:]) + + +def _principal(owner_email: str) -> UserContext: + """The identity ``create_record`` attributes the lead to. + + Carries the account owner's address and **no access at all** + (``UserContext`` defaults to ``NO_ACCESS``): the only thing this principal + is for is the ``owner_email`` default, and an unattended writer must not + carry authority it could later be asked for. ``UserRole.AGENT`` says out + loud that no human is behind this write — the same principal shape + ``routes/apps/actions.py`` uses for an agent-initiated action. + """ + return UserContext(email=owner_email.strip(), role=UserRole.AGENT) + + +async def _consider( + db: Any, account: Any, message: Any, internal_domains: frozenset[str], + seen: set[str], stats: dict[str, int], +) -> None: + """Decide about ONE candidate message, and count the decision.""" + address, display_name = _sender(message) + if not address: + stats["skipped_unusable"] += 1 + return + if address in seen: + # In-batch de-duplication. Not an optimisation: the SELECT guard in + # `_is_unknown_sender` reads a row `create_record` has committed on + # ANOTHER session, so without this a sender who wrote twice in one + # batch is a coin flip between one lead and two. + stats["deduped_in_batch"] += 1 + return + seen.add(address) + if not _is_external_sender(address, account.email_address, internal_domains): + stats["skipped_internal"] += 1 + return + if not await _is_unknown_sender(db, str(account.id), address): + stats["skipped_known"] += 1 + return + try: + async with savepoint(db): + await _mint_lead(db, account, message, address, display_name) + except Exception as exc: + stats["errors"] += 1 + _log.warning("crm.auto_lead_message_failed", + account_id=str(account.id), + message_id=str(getattr(message, "id", "")), + error=str(exc)[:200]) + return + stats["created"] += 1 + + +async def _mint_lead( + db: Any, account: Any, message: Any, address: str, display_name: str, +) -> None: + """Create the lead, then log the originating message against it. + + The lead goes through ``records.create_record`` — never a raw INSERT. + That path is where ``_resolve_status`` (the NOT NULL ``status_id``), the + ``owner_email`` default, ``validate_source`` and, through ``insert_row``, + ``mark_dirty_on_insert`` all live; raw SQL silently loses all four, and + only the last of them is visible in the row afterwards. + + ``create_record`` opens and commits its OWN session (it is the same + function ``POST /crm/leads`` calls), so the lead is committed before the + activity is written. A failure between the two therefore leaves a lead with + an empty timeline — logged, counted, and the lesser of the two evils: the + alternative is a second, divergent write path for the record itself. + """ + first_name, last_name = _split_display_name(display_name) + # Absent, never explicitly null: ``clean_payload`` is ``exclude_unset``, so + # a field passed as None is a deliberate "clear this" that lands in the + # INSERT as a NULL column. Omitting it lets the column's own default apply, + # which is the difference between "we did not learn a surname" and "this + # lead has no surname". + fields: dict[str, Any] = {"email": address, "source": LEAD_SOURCE} + if first_name: + fields["first_name"] = first_name + if last_name: + fields["last_name"] = last_name + lead = await create_record( + LEADS, LeadIn(**fields), _principal(str(account.user_id)), + ) + await _log_origin_activity(db, lead, account, message, address, display_name) + + +async def _log_origin_activity( + db: Any, lead: dict[str, Any], account: Any, message: Any, + address: str, display_name: str, +) -> None: + """The lead's first timeline entry: metadata about the mail, never the mail. + + ``type='system'`` keeps it out of ``sync_zoho.push_activities``' ``type IN + ('note', 'task')`` predicate, so the subject and the sender stay inside the + native CRM even though the lead itself is queued for the tenant (D-CRM-9). + ``body`` is empty and stays empty: the record is org-visible to every + ``feature:crm`` holder (D-CRM-3), and sender + subject is the proportionate + disclosure for a cold inbound inquiry — the same line D-CRM-12 draws for + what the agent renders, applied to what a machine writes. + """ + await insert_row(db, "crm_activities", { + "type": ACTIVITY_TYPE, + "subject": getattr(message, "subject", None), + "body": None, + "occurred_at": getattr(message, "received_at", None), + "meta": { + "source": "auto_lead", + "sender_name": display_name or None, + "sender_address": address, + "received_at": _instant(getattr(message, "received_at", None)), + "message_id": str(getattr(message, "id", "") or "") or None, + "internet_message_id": getattr(message, "internet_message_id", None), + "thread_id": getattr(message, "thread_id", None), + "account_id": str(account.id), + }, + "created_by": str(account.user_id), + LEADS.activity_column: lead["id"], + }) + await bump_last_activity(db, LEADS.table, lead["id"]) + + +def _instant(value: Any) -> str | None: + """A timestamp as text for the ``meta`` blob — jsonb holds no datetimes.""" + if value is None: + return None + isoformat = getattr(value, "isoformat", None) + return isoformat() if callable(isoformat) else str(value) + + +def _emit(account_id: str, stats: dict[str, int]) -> None: + """One line per cycle, carrying every counter — including ``overflow``. + + ⚠️ Never pass ``event=`` to a structlog logger; it is the message parameter + and raises at call time. + """ + _log.info("crm.auto_lead_cycle", account_id=account_id, **stats) diff --git a/apps/services/gateway/gateway/routes/email/scheduler_hooks.py b/apps/services/gateway/gateway/routes/email/scheduler_hooks.py index 1cdcaf17e..fb82d310c 100644 --- a/apps/services/gateway/gateway/routes/email/scheduler_hooks.py +++ b/apps/services/gateway/gateway/routes/email/scheduler_hooks.py @@ -56,7 +56,8 @@ async def auto_run_rules_for_account(account_id: str) -> None: async def process_new_mail(account_id: str) -> None: """The shared new-mail pipeline (H1): auto-run rules → sweep the leftovers → - categorize senders → classify threads (Reply Zero) → auto-archive. + categorize senders → classify threads (Reply Zero) → auto-archive → CRM + auto-lead. Order matters. The rules run first and are the only thing that *classifies*. The sweep then projects that (plus learned patterns) onto inbox mail the @@ -66,6 +67,13 @@ async def process_new_mail(account_id: str) -> None: invisible to the Email Cleaner. Sender rollup runs after both so it sees the complete label set. + The CRM auto-lead step (WS-26d-autolead) runs LAST, and after auto-archive + on purpose: it considers what is still in the INBOX once the account's own + automation has finished with it, so mail the user's rules said "I do not + need to see this" about never becomes a lead. It is also the only step here + that belongs to another app — it lives in ``routes/crm/auto_lead.py``, + because what it does is write a CRM record. + Each step is isolated so one failure never skips the rest (same guarantee the scheduler loop gave when these were separate steps). Registered as the ``on_new_mail`` hook AND called directly by the manual-sync route + webhook, @@ -110,6 +118,24 @@ async def process_new_mail(account_id: str) -> None: except Exception as exc: # noqa: BLE001 _log.warning("sync.auto_archive_failed", account_id=account_id, error=str(exc)[:200]) + try: + # WS-26d-autolead. ⚠️ The flag is checked HERE, before the step is + # entered — never inside it. With CRM_AUTO_LEAD off no CRM code runs + # and no CRM query is issued on the mail path at all; a gate that + # lived inside `create_leads_from_new_mail` would open a database + # session on every sync cycle of every mailbox to discover it had + # nothing to do. `auto_lead_enabled` is the flag's ONE definition and + # is imported rather than restated for the same reason. + from gateway.routes.crm.auto_lead import ( + auto_lead_enabled, + create_leads_from_new_mail, + ) + + if auto_lead_enabled(): + await create_leads_from_new_mail(account_id) + except Exception as exc: # noqa: BLE001 + _log.warning("sync.auto_lead_failed", account_id=account_id, + error=str(exc)[:200]) async def learn_label_changes(account_id: str, changes: list) -> None: diff --git a/infra/AGENTS.md b/infra/AGENTS.md index fa747e818..da0f83ed8 100644 --- a/infra/AGENTS.md +++ b/infra/AGENTS.md @@ -5,7 +5,7 @@ Docker Compose, Postgres schema, LiteLLM tier config. LLM routing is via the gat ## Key Files - docker-compose.yml -- core services (Postgres 16 + pgvector, Redis 7) -- postgres/ -- schema files (00-10) + 09_app_user.sql (NextAuth users) + 11_integration_credentials.sql (unified credential store) + 130_org_access_control.sql (organization, membership lifecycle on app_user, org_role/org_role_permission/user_role, user_permission_override, feature_catalog — spec: ai-company-brain/specs/org_access_control.md) + 131_integration_memory_permissions.sql (additive: grants `integrations:use:*` + org-memory permissions to the seeded roles; `member` reads org memory but does not write it) + 143_access_request.sql (the sign-in queue — one row per address that authenticated with no `app_user` row, unique on `lower(email)`; `status` carries a `CHECK (pending|approved|denied)` because the vocabulary is load-bearing for ACCESS — approve acts only on `pending`, so a typo'd status would fall out of both the queue and the decided record; deliberately a SEPARATE table and not a fifth `app_user.status`, because an `app_user` row IS the org's member record and a stranger who merely knocked must not acquire one that a future join can surface — spec: ai-company-brain/specs/colleague_onboarding.md §6) + 144_crm.sql (the native CRM spine — organizations/contacts/leads/deals, **statuses as data** rather than enums, one `crm_activities` timeline whose four target FKs are all nullable under a CHECK requiring at least one, and a `crm_status_changes` dwell log; also seeds the `crm` feature_catalog row. Contains the schema's **one FK cycle** — `crm_leads.converted_deal_id` ⟷ `crm_deals.lead_id` — closed by a guarded `DO $$` on `pg_constraint`, because `ALTER TABLE … ADD CONSTRAINT` has no `IF NOT EXISTS`. Idempotency is pinned STATICALLY by `tests/unit/test_crm_migration.py`, which reads the file as text: the unit suite runs no database, so an idempotency claim that holds only by inspection is unenforceable. Spec: ai-company-brain/specs/crm_app.md §3) + 145_crm_zoho_sync.sql (what the two-way Zoho sync needs: `zoho_dirty`/`zoho_synced_at` on the four CRM record tables — and **only** those four, since pipeline vocabulary flows down-only and an activity's push signal is its NULL `zoho_id` — plus `crm_zoho_tombstones` (FK-less on purpose: the row it describes is gone by the time anyone reads it) and `crm_sync_cursors` (`module` PK, so the pull cannot silently rewind). Idempotent via `ADD COLUMN IF NOT EXISTS`, which — unlike `ADD CONSTRAINT` — Postgres supports directly, so no guarded `DO $$` is needed. Pinned by the same static `tests/unit/test_crm_migration.py`, which finds BOTH CRM migrations by CONTENT rather than by number. Spec: crm_app.md §7.1). ⚠️ `CREATE TABLE IF NOT EXISTS` means a column or constraint added to an ALREADY-APPLIED migration file is silently skipped on that deployment; 143, 144 and 145 have never been applied anywhere (merging 143 is the OWNER-GATE), which is the only reason they are still editable in place. +- postgres/ -- schema files (00-10) + 09_app_user.sql (NextAuth users) + 11_integration_credentials.sql (unified credential store) + 130_org_access_control.sql (organization, membership lifecycle on app_user, org_role/org_role_permission/user_role, user_permission_override, feature_catalog — spec: ai-company-brain/specs/org_access_control.md) + 131_integration_memory_permissions.sql (additive: grants `integrations:use:*` + org-memory permissions to the seeded roles; `member` reads org memory but does not write it) + 143_access_request.sql (the sign-in queue — one row per address that authenticated with no `app_user` row, unique on `lower(email)`; `status` carries a `CHECK (pending|approved|denied)` because the vocabulary is load-bearing for ACCESS — approve acts only on `pending`, so a typo'd status would fall out of both the queue and the decided record; deliberately a SEPARATE table and not a fifth `app_user.status`, because an `app_user` row IS the org's member record and a stranger who merely knocked must not acquire one that a future join can surface — spec: ai-company-brain/specs/colleague_onboarding.md §6) + 144_crm.sql (the native CRM spine — organizations/contacts/leads/deals, **statuses as data** rather than enums, one `crm_activities` timeline whose four target FKs are all nullable under a CHECK requiring at least one, and a `crm_status_changes` dwell log; also seeds the `crm` feature_catalog row. Contains the schema's **one FK cycle** — `crm_leads.converted_deal_id` ⟷ `crm_deals.lead_id` — closed by a guarded `DO $$` on `pg_constraint`, because `ALTER TABLE … ADD CONSTRAINT` has no `IF NOT EXISTS`. Idempotency is pinned STATICALLY by `tests/unit/test_crm_migration.py`, which reads the file as text: the unit suite runs no database, so an idempotency claim that holds only by inspection is unenforceable. Spec: ai-company-brain/specs/crm_app.md §3) + 145_crm_zoho_sync.sql (what the two-way Zoho sync needs: `zoho_dirty`/`zoho_synced_at` on the four CRM record tables — and **only** those four, since pipeline vocabulary flows down-only and an activity's push signal is its NULL `zoho_id` — plus `crm_zoho_tombstones` (FK-less on purpose: the row it describes is gone by the time anyone reads it) and `crm_sync_cursors` (`module` PK, so the pull cannot silently rewind). Idempotent via `ADD COLUMN IF NOT EXISTS`, which — unlike `ADD CONSTRAINT` — Postgres supports directly, so no guarded `DO $$` is needed. Pinned by the same static `tests/unit/test_crm_migration.py`, which finds BOTH CRM migrations by CONTENT rather than by number. Spec: crm_app.md §7.1) + 157_crm_auto_lead_cursor.sql (WS-26d-autolead: `crm_auto_lead_cursors`, one row per `email_accounts` id, carrying `activated_at` — stamped ONCE on the first ON-state run and **never advanced**, because `received_at > activated_at` is the only thing that tells a deep resync's year-old backlog apart from new mail on the shared `process_new_mail` hook — and `processed_watermark`, the incremental cursor. Both NOT NULL: a NULL cursor is a predicate that matches nothing, which reads exactly like a working feature. ⚠️ **It deliberately adds NO unique index on `crm_leads.email`**: the cross-invocation double-mint race is accepted (one visible, hand-deletable duplicate) because a UNIQUE constraint on a column where 1,516 imported rows may already carry duplicates is a deploy-blocking migration of exactly the shape 148 had to defuse. Inert until `CRM_AUTO_LEAD` is flipped, which is OWNER-GATE. Spec: crm_app.md §9 WS-26d-autolead). ⚠️ `CREATE TABLE IF NOT EXISTS` means a column or constraint added to an ALREADY-APPLIED migration file is silently skipped on that deployment; 143, 144 and 145 have never been applied anywhere (merging 143 is the OWNER-GATE), which is the only reason they are still editable in place. ## Conventions - Postgres migrations are numbered SQL files diff --git a/infra/postgres/157_crm_auto_lead_cursor.sql b/infra/postgres/157_crm_auto_lead_cursor.sql new file mode 100644 index 000000000..cc859b841 --- /dev/null +++ b/infra/postgres/157_crm_auto_lead_cursor.sql @@ -0,0 +1,82 @@ +-- 157_crm_auto_lead_cursor.sql — WS-26d-autolead +-- +-- What: one row per email account recording when CRM auto-lead first became +-- active on that mailbox, and how far the step has processed. +-- Why: the auto-lead step hangs off `process_new_mail`, and that hook is +-- reached by DEEP RESYNCS as well as by new mail. `resync_account` runs +-- a ~1-year all-folder backfill and then fires the hook; a first-ever +-- sync of a newly connected mailbox is deep by the same heuristic; and +-- neither path stamps `rules_held_back_at` (its only writer is +-- `_backfill_and_clean_job`, which does not go through this hook). A +-- candidate query of "everything classified" would therefore mint a lead +-- per unknown external sender across a YEAR of mail the moment a second +-- mailbox connects — each born `zoho_dirty`, each queued for the live +-- Zoho tenant within one 600s cycle (D-CRM-9), with no confirmation card +-- anywhere on a scheduler hook and no delete tool to take them back. +-- +-- Two timestamps, because "is this message history?" and "have I already +-- looked at this message?" are different questions and one column cannot +-- answer both: +-- +-- activated_at set ONCE, on the step's first ON-state run for +-- the account, and NEVER advanced. The backfill +-- discriminator is `received_at > activated_at`: +-- mail that ARRIVED before auto-lead was first +-- active mints nothing, no matter when a resync +-- gets around to classifying it. A moving cursor +-- cannot express that — it would let a resync +-- re-present year-old mail as newly processed. +-- +-- processed_watermark the incremental cursor, compared against +-- `rules_processed_at` and advanced only after a +-- batch has been written. It is what makes a +-- re-run of the same sync consider nothing. +-- +-- Both predicates apply together. `processed_watermark` starts equal to +-- `activated_at`, so the activating run itself mints nothing. +-- +-- ⚠️ There is deliberately NO unique index on `crm_leads.email` to go +-- with this. Two concurrent `process_new_mail` invocations for one +-- account can read the same watermark and double-mint; the cost is one +-- visible, hand-deletable duplicate lead, and the alternative — a UNIQUE +-- constraint minted on a column where 1,516 imported rows may already +-- carry duplicates — is a deploy-blocking constraint of exactly the shape +-- migration 148 had to defuse. The accepted race is recorded in +-- `crm_app.md` §9 WS-26d-autolead. Do not "fix" it with that index. +-- +-- Spec: ai-company-brain/specs/crm_app.md §9 WS-26d-autolead (the cursor +-- paragraph) · D-CRM-9. +-- Depends on: 17_email_accounts.sql (email_accounts, the FK target) and +-- 144_crm.sql (the CRM spine this cursor guards writes into). +-- +-- Idempotent: CREATE TABLE / CREATE INDEX IF NOT EXISTS only. No seed, no +-- ALTER, nothing dropped. + +BEGIN; + +CREATE TABLE IF NOT EXISTS crm_auto_lead_cursors ( + -- The mailbox, not the CRM record: the step is per account because + -- `process_new_mail` is. CASCADE because a disconnected mailbox's cursor + -- describes nothing — the leads it already minted are CRM rows and are + -- untouched by this. + account_id UUID PRIMARY KEY + REFERENCES email_accounts (id) ON DELETE CASCADE, + + -- Written once, by the first ON-state run. Never advanced. See above. + activated_at TIMESTAMPTZ NOT NULL, + + -- Advanced to MAX(rules_processed_at) of each committed batch. + processed_watermark TIMESTAMPTZ NOT NULL, + + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT now() +); + +-- The candidate query reads this row by primary key, so no second index is +-- needed here. This one supports the operator question the log line raises — +-- "which mailboxes has auto-lead ever been active on?" — without a seq scan +-- growing with the number of connected accounts. +CREATE INDEX IF NOT EXISTS idx_crm_auto_lead_cursors_activated_at + ON crm_auto_lead_cursors (activated_at); + +COMMIT; diff --git a/packages/acb_common/acb_common/settings.py b/packages/acb_common/acb_common/settings.py index 8e40976cd..5a2d1d830 100644 --- a/packages/acb_common/acb_common/settings.py +++ b/packages/acb_common/acb_common/settings.py @@ -120,6 +120,17 @@ class Settings(BaseSettings): # deletes both ways. Flipping it is an OWNER-GATE act (work_plan.md §6). crm_zoho_sync: bool = False + # CRM auto-lead from inbound email (spec crm_app.md §9 WS-26d-autolead, + # D-CRM-9). Gates ONLY the CRM step inside + # `routes/email/scheduler_hooks.py::process_new_mail`, and the gate is read + # BEFORE the step is entered, so with this off no CRM code runs and no CRM + # query is issued on the mail path at all. Ships OFF: ON means unknown + # inbound senders become `crm_leads` rows unattended — each born + # `zoho_dirty`, i.e. queued for the LIVE Zoho tenant on the next sync cycle, + # with no confirmation card anywhere on a scheduler hook and no delete tool. + # Flipping it is an OWNER-GATE act (work_plan.md §6 (b)). + crm_auto_lead: bool = False + # Gmail (Phase 1, WBS 1.3) gmail_sa_json_path: str = "" # service-account key file gmail_workspace_domain: str = "" # e.g. fracktal.in diff --git a/tests/unit/_crm_fakes.py b/tests/unit/_crm_fakes.py index 3c3697ecd..4d709c46e 100644 --- a/tests/unit/_crm_fakes.py +++ b/tests/unit/_crm_fakes.py @@ -32,6 +32,7 @@ from __future__ import annotations +import json import re from datetime import UTC, datetime, timedelta from types import SimpleNamespace @@ -154,6 +155,16 @@ r"LOWER\((?:(\w+)\.)?(\w+)->>'(\w+)'\)\s*(?:=\s*:(\w+)|IN\s*\(([^)]*)\))", re.I, ) +#: ``to_addresses @> :recipient`` — jsonb CONTAINMENT. The "have we ever +#: emailed them" probe `senders._maybe_block_cold` answers with, and the one +#: WS-26d-autolead's `_is_unknown_sender` mirrors. **No other reader here sees +#: `@>`**: `_PLAIN_EQ` needs an `=` and `_NUM_CMP` needs a bare `>`, so without +#: this the clause is invisible and the fake answers "yes, we have emailed +#: them" for every message in the Sent folder regardless of who it went to — +#: which turns the already-known-contact case into a test of nothing. Same +#: reason `_IN_LITERALS` exists: a fake that cannot see a predicate agrees with +#: the bug that deletes it. +_JSONB_CONTAINS = re.compile(r"(?:(\w+)\.)?(\w+)\s*@>\s*:(\w+)", re.I) #: ``LEFT JOIN email_thread_status ts ON ts.account_id = em.account_id AND #: ts.thread_id = em.thread_id`` — a COMPOSITE key. `_LEFT_JOIN` above demands #: literally ``ON .id = base.`` and matches nothing here. @@ -747,6 +758,13 @@ def _email_predicates( row for row in rows if str((row.get(column) or {}).get(key) or "").lower() in wanted ] + for _alias, column, param in _JSONB_CONTAINS.findall(where): + seen = True + rows = [ + row for row in rows + if _jsonb_contains(row.get(column), args.get(param)) + ] + where = _JSONB_CONTAINS.sub("", where) return _JSONB_LOWER_CMP.sub("", where), rows, seen def _matching( @@ -875,6 +893,37 @@ def _weighted(statement: str, rows: list[dict], args: dict) -> float: return total +def _jsonb_contains(haystack: Any, needle: Any) -> bool: + """Postgres ``@>`` for the shapes this repo binds to it. + + An array contains another array when EVERY element of the right side is + contained by SOME element of the left; an object contains another object + when every key/value pair of the right side is present on the left. + + ⚠️ Comparison is EXACT, because ``@>`` is: ``'[{"email":"A@x.com"}]'`` does + not contain ``'[{"email":"a@x.com"}]'``. Case-folding here would make the + fake kinder than the database and hide a real miss in the "have we ever + emailed them" probe, whose parameter is lowercased while the stored + recipient is whatever the provider sent. + """ + if isinstance(needle, str): + try: + needle = json.loads(needle) + except ValueError: # a plain string value, not a JSON document + return haystack == needle + if isinstance(needle, list): + candidates = haystack if isinstance(haystack, list) else [] + return all( + any(_jsonb_contains(item, want) for item in candidates) + for want in needle + ) + if isinstance(needle, dict): + return isinstance(haystack, dict) and all( + haystack.get(key) == value for key, value in needle.items() + ) + return haystack == needle + + def _normalized(value: str, wrap: str) -> str: """Apply the normalisation the predicate wraps its column in, then lower(). diff --git a/tests/unit/test_crm_auto_lead.py b/tests/unit/test_crm_auto_lead.py new file mode 100644 index 000000000..2d5b195a8 --- /dev/null +++ b/tests/unit/test_crm_auto_lead.py @@ -0,0 +1,1092 @@ +"""CRM · auto-lead from inbound email (WS-26d-autolead). + +Spec: ``ai-company-brain/specs/crm_app.md`` §9 ``WS-26d-autolead`` — +done-when 1-7, each named in a test below · D-CRM-9 · D-CRM-12. + +**What this file is defending.** With ``CRM_AUTO_LEAD`` on, a scheduler hook +writes CRM records with no human in the loop, and by D-CRM-9 each one is born +``zoho_dirty`` and reaches the LIVE Zoho tenant within one 600s sync cycle. +There is no confirmation card on a scheduler hook and there is no delete tool. +So the tests that matter are the ones about what it must NOT do: + +* ``test_dw2_*`` — with the flag off the step is not entered. Asserted twice: + once at runtime (a sentinel that raises if it is called) and once + structurally (the call is lexically inside an ``if auto_lead_enabled()``), so + a future refactor that moves the gate INSIDE the step fails here rather than + quietly opening a database session per mailbox per cycle. +* ``test_dw7_*`` — a deep resync of a newly connected mailbox mints NOTHING. + ``process_new_mail`` is reached by ~1-year backfills that stamp no + held-back marker, so "everything classified" would mint a lead per unknown + sender across a year of mail. +* ``test_dw5_*`` — colleague, self and already-known senders create nothing. + Two colleague cases, because there are two gates and they answer different + questions: same-domain is ``sender_scope``'s, a configured second org domain + is the internal-domain list's. +* ``test_dw6_*`` — the lead IS born ``zoho_dirty`` (that is the design, not an + accident), and its first activity's ``type='system'`` is asserted absent from + ``sync_zoho``'s own push statement rather than from a comment about it. + +Hermetic, in ``test_crm_routes.py``'s convention: no Postgres, no network, no +TestClient; the step is called directly with ``_get_db`` monkeypatched onto the +SUT submodules against the shared ``_crm_fakes.FakeCrmDB``. The real settings +object is never mutated — the ON state is the module's own +``auto_lead_enabled`` predicate, monkeypatched, exactly as +``test_crm_zoho_sync.py`` does for ``sync_enabled``. +""" + +from __future__ import annotations + +import ast +import inspect +import json +import re +from datetime import UTC, datetime, timedelta +from pathlib import Path +from typing import Any + +import pytest +from gateway.routes.crm import auto_lead +from gateway.routes.crm import core as crm_core +from gateway.routes.crm import pipeline as crm_pipeline +from gateway.routes.crm import records as crm_records +from gateway.routes.crm import sync_zoho as crm_sync +from gateway.routes.email import scheduler_hooks + +from tests.unit._crm_fakes import FakeCrmDB, bind_db + +REPO = Path(__file__).resolve().parents[2] +MIGRATIONS = REPO / "infra" / "postgres" +HOOKS = REPO / "apps/services/gateway/gateway/routes/email/scheduler_hooks.py" + +#: One mailbox: `vjvarada@fracktal.in`, owned by the same person. +#: `email_accounts.user_id` holds the owner's EMAIL — see +#: `routes/email/core._account_scope`, which compares it to one. +ACCOUNT_ID = "11111111-1111-1111-1111-111111111111" +OWNER = "vjvarada@fracktal.in" + +STRANGER = "asha@acmerobotics.com" + + +def _at(day: int, hour: int = 9, *, year: int = 2026) -> datetime: + return datetime(year, 8, day, hour, 0, tzinfo=UTC) + + +#: When auto-lead became active on this account, in every test that does not +#: care about activation itself. +ACTIVATED = _at(1) + + +# ── Fixtures ──────────────────────────────────────────────────────────────── + +@pytest.fixture +def db(monkeypatch: pytest.MonkeyPatch) -> FakeCrmDB: + """The in-memory schema, bound to every module that opens a session. + + ``auto_lead`` is in the tuple because it imports ``_get_db`` from ``core`` + by name; ``records`` is in it because ``create_record`` opens its OWN + session and the lead has to land in the same fake. + """ + fake = FakeCrmDB() + bind_db(monkeypatch, fake, + (crm_core, crm_records, crm_pipeline, auto_lead)) + return fake + + +@pytest.fixture +def on(monkeypatch: pytest.MonkeyPatch) -> None: + """The ON state, without touching the real settings object. + + Monkeypatching the module's own predicate is the pattern + ``test_crm_zoho_sync.py`` uses for ``CRM_ZOHO_SYNC``: flipping the shared + ``get_settings()`` singleton leaks an owner-gated flag into every test that + runs afterwards. + """ + monkeypatch.setattr(auto_lead, "auto_lead_enabled", lambda: True) + + +@pytest.fixture +def quiet_pipeline(monkeypatch: pytest.MonkeyPatch) -> None: + """Silence the five mail steps ``process_new_mail`` runs before ours. + + They are each try/except-isolated, so leaving them live would not fail the + test — it would make it spend its time opening real database connections + to discover that. + """ + from gateway.routes.email.automation import cleanup, replyzero, senders + + async def _noop(*args: Any, **kwargs: Any) -> None: + return None + + monkeypatch.setattr(scheduler_hooks, "auto_run_rules_for_account", _noop) + monkeypatch.setattr(cleanup, "sweep_uncategorized", _noop) + monkeypatch.setattr(senders, "_categorize_senders_job", _noop) + monkeypatch.setattr(senders, "_maybe_auto_archive", _noop) + monkeypatch.setattr(replyzero, "_maybe_classify_threads", _noop) + + +# ── Seeding ───────────────────────────────────────────────────────────────── + +def _seed_account( + db: FakeCrmDB, *, owner: str = OWNER, address: str = OWNER, + org_domains: list[str] | None = None, +) -> None: + db.seed("email_accounts", id=ACCOUNT_ID, user_id=owner, + email_address=address) + if org_domains is not None: + db.seed("email_assistant_settings", account_id=ACCOUNT_ID, + org_domains=org_domains) + + +def _seed_status(db: FakeCrmDB) -> Any: + """The lane a new lead lands in — ``_resolve_status`` requires one.""" + return db.seed("crm_lead_statuses", name="New", is_default=True, + position=0, type="open") + + +def _seed_cursor( + db: FakeCrmDB, *, activated_at: datetime = ACTIVATED, + watermark: datetime | None = None, +) -> Any: + return db.seed("crm_auto_lead_cursors", account_id=ACCOUNT_ID, + activated_at=activated_at, + processed_watermark=watermark or activated_at) + + +#: "you did not say" — distinct from an explicit ``None``, which is how a test +#: seeds mail the rules have NOT classified. ``processed_at=None`` defaulting +#: to ``received_at`` would silently classify the one message that must not be. +_UNSET = object() + + +def _message( + db: FakeCrmDB, *, address: str = STRANGER, name: str = "Asha Menon", + received_at: datetime | None = None, processed_at: Any = _UNSET, + subject: str = "Quote for 40 printers", folder: str = "INBOX", + held_back_at: datetime | None = None, thread_id: str = "t-1", + body_text: str = "SECRET BODY — must never leave the mailbox", +) -> Any: + received_at = received_at or _at(5) + if processed_at is _UNSET: + processed_at = received_at + return db.seed( + "email_messages", account_id=ACCOUNT_ID, folder=folder, + from_address={"name": name, "email": address}, + to_addresses=[{"email": OWNER}], subject=subject, + body_text=body_text, snippet=body_text[:40], + received_at=received_at, + rules_processed_at=processed_at, + rules_held_back_at=held_back_at, + thread_id=thread_id, internet_message_id="", + ) + + +def _ready(db: FakeCrmDB) -> None: + """The steady state: an activated account with a pipeline lane.""" + _seed_account(db) + _seed_status(db) + _seed_cursor(db) + + +def _leads(db: FakeCrmDB) -> list[dict[str, Any]]: + return db.rows("crm_leads") + + +def _activities(db: FakeCrmDB) -> list[dict[str, Any]]: + return db.rows("crm_activities") + + +def _meta(row: dict[str, Any]) -> dict[str, Any]: + """``meta`` as the route bound it — jsonb rides the wire as text.""" + value = row.get("meta") + return json.loads(value) if isinstance(value, str) else (value or {}) + + +# ── done-when 1: the flag exists and ships OFF ────────────────────────────── + +def test_dw1_the_flag_ships_off() -> None: + from acb_common import get_settings + + assert get_settings().crm_auto_lead is False + assert auto_lead.auto_lead_enabled() is False + + +def test_dw1_the_flag_is_a_settings_field_not_org_settings() -> None: + """``acb_common/settings.py``, beside ``crm_zoho_sync`` — the precedent + shape for a flag whose flip is an owner gate. An `org_settings` row would + put an OWNER-GATE flip behind an admin UI.""" + from acb_common.settings import Settings + + assert "crm_auto_lead" in Settings.model_fields + assert Settings.model_fields["crm_auto_lead"].default is False + + +def test_dw1_the_flag_is_not_written_into_the_env_example() -> None: + """Deliberate absence. ``.env.example`` is plan-guard territory; the flag + is documented in ``settings.py`` and ``crm_app.md`` and nowhere else.""" + example = (REPO / ".env.example") + if not example.exists(): # pragma: no cover — it is committed + pytest.skip(".env.example is absent from this checkout") + assert "CRM_AUTO_LEAD" not in example.read_text(encoding="utf-8").upper() + + +# ── done-when 2: the OFF state never enters the step ──────────────────────── + +async def test_dw2_with_the_flag_off_the_step_is_not_entered( + db: FakeCrmDB, quiet_pipeline: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + """The flag ships off, so this is the DEFAULT behaviour of every sync. + + Asserted as "the step was never called", not as "the step created + nothing": a gate that lived inside `create_leads_from_new_mail` would pass + the second assertion while still opening a session per mailbox per cycle. + """ + calls: list[str] = [] + + async def _sentinel(account_id: str) -> dict[str, int]: + calls.append(account_id) + raise AssertionError("the CRM step ran with CRM_AUTO_LEAD off") + + monkeypatch.setattr(auto_lead, "create_leads_from_new_mail", _sentinel) + _ready(db) + _message(db, received_at=_at(5)) + + await scheduler_hooks.process_new_mail(ACCOUNT_ID) + + assert calls == [] + assert db.statements == [] + assert _leads(db) == [] + + +async def test_dw2_with_the_flag_on_the_step_is_entered( + db: FakeCrmDB, on: None, quiet_pipeline: None, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The control for the test above: without this, "never called" would also + be satisfied by a hook that lost the call entirely.""" + calls: list[str] = [] + + async def _recorder(account_id: str) -> dict[str, int]: + calls.append(account_id) + return {} + + monkeypatch.setattr(auto_lead, "create_leads_from_new_mail", _recorder) + + await scheduler_hooks.process_new_mail(ACCOUNT_ID) + + assert calls == [ACCOUNT_ID] + + +def _process_new_mail_ast() -> ast.AsyncFunctionDef: + tree = ast.parse(HOOKS.read_text(encoding="utf-8")) + for node in ast.walk(tree): + if (isinstance(node, ast.AsyncFunctionDef) + and node.name == "process_new_mail"): + return node + raise AssertionError("process_new_mail is gone from scheduler_hooks.py") + + +def _calls_named(node: ast.AST, name: str) -> bool: + return any( + isinstance(child, ast.Call) + and isinstance(child.func, ast.Name) + and child.func.id == name + for child in ast.walk(node) + ) + + +def test_dw2_the_gate_is_lexically_outside_the_step() -> None: + """Structural, because the runtime test above cannot see a refactor. + + The call to ``create_leads_from_new_mail`` must sit inside an ``if`` whose + test calls ``auto_lead_enabled``. Moving the flag check into the step — + the exact short-circuit done-when 2 names — leaves the runtime sentinel + unhappy and this assertion is what says WHY. + """ + function = _process_new_mail_ast() + guarded = [ + branch for branch in ast.walk(function) + if isinstance(branch, ast.If) + and _calls_named(branch.test, "auto_lead_enabled") + and _calls_named(branch, "create_leads_from_new_mail") + ] + assert guarded, ( + "process_new_mail no longer calls create_leads_from_new_mail from " + "inside `if auto_lead_enabled():` — with the flag off the step is " + "entered, which is the short-circuit done-when 2 forbids" + ) + # …and nowhere else: one guarded call site, not a guarded one plus a + # forgotten unguarded one. + total = sum( + 1 for node in ast.walk(function) + if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) + and node.func.id == "create_leads_from_new_mail" + ) + assert total == 1 + + +def test_dw2_a_crm_failure_never_breaks_mail_sync( + on: None, quiet_pipeline: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + """The shape every sibling step in ``process_new_mail`` has (`:81-112`): + the CRM is the newest and least important thing on this path.""" + source = inspect.getsource(scheduler_hooks.process_new_mail) + assert "sync.auto_lead_failed" in source, ( + "the CRM step's failure log key must follow the sync.*_failed " + "convention its five siblings use" + ) + + +async def test_dw2_a_raising_step_is_swallowed_and_logged( + on: None, quiet_pipeline: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + async def _boom(account_id: str) -> dict[str, int]: + raise RuntimeError("crm exploded") + + monkeypatch.setattr(auto_lead, "create_leads_from_new_mail", _boom) + + # No exception escapes: mail sync must complete even when the CRM does not. + await scheduler_hooks.process_new_mail(ACCOUNT_ID) + + +def test_dw2_the_crm_step_runs_after_every_mail_step() -> None: + """Order is a decision: the step considers what is STILL in the inbox once + the account's own automation has finished, so mail the user's own rules + archived never becomes a lead.""" + source = inspect.getsource(scheduler_hooks.process_new_mail) + positions = [ + source.index(step) for step in ( + "auto_run_rules_for_account(", "sweep_uncategorized(", + "_categorize_senders_job(", "_maybe_classify_threads(", + "_maybe_auto_archive(", "create_leads_from_new_mail(", + ) + ] + assert positions == sorted(positions) + + +# ── done-when 3: the ON-state mint ────────────────────────────────────────── + +async def test_dw3_an_unknown_external_sender_becomes_exactly_one_lead( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db, received_at=_at(5)) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 1 + assert stats["created"] == 1 + rows = _leads(db) + assert len(rows) == 1 + assert rows[0]["email"] == STRANGER + assert rows[0]["source"] == "email" + assert rows[0]["owner_email"] == OWNER + assert rows[0]["lead_name"] == "Asha Menon" + assert rows[0]["status_id"] # `_resolve_status` ran — NOT NULL satisfied + + +async def test_dw3_the_display_name_is_stripped_before_it_is_split( + db: FakeCrmDB, on: None, +) -> None: + """The "Asha Asha" trap (§8 B-series), in its inbound form: a quoted, + padded display name split without stripping produces a first name of + ``'"Asha'`` and a surname of ``'Menon"'``.""" + _ready(db) + _message(db, name=' "Asha Menon" ') + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + row = _leads(db)[0] + assert row["first_name"] == "Asha" + assert row["last_name"] == "Menon" + assert row["lead_name"] == "Asha Menon" + + +async def test_dw3_a_one_word_display_name_is_not_doubled( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db, name="Asha") + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + row = _leads(db)[0] + assert row["first_name"] == "Asha" + assert row.get("last_name") is None + assert row["lead_name"] == "Asha" + + +async def test_dw3_a_display_name_that_is_an_address_falls_through( + db: FakeCrmDB, on: None, +) -> None: + """`compute_lead_name`'s chain ends at the email local part, which is why + a sender with no display name cannot 422 (`LEADS.required` is empty). A + lead called `noreply@vendor.com` is what building the name by hand gives + you.""" + _ready(db) + _message(db, address="noreply@vendor.com", name="noreply@vendor.com") + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + row = _leads(db)[0] + assert row.get("first_name") is None + assert row["lead_name"] == "noreply" + + +async def test_dw3_the_lead_goes_through_the_service_write_path( + db: FakeCrmDB, on: None, +) -> None: + """`records.create_record`, never raw SQL. Four things live only there — + `_resolve_status`, the `owner_email` default, `validate_source` and + `mark_dirty_on_insert` — and a hand-rolled INSERT loses all four while + only the last is visible in the row.""" + _ready(db) + _message(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + source = inspect.getsource(auto_lead) + assert "INSERT INTO crm_leads" not in source, ( + "the step writes crm_leads directly — that path has no _resolve_status," + " no owner default, no validate_source and no dirty marking" + ) + assert "create_record(" in source + row = _leads(db)[0] + assert row["status_id"] and row["owner_email"] and row["lead_name"] + + +async def test_dw3_the_first_activity_is_metadata_never_content( + db: FakeCrmDB, on: None, +) -> None: + """D-CRM-12 applied to what a machine writes: the lead row is org-visible + to every `feature:crm` holder and goes to Zoho; sender + subject is the + proportionate disclosure for a cold inquiry, the body is not.""" + _ready(db) + _message(db, subject="Quote for 40 printers", received_at=_at(5, 11)) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert len(_activities(db)) == 1 + activity = _activities(db)[0] + assert activity["type"] == "system" + assert activity["subject"] == "Quote for 40 printers" + assert activity["body"] is None + assert activity["lead_id"] == _leads(db)[0]["id"] + assert activity["created_by"] == OWNER + meta = _meta(activity) + assert meta["sender_address"] == STRANGER + assert meta["sender_name"] == "Asha Menon" + assert meta["thread_id"] == "t-1" + assert meta["received_at"].startswith("2026-08-05") + assert meta["message_id"] + + +async def test_dw3_no_body_or_snippet_is_ever_read( + db: FakeCrmDB, on: None, +) -> None: + """The privacy boundary is the PROJECTION: nothing downstream can leak a + body it was never handed. Asserted against every statement the cycle + issued, not only against the candidate query's source.""" + _ready(db) + _message(db, body_text="wire transfer details, account 1234") + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + for statement in db.statements: + assert "body_text" not in statement + assert "snippet" not in statement + stored = json.dumps(db.tables.get("crm_activities", []), default=str) + assert "wire transfer" not in stored + + +async def test_dw3_only_classified_inbox_mail_is_a_candidate( + db: FakeCrmDB, on: None, +) -> None: + """Four disqualifications, one per predicate — each one is mail that must + not mint a lead even though it arrived after activation.""" + _ready(db) + _message(db, address="unclassified@a.com", processed_at=None) + _message(db, address="heldback@b.com", held_back_at=_at(5)) + _message(db, address="sentmail@c.com", folder="SENT") + _message(db, address="archived@d.com", folder="ARCHIVE") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 0 + assert _leads(db) == [] + + +# ── done-when 4: idempotency and in-batch de-duplication ──────────────────── + +async def test_dw4_re_running_the_same_sync_creates_no_second_lead( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert len(_leads(db)) == 1 + assert len(_activities(db)) == 1 + + +async def test_dw4_the_watermark_means_the_second_run_reconsiders_nothing( + db: FakeCrmDB, on: None, +) -> None: + """The sharper half of idempotency, and the one that fails when the + watermark stops advancing: "one lead" would still hold — the third + unknown-sender step would find the lead the first run created — while the + step re-read, re-classified and re-probed the same mail forever.""" + _ready(db) + _message(db, processed_at=_at(5)) + + first = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + second = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert first["candidates"] == 1 + assert second["candidates"] == 0 + cursor = db.rows("crm_auto_lead_cursors")[0] + assert cursor["processed_watermark"] == _at(5) + assert cursor["activated_at"] == ACTIVATED # never advanced + + +async def test_dw4_one_sender_emailing_twice_in_a_batch_mints_one_lead( + db: FakeCrmDB, on: None, +) -> None: + """In-batch de-duplication, asserted so that deleting it goes red. + + "One lead" alone would not: `create_record` commits on another session and + the third unknown-sender step would find that lead. What only the dedup + buys is that the second message is never CONSIDERED — one probe per + address, not one per message — and in production, where those two sessions + are genuinely concurrent, that is the difference between one lead and two. + """ + _ready(db) + _message(db, processed_at=_at(5, 9), thread_id="t-1") + _message(db, processed_at=_at(5, 10), thread_id="t-2", + subject="Following up") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 2 + assert stats["created"] == 1 + assert stats["deduped_in_batch"] == 1 + assert len(_leads(db)) == 1 + probes = db.statements_touching("FROM crm_leads WHERE lower(email)") + assert len(probes) == 1, ( + "the second message from the same sender was probed again — in-batch " + "de-duplication is gone and only the fake's shared dict is hiding it" + ) + + +# ── done-when 5: the three nothing-cases ──────────────────────────────────── + +async def test_dw5_a_colleague_on_the_accounts_own_domain_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + """Gate 1 — ``sender_scope`` answers "internal" for the owner's own + domain. A lead row for your own CFO, pushed into the live Zoho tenant, is + the failure this pair of gates exists to prevent.""" + _ready(db) + _message(db, address="cfo@fracktal.in", name="Our CFO") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_internal"] == 1 + assert _leads(db) == [] + + +async def test_dw5_a_colleague_on_a_configured_org_domain_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + """Gate 2 — and the case that proves it is not a restatement of gate 1. + + The org domain was typed as an ADDRESS, which is what people do. + ``resolve_org_domains`` runs ``normalize_domain`` over it and gets + ``fracktalworks.com``; ``sender_scope``'s own extra-domain arm only + ``lstrip('@')``s and gets ``ops@fracktalworks.com``, which matches no + sender's domain — the exact defect ``runner.py`` documents. So this + colleague is EXTERNAL to gate 1 and internal to gate 2, and deleting the + internal-domain gate mints a lead for him. + """ + _seed_account(db, org_domains=["ops@fracktalworks.com"]) + _seed_status(db) + _seed_cursor(db) + _message(db, address="ishaan@fracktalworks.com", name="Ishaan Pilar") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_internal"] == 1 + assert _leads(db) == [] + + +async def test_dw5_the_mailbox_owner_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db, address=OWNER, name="Vijay") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_internal"] == 1 + assert _leads(db) == [] + + +async def test_dw5_an_already_known_contact_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + db.seed("crm_contacts", first_name="Asha", email=STRANGER) + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_known"] == 1 + assert _leads(db) == [] + + +async def test_dw5_an_existing_lead_with_that_address_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + db.seed("crm_leads", lead_name="Asha", email=STRANGER, source="manual") + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_known"] == 1 + assert len(_leads(db)) == 1 # the seeded one, unchanged + + +async def test_dw5_a_sender_we_have_emailed_before_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + """`_maybe_block_cold`'s second step, mirrored: someone we have written to + is a correspondent, not a cold inbound inquiry.""" + _ready(db) + db.seed("email_messages", account_id=ACCOUNT_ID, folder="SENT", + from_address={"email": OWNER}, + to_addresses=[{"email": STRANGER}], + rules_processed_at=None, rules_held_back_at=None) + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_known"] == 1 + assert _leads(db) == [] + + +async def test_dw5_mail_sent_to_someone_else_does_not_suppress_the_lead( + db: FakeCrmDB, on: None, +) -> None: + """The control for the case above — and the one the shared fake could not + see before this slice taught it ``@>``. Without a containment reader the + probe matches every Sent message and no lead is ever created, which reads + as a passing "no lead" test for entirely the wrong reason.""" + _ready(db) + db.seed("email_messages", account_id=ACCOUNT_ID, folder="SENT", + from_address={"email": OWNER}, + to_addresses=[{"email": "somebody.else@elsewhere.com"}], + rules_processed_at=None, rules_held_back_at=None) + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["created"] == 1 + + +async def test_dw5_a_sender_in_the_cold_memo_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + """`_maybe_block_cold`'s first step: this account has already decided + something about this address — flagged it cold, or whitelisted it. Either + way it is not a stranger who just wrote in.""" + _ready(db) + db.seed("email_cold_senders", account_id=ACCOUNT_ID, from_email=STRANGER, + status="AI_LABELED_COLD") + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_known"] == 1 + assert _leads(db) == [] + + +async def test_dw5_a_message_with_no_sender_address_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + """`sender_scope` fails SAFE to "external", which is the wrong direction + here. An address we could not parse has no domain, so the second gate + refuses it.""" + _ready(db) + _message(db, address="", name="Anonymous") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_unusable"] == 1 + assert _leads(db) == [] + + +# ── done-when 6: born dirty, and the activity that never leaves ───────────── + +async def test_dw6_each_created_lead_is_born_zoho_dirty( + db: FakeCrmDB, on: None, +) -> None: + """D-CRM-9, asserted rather than left implied: an auto-lead enters the + push queue exactly like a human's, which is precisely why the flip is an + owner gate. `mark_dirty_on_insert` lives inside `core.insert_row`, so this + also fails if the write path is swapped for raw SQL.""" + _ready(db) + _message(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert _leads(db)[0]["zoho_dirty"] is True + assert _leads(db)[0].get("zoho_id") is None + + +def test_dw6_the_first_activity_type_is_excluded_from_the_push_predicate( +) -> None: + """Asserted against ``push_activities``' OWN statement text, not against + the constant beside it: the predicate is spelled inline in the SQL, and a + test that only read ``PUSHABLE_ACTIVITY_TYPES`` would keep passing if the + statement drifted from it.""" + source = inspect.getsource(crm_sync.push_activities) + match = re.search(r"type\s+IN\s*\(([^)]*)\)", source, re.I) + assert match, "push_activities no longer filters on an activity type list" + pushed = {value.strip().strip("'\"") for value in match.group(1).split(",")} + assert auto_lead.ACTIVITY_TYPE not in pushed, ( + f"the auto-lead activity type {auto_lead.ACTIVITY_TYPE!r} is now in " + "the Zoho push predicate — the mail's subject and sender would leave " + "the native CRM for the live tenant" + ) + assert pushed == set(crm_sync.PUSHABLE_ACTIVITY_TYPES) + + +async def test_dw6_the_activity_carries_no_zoho_id_and_is_not_pushable( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + activity = _activities(db)[0] + assert activity["type"] not in crm_sync.PUSHABLE_ACTIVITY_TYPES + + +# ── done-when 7: a deep resync mints nothing ──────────────────────────────── + +async def test_dw7_a_deep_resync_of_a_year_old_backlog_mints_nothing( + db: FakeCrmDB, on: None, +) -> None: + """The failure this whole cursor exists to prevent. + + ``resync_account`` runs a ~1-year all-folder backfill and then fires + ``process_new_mail``; a first-ever sync of a newly connected mailbox is + deep by the same heuristic; and neither stamps ``rules_held_back_at``. So + the backlog below is INDISTINGUISHABLE from new mail on every predicate + except ``received_at > activated_at``. Delete that one and this seeds 30 + leads into a live Zoho push queue. + """ + _ready(db) + for index in range(30): + _message( + db, address=f"stranger{index}@elsewhere.com", + received_at=_at(1, year=2025) + timedelta(days=index), + # Classified NOW, by the resync — which is the trap: the + # incremental cursor alone says every one of these is new. + processed_at=_at(6, 12), + ) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 0 + assert stats["created"] == 0 + assert _leads(db) == [] + assert _activities(db) == [] + + +async def test_dw7_the_first_on_state_run_activates_and_mints_nothing( + db: FakeCrmDB, on: None, +) -> None: + """No cursor row yet — which is every mailbox on the day the flag flips. + ``activated_at`` and ``processed_watermark`` are stamped to the same + instant, so the activating cycle itself considers nothing.""" + _seed_account(db) + _seed_status(db) + _message(db, received_at=_at(5)) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + cursors = db.rows("crm_auto_lead_cursors") + assert len(cursors) == 1 + assert cursors[0]["activated_at"] == cursors[0]["processed_watermark"] + assert stats["candidates"] == 0 + assert _leads(db) == [] + + +async def test_dw7_activation_writes_the_row_once( + db: FakeCrmDB, on: None, +) -> None: + _seed_account(db) + _seed_status(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + stamped = db.rows("crm_auto_lead_cursors")[0]["activated_at"] + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert len(db.rows("crm_auto_lead_cursors")) == 1 + assert db.rows("crm_auto_lead_cursors")[0]["activated_at"] == stamped + + +async def test_dw7_mail_that_arrived_after_activation_still_mints( + db: FakeCrmDB, on: None, +) -> None: + """The control: the discriminator must not be "mint nothing, ever".""" + _ready(db) + _message(db, received_at=_at(2), processed_at=_at(2)) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["created"] == 1 + + +# ── The per-cycle cap ─────────────────────────────────────────────────────── + +async def test_the_cycle_is_capped_and_the_overflow_is_counted( + db: FakeCrmDB, on: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + """Silent truncation reads as "covered everything", so the remainder is a + number in the log line.""" + monkeypatch.setattr(auto_lead, "MAX_CANDIDATES_PER_CYCLE", 3) + _ready(db) + for index in range(7): + _message(db, address=f"stranger{index}@elsewhere.com", + received_at=_at(5) + timedelta(minutes=index), + processed_at=_at(5) + timedelta(minutes=index)) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 3 + assert stats["overflow"] == 4 + assert stats["created"] == 3 + + +async def test_the_cap_leaves_the_remainder_for_the_next_cycle( + db: FakeCrmDB, on: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + """The watermark advances over what was CONSIDERED, so the overflow is + deferred rather than dropped. Ordering by ``rules_processed_at`` is what + makes that true — ordering by ``received_at`` would skip the remainder.""" + monkeypatch.setattr(auto_lead, "MAX_CANDIDATES_PER_CYCLE", 3) + _ready(db) + for index in range(5): + _message(db, address=f"stranger{index}@elsewhere.com", + received_at=_at(5) + timedelta(minutes=index), + processed_at=_at(5) + timedelta(minutes=index)) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + second = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert second["candidates"] == 2 + assert len(_leads(db)) == 5 + + +async def test_a_quiet_cycle_costs_no_count_query( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db, received_at=_at(5)) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert db.statements_touching("SELECT COUNT(*) FROM email_messages") == [] + + +# ── Robustness ────────────────────────────────────────────────────────────── + +async def test_an_account_with_no_recorded_owner_is_skipped( + db: FakeCrmDB, on: None, +) -> None: + """Attributing a lead to 'anonymous' is worse than not creating it: the + `owner` filter has nothing to match and it is nobody's follow-up.""" + _seed_account(db, owner="") + _seed_status(db) + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 0 + assert _leads(db) == [] + # It returns before the cursor is even activated: an account it will never + # write for has nothing to remember. + assert db.rows("crm_auto_lead_cursors") == [] + + +async def test_a_missing_account_row_is_skipped( + db: FakeCrmDB, on: None, +) -> None: + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 0 + assert db.rows("crm_auto_lead_cursors") == [] + + +async def test_one_bad_message_does_not_lose_the_batch( + db: FakeCrmDB, on: None, +) -> None: + """The WS-26b lesson: Postgres aborts the TRANSACTION on a statement + error, so a bare per-record try/except loses every row after the bad one. + The savepoint is what makes "one bad message" true, and the error is + counted rather than swallowed.""" + _ready(db) + _message(db, address="first@elsewhere.com", processed_at=_at(5, 9)) + _message(db, address="second@elsewhere.com", processed_at=_at(5, 10)) + db.fail_on("INSERT INTO crm_activities", times=1) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["errors"] == 1 + assert stats["created"] == 1 + assert db.savepoints == 2 + assert db.savepoint_rollbacks == 1 + + +async def test_the_watermark_advances_over_messages_that_minted_nothing( + db: FakeCrmDB, on: None, +) -> None: + """Otherwise a colleague's message is re-read, re-probed and re-rejected + on every sync cycle for the life of the mailbox.""" + _ready(db) + _message(db, address="cfo@fracktal.in", processed_at=_at(5, 9)) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert db.rows("crm_auto_lead_cursors")[0]["processed_watermark"] == _at(5, 9) + + +async def test_the_cursor_is_read_and_written_per_account( + db: FakeCrmDB, on: None, +) -> None: + """One mailbox's activation must not silence another's: the cursor is + keyed on ``account_id`` and every statement binds it.""" + _ready(db) + _message(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + for statement in db.statements_touching("crm_auto_lead_cursors"): + assert ":account_id" in statement + + +def test_the_module_registers_no_routes() -> None: + """Like ``broker_handlers``: it serves no HTTP surface, so it is not + imported from ``routes/crm/__init__.py`` and defines no route.""" + source = inspect.getsource(auto_lead) + assert "@router." not in source + package = ( + REPO / "apps/services/gateway/gateway/routes/crm/__init__.py" + ).read_text(encoding="utf-8") + assert "auto_lead" not in package + + +# ── The migration, read as text ───────────────────────────────────────────── + +def _cursor_migration() -> Path: + """Found by CONTENT, never by number — R1 forbids writing an absolute + future migration number anywhere, and a test pinned to a number is the + same mistake in a different file.""" + found = [ + path for path in sorted(MIGRATIONS.glob("*.sql")) + if path.name != "schema.generated.sql" + and "CREATE TABLE IF NOT EXISTS crm_auto_lead_cursors" in path.read_text( + encoding="utf-8", + ) + ] + assert len(found) == 1, ( + f"expected exactly one migration creating crm_auto_lead_cursors, " + f"found {[p.name for p in found]}" + ) + return found[0] + + +@pytest.fixture(scope="module") +def migration() -> str: + return _cursor_migration().read_text(encoding="utf-8") + + +@pytest.fixture(scope="module") +def bare(migration: str) -> str: + """Statements only — a check its own explanatory comment can satisfy is a + check that fails open.""" + return "\n".join(re.sub(r"--.*$", "", line) for line in migration.splitlines()) + + +def test_the_migration_takes_the_next_free_number() -> None: + numbers = sorted( + int(path.name.split("_", 1)[0]) + for path in MIGRATIONS.glob("*.sql") + if path.name.split("_", 1)[0].isdigit() + ) + mine = int(_cursor_migration().name.split("_", 1)[0]) + assert numbers.count(mine) == 1, ( + f"two migrations share number {mine} — the ladder replays them in " + "filename order, so one runs against the wrong schema" + ) + assert mine - 1 in numbers, "the ladder has a gap immediately below it" + + +def test_the_migration_header_says_what_why_and_what_it_depends_on() -> None: + lines = _cursor_migration().read_text(encoding="utf-8").splitlines() + header = "\n".join( + line for line in lines[: next( + index for index, line in enumerate(lines) + if line.strip() and not line.lstrip().startswith("--") + )] + ) + for required in ("What:", "Why:", "Depends on:"): + assert required in header, f"the migration header has no '{required}'" + + +def test_the_migration_is_idempotent(bare: str) -> None: + unguarded = [ + match.group(0) + for match in re.finditer(r"CREATE\s+(TABLE|INDEX)\s+(\S+)", bare, re.I) + if not match.group(2).upper().startswith("IF") + ] + assert not unguarded, ( + f"unguarded CREATE: {unguarded}. apply_migrations.sh replays the whole " + "ladder on every deploy" + ) + + +def test_the_migration_drops_or_truncates_nothing(bare: str) -> None: + assert not re.search(r"\b(DROP|TRUNCATE|DELETE\s+FROM)\b", bare, re.I) + + +def test_the_cursor_carries_both_timestamps_not_null(bare: str) -> None: + for column in ("activated_at", "processed_watermark"): + assert re.search(rf"{column}\s+TIMESTAMPTZ\s+NOT NULL", bare), ( + f"{column} must be NOT NULL — a NULL cursor is a predicate that " + "matches nothing, which reads exactly like a working feature" + ) + + +def test_the_cursor_is_keyed_and_cascaded_on_the_account(bare: str) -> None: + assert re.search( + r"account_id\s+UUID\s+PRIMARY KEY\s+REFERENCES\s+email_accounts\s*\(\s*id\s*\)" + r"\s+ON DELETE CASCADE", + bare, + ), "the cursor must be one row per account, cascading with the mailbox" + + +def test_the_migration_adds_no_unique_index_on_a_lead_address( + bare: str, +) -> None: + """The recorded refusal (spec §9): the cross-invocation double-mint race is + ACCEPTED. A UNIQUE index on `crm_leads.email` — where 1,516 imported rows + may already carry duplicates — is a deploy-blocking constraint of exactly + the shape migration 148 had to defuse.""" + assert not re.search(r"UNIQUE.*crm_leads", bare, re.I | re.S) + assert "crm_leads" not in bare From a64c5175a0870291a5cadeb7f0ee4ea73dc57724 Mon Sep 17 00:00:00 2001 From: Vijay Raghav Varada Date: Sat, 8 Aug 2026 04:06:37 +0530 Subject: [PATCH 2/4] fix(WS-26d-autolead): two ways the cursor lost leads, and the migration number somebody else had claimed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Diff-review repair round. Both P1s needed a cursor that had actually been running to see, which is why the first cut passed its own seven mutants. P1-1 — activated_at guarded the wrong thing. It stops a deep resync, and it does nothing at all about a flag that was on, turned off for four weeks, and turned back on: the cursor still carried day-1's anchor, so the first ON cycle minted the entire OFF window in one batch. The reviewer measured 27 of 27, each born zoho_dirty and each pushing unattended into the live tenant. The anchor now means the CURRENT ON epoch: a gap in the step's own runs beyond REANCHOR_GAP_SECONDS (3600, six scheduler periods) re-stamps the cursor, mints nothing from the gap, and says so at WARNING with gap_seconds. Fail-closed both ways — an OFF window and a real outage each skip their backlog, because a missed lead is hand-creatable and visible in the mailbox and 27 unattended tenant pushes are neither. ⚠️ DEVIATION, with its reason: dormancy is measured on a THIRD column, last_run_at, not on processed_watermark as ruled. The watermark tracks MAIL, not runs, and the literal predicate has two production failures. A mailbox that is merely quiet over a weekend carries a 60-hour-old watermark while the step has run faithfully every 600s — it would be re-anchored, and Monday's first message, the one this whole feature exists to catch, falls before the new anchor and mints nothing. Every Monday. And a poison head message holds the watermark still ON PURPOSE under P1-2 below; a watermark-based test would re-anchor past it after an hour and quietly undo the stall that was supposed to stay visible. last_run_at is stamped on every cycle including empty and stalled ones, so "quiet" and "not running" stay different facts. The ruling's constant, log key and behaviour on a genuine OFF window are unchanged. Both named tests are in the file, including the control that a quiet-but-running mailbox is never re-anchored. P1-2 — a failure advanced the cursor over the work it lost. This step opens a SECOND session per lead through create_record while holding the batch's own, so pool exhaustion fails many candidates at once; the old code advanced past all of them. Measured: 3 candidates, 3 errors, watermark advanced, three leads gone for good. The watermark now advances over the contiguous PREFIX that actually wrote its leads and stops at the first that raised. Later successes in the same batch are simply re-considered next cycle, which is free — the third unknown-sender step finds the lead they already created. A held cursor logs sync.auto_lead_stalled at WARNING EVERY cycle, because a held cursor and a quiet mailbox both create nothing and only the level can tell them apart; that trades silent loss for a visible stall on a genuinely poison head message, deliberately, fail-closed toward the CRM. The counter-case matters as much: a failed first ACTIVITY does NOT hold the cursor. The lead is already committed, so the message would be skipped on re-consideration and the activity never retried — holding for it would stall forever on work that cannot be redone. It is counted separately as activity_errors, and `created` now increments the moment the lead commits rather than after its activity: the row exists and will push either way, and a cycle logging created=0 beside a queued lead sends whoever reads it looking in the wrong place. P2-4 — the Sent probe inherited @>'s case-exactness. The owner wrote to Asha@AcmeRobotics.com; she replied from asha@acmerobotics.com and was minted as a cold lead. This module now asks with EXISTS over jsonb_array_elements and lower(). _maybe_block_cold is left alone on purpose: it is the email package's predicate with its own blast radius. The fake learned to READ the new shape — and the hard part was stripping it, since its inner lower(recipient->>'email') = :address is exactly what _JSONB_LOWER_CMP matches, which would have filtered the outer rows on a column they do not have. P2-6 — cfo@mail.fracktal.in passed both colleague gates. Gate 2 is now suffix aware, anchored on a leading dot so notfracktal.in is still a prospect. P2-5 — the cap had no tiebreak. ORDER BY rules_processed_at, id; fetch cap+1; and when the cap falls INSIDE a group sharing one timestamp, defer the whole group. Only then — deferring the last group unconditionally would shrink every capped batch by one message for nothing. Latent in production, where the rules runner stamps one transaction per message. Also: lead_name and the activity subject are clipped (120/500, with a marker). Nothing upstream bounds a display name, and it becomes a column every list, board card and Zoho push carries. MIGRATION RENUMBERED 157 → 158. Open PR #399 claims 157, and two migrations sharing a number replay in filename order against the wrong schema. The ladder carries a deliberate reservation gap until #399 lands; the header names it, the test finds the file by content, and the contiguity assertion is replaced by the uniqueness one — which is the property that actually protects the ladder and the only one this branch can hold. The step's import moved inside the gated branch so the OFF state does not load routes/crm on the mail path. The predicate stays above the gate on purpose: auto_lead_enabled is the flag's ONE definition, and reading settings.crm_auto_lead in the hook would make two places responsible for agreeing what the flag means. The import-inside-try divergence from the five sibling steps is now recorded rather than left to be rediscovered. 52 → 73 tests; 7 → 13 mutants red and reverted, one of them precision-checked (reverting the Sent probe must not redden the lower-case already-emailed case, or the mutant broke it rather than narrowing it). _crm_fakes gained fail_on(after=N), because where in a batch a failure lands IS the property under test. Still BUILT, NOT FLIPPED, NOT DEPLOYED. R4 in the same change: the ticket's cursor paragraph rewritten to the epoch-anchor semantics, done-when 8 and 9 added, the as-built block's deviation list grown to seven, plus work_plan's WS-26 row and §6(b) — which now tells an owner that turning the flag off is a stop, not a pause that accumulates. Co-Authored-By: Claude Fable 5 --- ai-company-brain/specs/crm_app.md | 198 +++++-- ai-company-brain/work_plan.md | 28 +- apps/services/gateway/AGENTS.md | 2 +- .../gateway/gateway/routes/crm/auto_lead.py | 473 ++++++++++++---- .../gateway/routes/email/scheduler_hooks.py | 28 +- infra/AGENTS.md | 2 +- infra/postgres/157_crm_auto_lead_cursor.sql | 82 --- infra/postgres/158_crm_auto_lead_cursor.sql | 105 ++++ tests/unit/_crm_fakes.py | 53 +- tests/unit/test_crm_auto_lead.py | 518 +++++++++++++++++- 10 files changed, 1223 insertions(+), 266 deletions(-) delete mode 100644 infra/postgres/157_crm_auto_lead_cursor.sql create mode 100644 infra/postgres/158_crm_auto_lead_cursor.sql diff --git a/ai-company-brain/specs/crm_app.md b/ai-company-brain/specs/crm_app.md index 0cdd36076..498f1a065 100644 --- a/ai-company-brain/specs/crm_app.md +++ b/ai-company-brain/specs/crm_app.md @@ -55,13 +55,18 @@ > `acb_common/settings.py` and ships **False**; the CRM step is > `routes/crm/auto_lead.py`, called from `process_new_mail` from **inside > `if auto_lead_enabled():`** so the OFF state enters nothing; the new -> `crm_auto_lead_cursors` table (migration **157**) carries the -> `activated_at` / `processed_watermark` pair the deep-resync discriminator -> needs. **Nothing has been flipped and nothing has been deployed** — the -> flip stays OWNER-GATE (`work_plan.md` §6 (b)), and while the flag is off +> `crm_auto_lead_cursors` table (migration **158** — 157 is held by open PR +> #399) carries the `activated_at` / `processed_watermark` / `last_run_at` +> trio the deep-resync discriminator, the incremental cursor and the dormancy +> re-anchor need. **Nothing has been flipped and nothing has been deployed** — +> the flip stays OWNER-GATE (`work_plan.md` §6 (b)), and while the flag is off > this changes no runtime behaviour at all. Tests: -> `tests/unit/test_crm_auto_lead.py` (52 cases); seven mutants run red and -> reverted. +> `tests/unit/test_crm_auto_lead.py` (73 cases); thirteen mutants run red and +> reverted. ⚠️ **One diff-review round landed on this branch** and closed two +> P1s that only a running cursor would have shown: an OFF→ON round trip minted +> the whole OFF window (27 leads measured), and a single failure advanced the +> cursor over every candidate behind it (3 leads lost, measured). See the +> ticket's done-when 8 and 9. > · **WS-26e: 🟡 SPEC, nothing built.** > **26f** — 🟢 **MERGED + DEPLOYED 2026-08-07 (PR #391), NOT RUN against the tenant.** f1 > `POST /crm/import/zoho/stages` (`routes/crm/stage_metadata.py`, floor @@ -969,7 +974,7 @@ unknown sender becomes a lead on its own. | D2 | **WS-26d-email** | The "this is not a toy" moment. Disjoint files from D1 (`activities.py`/`Timeline.tsx` vs. importer/admin/settings). | ∥ with D1 | | D3 | **WS-26g** — ✅ **BUILT 2026-08-07** (branch `ws-26g-reports`, no migration) | The forecast number. **After D1** — f2 and the reports tab both extend the `page.tsx`/`urlState.ts` tab grammar, and two parallel PRs there is a needless conflict. | after D1 | | D4 | **WS-26d-write** ✅ **BUILT 2026-08-08** | The AI-creates-a-lead demo beat. Lives in `apps/agents/agent-crm/` — collides with nothing above. No migration. | ∥ with any | -| D5 | **WS-26d-autolead** — ✅ **BUILT 2026-08-08, flag OFF, NOT flipped, NOT deployed** (migration 157) | Built whenever; the **flip is OWNER-GATE** and pushes real leads into Zoho (D-CRM-9) — demo it only if the owner wants that story told live. | ∥ with any | +| D5 | **WS-26d-autolead** — ✅ **BUILT 2026-08-08, flag OFF, NOT flipped, NOT deployed** (migration 158) | Built whenever; the **flip is OWNER-GATE** and pushes real leads into Zoho (D-CRM-9) — demo it only if the owner wants that story told live. | ∥ with any | **Deferred until after the demo, deliberately — not demoted:** WS-26h (discipline), WS-26i (data management), WS-26e (cutover). No demo viewer sees them; they lose nothing @@ -1419,8 +1424,8 @@ Frontend: extend the existing CRM vitest for the third `kind`. ### WS-26d-autolead — `CRM_AUTO_LEAD` · ✅ **BUILT 2026-08-08** · 🔴 **OWNER-GATE to flip — NOT FLIPPED, NOT DEPLOYED** *(Closes B4.)* -> **As built** (branch `ws-26d-autolead`, migration **157** -> `157_crm_auto_lead_cursor.sql` — the number taken from the directory at +> **As built** (branch `ws-26d-autolead`, migration **158** +> `158_crm_auto_lead_cursor.sql` — the number taken from the directory at > build time per R1, and `test_crm_auto_lead.py` finds the file by CONTENT, > so a renumber in review breaks nothing). **The flag is `False` everywhere and nothing has been > deployed**: with `CRM_AUTO_LEAD` off this branch changes no runtime @@ -1440,20 +1445,37 @@ Frontend: extend the existing CRM vitest for the third `kind`. > another package's *private* helper, and a third copy of "is this person a > colleague?" is the drift that rule exists to prevent. > * The call site in `routes/email/scheduler_hooks.py::process_new_mail` is -> `if auto_lead_enabled(): await create_leads_from_new_mail(account_id)` -> inside the sibling `try/except` shape, logging `sync.auto_lead_failed`. -> **It runs LAST, after auto-archive**, so the step considers what is still -> in the INBOX once the account's own automation has finished — mail the -> user's own rules archived never becomes a lead. -> * `tests/unit/test_crm_auto_lead.py` — **52 cases**, each done-when named in -> a test. `tests/unit/_crm_fakes.py` gained ONE reader (`@>` jsonb -> containment) because without it the "have we ever emailed them" probe was -> invisible to the fake, which answered "yes" for every Sent message and -> would have made the already-known-contact case a test of nothing. -> * **Seven mutants run red and were reverted**: the flag check, the -> `received_at > activated_at` predicate, the watermark advance, the -> internal-domain second gate, `type='system'`, the service write path, and -> the in-batch dedup. +> `if auto_lead_enabled():` with the step's own import INSIDE the branch, so +> the OFF state does not load `routes/crm` on the mail path at all. The +> predicate stays above the gate on purpose: `auto_lead_enabled` is the +> flag's ONE definition, and reading `settings.crm_auto_lead` in the hook +> instead would make two places responsible for agreeing what the flag means +> (the `sync_enabled` precedent). **It runs LAST, after auto-archive**, so +> the step considers what is still in the INBOX once the account's own +> automation has finished — mail the user's own rules archived never becomes +> a lead. ⚠️ **One divergence from the five sibling steps, recorded:** the +> import sits INSIDE the `try`, so a `routes/crm` module that fails to import +> is logged on `sync.auto_lead_failed` like any other CRM failure rather than +> raised out of the mail path. +> * `tests/unit/test_crm_auto_lead.py` — **73 cases**, each done-when named in +> a test. `tests/unit/_crm_fakes.py` gained two readers and one capability: +> `@>` jsonb containment and the case-folding +> `EXISTS (… jsonb_array_elements …)` form (each needed because a probe the +> fake cannot see is a probe it answers "yes" to for every Sent message — +> and the second must be STRIPPED before the existing `_JSONB_LOWER_CMP` +> reader misreads its inner comparison), plus `fail_on(..., after=N)`, +> because *where* in a batch a failure lands is the whole property under +> test in done-when 9. +> * **Thirteen mutants run red and were reverted** (seven pre-review, six more +> for the repair round): the flag check · the `received_at > activated_at` +> predicate · the watermark advance · the internal-domain second gate · +> `type='system'` · the service write path · in-batch dedup · the dormancy +> re-anchor · the prefix-only advance · the stall WARNING (demoted to INFO) · +> the case-folding Sent probe · suffix-aware internal domains · the +> `last_run_at` stamp on a quiet cycle. The Sent-probe mutant is additionally +> checked for PRECISION: reverting it must NOT redden the lower-case +> already-emailed case, or the mutant broke the probe rather than narrowing +> it. > > **Five decisions the ticket did not record, each with its reason:** > 1. **`activated_at` and `processed_watermark` are stamped to the SAME @@ -1469,12 +1491,13 @@ Frontend: extend the existing CRM vitest for the third `kind`. > what makes gate 2 load-bearing rather than a restatement of gate 1, and a > named test (a colleague on an org domain somebody typed as an address) > goes red when it is deleted. -> 3. **The watermark advances over messages that minted nothing, and over -> messages that raised.** This is a best-effort enrichment step, not a -> queue: a poison message holding the cursor would re-fail on every cycle -> for the life of the mailbox. Errors are COUNTED in the log line instead, -> and each candidate is wrapped in `core.savepoint` so one statement error -> cannot abort the batch's transaction (the WS-26b lesson). +> 3. **The watermark advances over the successful PREFIX** — over messages +> that minted nothing, never past one whose lead write raised (done-when 9, +> rewritten 2026-08-08 after the diff review measured the first version +> losing three leads permanently on a single pool exhaustion). A held cursor +> is reported at WARNING every cycle rather than left to look like a quiet +> mailbox. Each candidate is wrapped in `core.savepoint` so one statement +> error cannot abort the batch's transaction (the WS-26b lesson). > 4. **The lead and its first activity are two transactions.** > `create_record` opens and commits its own session — it is the same > function `POST /crm/leads` calls — so a failure between the two leaves a @@ -1485,11 +1508,40 @@ Frontend: extend the existing CRM vitest for the third `kind`. > even activated.** `actor()` would attribute the lead to `"anonymous"`, > and a lead that is nobody's follow-up and that the `owner` filter cannot > match is worse than no lead. +> 6. ⚠️ **Dormancy is measured on a THIRD column, `last_run_at`, not on +> `processed_watermark`** — a deliberate departure from the shape the P1-1 +> ruling prescribed, because the watermark tracks MAIL rather than runs and +> the literal version has two production failures. (a) A mailbox that is +> merely quiet over a weekend carries a 60-hour-old watermark while the step +> has run every 600s, so it would be re-anchored and **Monday's first +> message — the one this feature exists to catch — would fall before the new +> anchor and mint nothing. Every Monday.** (b) A poison head message holds +> the watermark still ON PURPOSE (decision 3); a watermark-based test would +> re-anchor past it after an hour and silently undo the stall that was +> supposed to stay visible. `last_run_at` is stamped on every cycle +> including empty and stalled ones, so "quiet" and "not running" stay +> different facts. The ruling's constant (3600s), log key +> (`sync.auto_lead_reanchored`) and behaviour on a real OFF window or outage +> are unchanged. +> 7. **Both attacker-controlled strings are clipped** (`MAX_NAME_CHARS` 120, +> `MAX_SUBJECT_CHARS` 500, with a `…` marker). Nothing upstream bounds +> either: a display name is whatever the sending server put in the header, +> and it becomes `lead_name` — a column every CRM list, board card and Zoho +> push then carries. > -> **What an owner still has to do, in order:** merge → deploy (migration 157 +> **What an owner still has to do, in order:** merge → deploy (migration 158 > applies automatically) → flip `CRM_AUTO_LEAD` (§6 (b)). The first ON-state > run on each mailbox activates the cursor and mints nothing; leads start -> appearing from mail that arrives after that moment. +> appearing from mail that arrives after that moment. The same is true after +> any period with the flag off or the service down for more than an hour: the +> next cycle re-anchors, mints nothing from the gap, and says so at WARNING. +> +> ⚠️ **The migration is numbered 158, not 157.** Open PR #399 +> (`157_projects_recurrence.sql`) holds 157, and two migrations sharing a +> number replay in filename order against the wrong schema. The ladder +> therefore carries a deliberate reservation gap at 157 until that PR lands; +> the migration header names it, and the test finds the file by CONTENT so a +> further renumber in review costs nothing. **The hook is `process_new_mail(account_id)` — `routes/email/scheduler_hooks.py:57`.** It is the shared new-mail pipeline (rules → sweep → categorize senders → classify threads @@ -1517,17 +1569,57 @@ does not go through this hook). A candidate query of "everything classified" wou therefore mint a lead per unknown external sender in a year of mail the moment a second mailbox connects — each born `zoho_dirty`, each pushed to the live tenant within one 600s cycle, with no confirmation card anywhere on a scheduler hook and no delete tool. -The step therefore keeps a **per-account two-timestamp cursor** in a new table +The step therefore keeps a **per-account three-timestamp cursor** in a new table (migration at the next free number at build time, R1): -- `activated_at` — set ONCE, to the clock time of the step's first ON-state run for - that account, never advanced. **The backfill discriminator is - `received_at > activated_at`**: mail received before auto-lead was first active on - the account is history and mints nothing, no matter when a resync classifies it. +- `activated_at` — **the start of the current ON epoch**, not "the first time anyone + ever enabled this". **The backfill discriminator is `received_at > activated_at`**: + mail that ARRIVED before this epoch began is history and mints nothing, no matter + when a resync classifies it. - `processed_watermark` — the incremental cursor: candidates are classified inbox mail (`rules_processed_at IS NOT NULL`, `rules_held_back_at IS NULL`) with - `rules_processed_at > processed_watermark`, advanced only after the batch commits. -Both predicates apply together. Per-cycle candidate cap (a named constant, ~200) with -the overflow COUNTED in the log line — silent truncation reads as "covered everything". + `rules_processed_at > processed_watermark`. +- `last_run_at` — when the step last RAN, stamped on **every** cycle including the ones + that considered nothing. This is the dormancy clock, and it is a third column rather + than a reading of the second because the watermark tracks MAIL: a mailbox that is + merely quiet over a weekend has a 60-hour-old watermark while the step has run + faithfully every cycle. + +**Re-anchoring, and why "set ONCE" was wrong (2026-08-08 diff review, P1-1).** +`activated_at` alone guards a deep resync and does nothing about a flag that was on, +turned OFF for four weeks, and turned back on: the cursor still carries the old anchor, +so the first ON cycle mints the whole OFF window in one batch — **measured at 27 leads +for a 27-day window**, each pushing unattended into the live tenant. So when an +ON-state cycle finds `now - last_run_at > REANCHOR_GAP_SECONDS` (a named constant, +3600 — six scheduler periods), it **re-stamps all three timestamps to now**, mints +nothing from the gap, and logs `sync.auto_lead_reanchored` with `gap_seconds`. +Fail-closed in both directions — an OFF window and a real outage each skip their +backlog. A missed lead is hand-creatable and visible in the mailbox; 27 unattended +pushes into the live tenant are neither. + +**A failure never advances the cursor past lost work (2026-08-08 diff review, P1-2).** +The watermark advances to the max `rules_processed_at` of the **contiguous prefix of +the batch that successfully wrote its leads**; on the first lead-write failure it stops +moving, and later successes in the same batch are simply re-considered next cycle (free +— the third unknown-sender step finds the lead they already created). This matters +because the step opens a SECOND session per lead through `create_record` while holding +the batch's own, so pool exhaustion fails many candidates at once and an unconditional +advance steps over every one of them permanently — **measured at 3 candidates, 3 +errors, watermark advanced, three leads lost for good.** When the watermark does not +move and errors > 0 the cycle logs `sync.auto_lead_stalled` at **WARNING, every cycle**: +a held cursor and a quiet mailbox both create nothing, so the counters cannot tell them +apart and only the level makes it visible. The accepted cost is a visible stall on a +genuinely poison head message, deliberately — fail closed toward the CRM. ⚠️ A failure +to write the first **activity** does NOT hold the cursor: the lead is already committed, +so the message would be skipped on re-consideration and the activity never retried. + +Per-cycle candidate cap (a named constant, ~200) with the overflow COUNTED in the log +line — silent truncation reads as "covered everything". The fetch takes cap+1 so +"there is more" is a fact rather than an inference, and if the cap falls INSIDE a group +of rows sharing one `rules_processed_at` the whole group is deferred to the next cycle +(the watermark is a timestamp, so advancing it into a cut group loses the rest of it); +`ORDER BY rules_processed_at, id` gives that group a stable order. Latent only — +production stamps one transaction per message, so ties do not occur today. + Residual race accepted and recorded: two concurrent `process_new_mail` invocations for one account can read the same watermark and double-mint; the cost is one visible, hand-deletable duplicate lead, and the alternative (a unique index minted on a column @@ -1545,6 +1637,15 @@ shape is the SELECT guard above plus **in-batch de-duplication** (one sender ema twice in a single batch mints one lead), and the cross-invocation race is accepted per the cursor paragraph. +⚠️ **The Sent probe folds case HERE, unlike `_maybe_block_cold` (2026-08-08 diff +review, P2-4).** Postgres's `@>` is case-EXACT, so an owner who wrote to +`Asha@AcmeRobotics.com` has, as far as containment is concerned, never emailed +`asha@acmerobotics.com` — and she replies in lower case, so her reply minted a lead for +somebody already mid-conversation. This step asks the question with +`EXISTS (SELECT 1 FROM jsonb_array_elements(to_addresses) … WHERE lower(…) = :addr)` +instead. `_maybe_block_cold` is deliberately **left alone**: it is the email package's +predicate and its blast radius is the cold-email blocker, not this. + **The first activity is metadata, never content (audit blocker G2).** The originating message is logged with **`type='system'`** — deliberately outside the Zoho push predicate (`sync_zoho.py` pushes `type IN ('note','task')` only), so the activity never @@ -1563,7 +1664,12 @@ colleague?" across the automation package, and it fails SAFE to `"external"` — the wrong direction here, so the CRM step must treat `"external"` as *necessary but not sufficient* and still apply the internal-domain list (`cleanup.py:298-303`). A lead row for your own CFO, pushed into the live Zoho tenant (D-CRM-9), is the failure this -paragraph exists to prevent. +paragraph exists to prevent. ⚠️ **That list matches SUBDOMAINS too** (2026-08-08 diff +review, P2-6): `cfo@mail.fracktal.in` is the CFO, and a company's own +`mail.`/`corp.`/regional subdomains are routine — exact matching alone let him through +while `cfo@fracktal.in` was caught. The suffix test is anchored on a leading dot, so +`notfracktal.in` is not a subdomain of `fracktal.in`; a bare `endswith` would refuse a +real prospect's leads, which is the same damage in the other direction. **Done-when:** 1. `crm_auto_lead: bool = False` in `acb_common/settings.py`, shipping OFF. @@ -1588,6 +1694,18 @@ paragraph exists to prevent. 7. **A deep resync / first sync of a newly-connected mailbox mints NOTHING**: the hook run against an account whose messages all predate `activated_at` creates zero leads — the test seeds a year-old classified backlog and runs the ON-state hook against it. +8. **An OFF→ON round trip mints nothing from the OFF window** (added 2026-08-08, P1-1): + the hook run against an account whose cursor was anchored weeks ago and whose + `last_run_at` is stale creates zero leads and re-anchors the cursor — seed a stale + cursor plus a backlog. Its control is named too: a mailbox that is merely QUIET but + still running is never re-anchored, because re-anchoring it would drop the first + message to arrive after the quiet spell — the one this feature exists to catch. +9. **A failure never advances the cursor past lost work** (added 2026-08-08, P1-2): + a batch of `[ok, raise, ok]` leaves the watermark at the FIRST message's stamp, the + next cycle re-considers messages 2 and 3, and a cycle whose watermark did not move + with errors > 0 logs `sync.auto_lead_stalled` at WARNING **on every cycle it + persists**. A failed first *activity* is the counter-case: it is counted separately + and does not hold the cursor. **Tests:** `tests/unit/test_crm_auto_lead.py` (B7). diff --git a/ai-company-brain/work_plan.md b/ai-company-brain/work_plan.md index e4980a0d7..18db7d39f 100644 --- a/ai-company-brain/work_plan.md +++ b/ai-company-brain/work_plan.md @@ -147,7 +147,7 @@ looks. Full statements live in `FOUNDATION_BUILDOUT_CHECKLIST.md`. | WS-20 | **WhatsApp activation + remainder** (search UI 🟢 AGENT-SAFE; OCR needs a vision-tier decision; Odoo/Zoho-bound items blocked) | `whatsapp_message_manager.md` §11 (header fixed 2026-08-01) | 🟡 owner | **OWNER-GATE:** Meta env/app review, enrichment cost flags. | | WS-21 | **Calendar F2/F3** (`gtd_time_blocks`, email windows, mobile timeline, external sync) | `calendar_focus_os.md` **§9** (canonical for all F2/F3 acceptance; **§5** canonical for `gtd_time_blocks`) + `calendar_timeboxing.md` **§13** (canonical for P4) — both rewritten 2026-08-03 | 🟡 partial | **Re-audited 2026-08-03 → GO-NARROWED.** P3 roll-over was already shipped (released-to-unscheduled, mig 78 + `start_auto_rollover`). ~~"ideal week"~~ **struck — substantially shipped** (mig 98 + settings round-trip + editor + grid render + packer honouring + 2 unit tests); only the unused-focus-window / template-adherence gap remains (§9.6). **Breaks-in-the-packer SHIPPED 2026-07-23** (`80722e17`, mig **97**) as *packer geometry* — a widened buffer plus lunch protection, **a gap, not a `kind='break'` row**, which is exactly why F2 survives (§5 residual 4, now closed). **The 2026-08-01 acceptance was satisfiable by doing nothing** — 2 of its 3 `gtd_time_blocks` clauses were already green against shipped code; they are deleted and replaced with four that all fail today. **`gtd_time_blocks` is 4 slices, not 1 PR** (§9.1 S1–S4): the "non-breaking `TimeBlock[]` swap" claim was **FALSE** — the measured blast radius is 17 TS files + 3 gateway modules + `apps/skills/skill-task-gtd/` + `apps/agents/agent-task-manager/`. **Focus Shield is AGENT-SAFE, not owner-gated** (§9.5) — it needs a design, not a credential; do not dispatch on §4.1 prose alone. **Top-5 outcomes (Horizons) — DO NOT DISPATCH:** it collides with WS-18; §4 assigns it here, and WS-18's title keeps it struck. **Verify by naming test files — never `pytest tests/unit -k calendar`**: `-k` still collects the whole directory, and whole-directory collection hangs on the Windows box. **Dispatchable today:** §9.1 S1 · the ritual-stamp localStorage residue (§9.1 done-when 4, independently shippable) · §9.6 · §9.7. **OWNER-GATE:** external sync (§9.11 / timeboxing §13 P4) needs Google Calendar and/or Microsoft Graph OAuth client credentials provisioned on the VPS. | | WS-22 | **draw.io** (all 13 tickets open, nothing built) | `drawio_integration.md` | 🟡 owner | Best acceptance structure in the corpus; needs an owner and re-verified anchors (~5 weeks stale). ST-DRW-02 is a decision gate. | -| **WS-26** | **CRM app — native CRM + Zoho retirement** *(minted 2026-08-05)* | `specs/crm_app.md` | ✅ **a + b + c BUILT + DEPLOYED** · ✅ **d read half BUILT + DEPLOYED** · ✅ **D2 = d-email BUILT 2026-08-07** (branch `ws-26d-email-timeline`, merged) · ✅ **D4 = d-write MERGED + DEPLOYED 2026-08-08 (PR #400, no migration; deploy 31217978773 log-verified)** · ✅ **D5 = d-autolead BUILT 2026-08-08 (branch `ws-26d-autolead`, migration 157; flag OFF, NOT flipped, NOT deployed)** · ✅ **D1 = f BUILT 2026-08-07 (branch `ws-26f-pipeline-truth`, NOT run against prod)** · ✅ **D3 = g BUILT 2026-08-07 (branch `ws-26g-reports`, no migration)** · 🟢 **DEMO CRITICAL PATH COMPLETE (owner-directed 2026-08-07, spec §9.0): ~~D1 f~~ (∥ D2 d-email) → ~~D3 g~~ → ~~D4 d-write~~ → ~~D5 d-autolead~~** · 🟡 **h/i/e deferred past the demo; i spec-thin** | Research pass 2026-08-05: `frappe/crm` (AGPL — **concepts only, no code**), `trycompai/crm` (MIT), full-tree Zoho sweep. **Zoho today is a read-only nightly mirror** into the Phase-0 graph tables (`person`/`customer`/`deal`) with no UI, no write path, and **no Leads pull** — so leaving Zoho is import-and-retire, not a live cutover. Spine: Frappe's lead→convert→deal+contact+organization with **statuses-as-data** (color/position/type/probability); trycompai's single activity-spine table + `source` provenance + `last_activity_at` discipline. **BO-10 contribution: WS-26a adds the shared engine seam (`gateway/db.py::get_engine()`, tasks converted as proof) instead of engine 13.** Tickets: **a** schema + feature registration + core API — **BUILT 2026-08-05** (mig `144_crm.sql`, `feature:crm`, `gateway/db.py` seam + tasks converted, `routes/crm/`; **migration 144 applied on prod and `/crm` live as of 2026-08-06**) · **b** **Zoho two-way sync — BUILT 2026-08-05** (branch `ws-26b-zoho-sync`: `list_leads` + `list_deleted` on the read client, the single write client `ingestion/sources/zoho/writer.py` with one grep-asserted caller, mig `145_crm_zoho_sync.sql` (dirty columns + `crm_zoho_tombstones` + `crm_sync_cursors`), `routes/crm/{import_zoho,sync_zoho,broker_handlers}.py`, `crm.zoho_*` broker handlers registered from `main.py`, 80 new hermetic tests). *(Re-scoped 2026-08-05, owner-directed D-CRM-7: "faithful two way sync until we do away with Zoho entirely" — coexistence is bidirectional, not import-once.)* **Measured 2026-08-06: mig 145 is applied on prod and the BACKFILL HAS RUN — 737 orgs / 1,189 contacts / 1,516 leads / 551 deals / 1,909 notes, zero dirty rows, zero unmatched owners; the §7.1 pre-flip curl confirmed the tenant honors RFC-1123 `If-Modified-Since` (304). The PUSH direction has still never run: `CRM_ZOHO_SYNC` ships OFF, nothing has ever written the live Zoho tenant, and enabling the flag or hand-running a push cycle against prod stays OWNER-GATE §6.** WS-1's "no Zoho write path anywhere" clause was corrected in the same change (done-when 6) · **c** UI + the API addendum — **BUILT 2026-08-05** on branch `ws-26c-crm-ui` atop 26a and **merged with b into `ws-26-crm-app` 2026-08-06** (`/crm` app + BFF proxy; the three frontend registration points with `CenterApp` re-typed so `live ⇒ href` is a compile error; `routes/crm/deal_contacts.py` with one-primary-per-deal enforced on the shared `core.link_deal_contact` seam the convert path now also uses — 26b's importer is the one excepted writer and computes `is_primary` in-statement so a backfill can never demote a hand-set primary; `organization_name` on the deal list + board via a derived-table LEFT JOIN; the three review residuals — `?status_id` on a pipeline-less entity → 422, explicit `null` on a defaulted NOT NULL column → 422 not a driver 500, and a hand-edited `lead_name` surviving a name-field PATCH. **Deployed:** migrations 144 and 145 are applied on prod as of 2026-08-06 and `/crm` is live, so live rendering, drag persistence and deep links are owner-verifiable now) · **d** integrations — **audited 2026-08-06 GO-NARROWED and the narrowed slice is BUILT** (branch `ws-26d-agent-crm`): `apps/agents/agent-crm/` (`crm-assistant`, MAF, four READ tools over the existing `/crm` routes carrying the caller's `X-User-Email`, read-only enforced at the transport by a GET-only method allowlist) registered in `_KNOWN_AGENTS` + `_AGENT_REGISTRY` + `agent_registry.json`, plus `"crm"` added to the WhatsApp `_KNOWN_SYSTEMS` allowlist **parse-only** (nothing writes `wa_contacts.entity_ref`, the `crm` context block stays `None`, both pinned by test). **The three held-back items are now DISPATCHABLE — their doc blockers (B3/B4/B5/B7) were closed 2026-08-06 in `crm_app.md` §9.1-§9.3, every anchor read off `origin/main` rather than recalled:** **WS-26d-email** (the timeline join is CALLER-scoped, never record-scoped — it reuses the email app's `_account_scope` predicate, copied into `routes/crm/` rather than imported per D-CRM-4, joins by thread not message, inbound `from_address` only, and needs a new address index at the next free migration number) · **WS-26d-autolead — BUILT 2026-08-08** (branch `ws-26d-autolead`, migration **157** `crm_auto_lead_cursors`). Hook = `routes/email/scheduler_hooks.py::process_new_mail`, the one seam scheduler+manual+webhook all funnel through; the per-message rules loop was considered and REJECTED because a classifier outage there double-fires and history backfills never reach it. ⚠️ **That same seam is ALSO reached by deep resyncs** (2026-08-08 audit blocker G1, closed by PR #402 before the build), which is why the step keeps a per-account TWO-timestamp cursor: `activated_at`, stamped once and never advanced, makes `received_at > activated_at` the backfill discriminator, and `processed_watermark` is the incremental one. Without the first, connecting a second mailbox mints a lead per unknown sender across a year of mail, each queued for the live tenant. Unknown-sender check mirrors `_maybe_block_cold`'s two steps and adds a third (no `crm_contacts`/`crm_leads` row with that `lower(email)`) — **the ticket's original `ON CONFLICT DO NOTHING` could not have fired** (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so dedup is that SELECT guard plus in-batch de-duplication, with the cross-invocation race accepted and the UNIQUE index refused. Colleague suppression is TWO gates: `sender_scope` (which fails SAFE to "external", the wrong direction here) and the normalised internal-domain list. The originating message is logged `type='system'` — outside `sync_zoho`'s `type IN ('note','task')` push predicate — with subject + sender in `meta` and **`body` empty**, so the mail's content never leaves the native CRM even though the lead does. The lead goes through `records.create_record`, never raw SQL. **Built, flag `False`, NOT flipped, NOT deployed; the flip stays §6 (b).** 52 hermetic cases; 7 mutants red and reverted · **WS-26d-write — BUILT 2026-08-08** (branch `ws-26d-write`, **no migration**: every route the four tools call already existed). `request_confirmation` awaited at the top of each tool before any mutating request is built, fail-closed, and the `non_interactive_default` keyword is asserted ABSENT from the whole module rather than asserted != "approve" — pinning the argument rather than the value means a mutant does not get to pick a spelling the fence has not heard of. `_ALLOWED_METHODS` **widened, never deleted**: `{GET, POST, PATCH}`, still checked inside `_request`, with `DELETE`/`PUT` and any `_delete`/`_put` helper still absent, so the check that used to enforce "read-only" now enforces "never destroys". Path fence extended past `ast.JoinedStr` to `.format`/`%`/`+` (the re-review's P2) and — the part that makes it maintainable — **tested against synthetic sources one per idiom**, so "the fence went blind" is a red test rather than a silent gap. Two supervisor rulings landed as built: `update_deal_status` resolves the stage BY NAME inside the tool against `GET /crm/statuses/deal` (no UUID on the LLM surface; an unknown name returns the real lane names), and a lost-type target requires a `lost_reason` resolved the same way against `GET /crm/lost-reasons` — pre-empting the 422 the "close this as lost" demo beat would otherwise hit — with the vocabulary **only ever read, never created**. `create_lead` takes **no `owner_email` argument at all** (the route derives it from the acting user), deleting an LLM-filled identity field from the surface entirely. ⚠️ **One recorded departure from done-when 1**: the invariant asserted is *no mutation before consent*, not *no HTTP before consent* — two tools must read to describe honestly what they are about to do, and every pre-card call being a GET is itself pinned; the two tools that owe nothing to a pre-read are still held to literally zero calls. `@_annotate_risk` is the shared annotation convention and is NOT enforcement; the Action Broker covers the Zoho push, not the native write — the two fail in OPPOSITE directions and are not interchangeable. 76 new hermetic cases + `test_crm_agent.py` 87 → 143; ten mutants run red and reverted. **Built, not deployed.** The push-queue question is CLOSED — **D-CRM-9 (owner, 2026-08-06): agent-originated writes queue for Zoho exactly like human ones** (🟡 remainder; the flip stays OWNER-GATE) · **e** cutover + retirement inventory + **Zoho refresh-token revoke, which executes part of WS-2's standing P0** (🔴 OWNER-GATE end-to-end). Data-visibility departure recorded: org-visible to `feature:crm` holders in v1, `owner_email` is assignment not ACL (D-CRM-3; workflows v1 is the precedent) — revisit at WS-14 `group:` grants / colleague #1. **Pipeline blueprint added 2026-08-07 (spec §5.1) after the owner's first live board session found lanes out of order and imported stages at 0% probability — root cause: the importer appends unseen Zoho stages past the seeds at probability 0, and 144's seeds renamed Zoho's defaults so name-match missed them; the admin API that could fix it is headless.** New tickets: **f** pipeline truth + settings UI — **BUILT 2026-08-07** (branch `ws-26f-pipeline-truth`, **no migration**: 144 already carried `position`, `probability`, `type`, `closed_at` and `expected_close_date`). `routes/crm/stage_metadata.py` = `POST /crm/import/zoho/stages`, floor `admin:access:manage`, **dry-run by default** and `?apply=true` to write; **>1 pipeline STOPS before the DB is opened** (D-CRM-11); f4's `closed_at` proxy is a direct UPDATE that bypasses `mark_dirty_on_update`, asserted statically against the statement text AND the module's call graph because the shared fake writes only what a SET clause names. Two settings readers on the Zoho read client with `ZohoScopeError`/`ZohoApiVersionError` so **no-scope, no-data and no-such-endpoint are three different reported outcomes** — and no-scope is the *expected* first answer, since the tenant's refresh token was never minted with `ZohoCRM.settings.*`. D-CRM-10's clamp landed in `admin.py::_validate_status` reading the ROW rather than the payload (a PATCH naming only `type`, or only `probability`, contradicts the rule only in combination with what is stored). `?tab=settings` is the headless-API fix and needs no Zoho token at all. **Nothing has been run against the tenant — dry run included.** · **g** forecast & funnel reports off `crm_status_changes` — **BUILT 2026-08-07** (branch `ws-26g-reports`, **no migration**: 144's `crm_status_changes` already carried every column). `routes/crm/reports.py` = four read-only endpoints (`/crm/reports/{pipeline,funnel,win-loss,owners}`) on the shared gated router, plus `?tab=reports`. **`WEIGHTED_SQL` lifted into `core.py`** (pipeline re-exports it) so the forecast formula has ONE definition, and `core.status_wire` absorbed two duplicate status projections instead of gaining a third. **The cross-language parity mechanism is minted here, not inherited** — the `priority.ts ⟷ priority.py` "precedent" is two hand-kept mirrors joined by a comment with no shared fixture anywhere: `tests/fixtures/crm_weighted_parity.json` (new directory) is read by BOTH pytest (through the emitted SQL, whose expression `_crm_fakes._WEIGHTED_SUM_RE` parses out of the statement text) and vitest (through `board.ts::weightedDeal`/`weightedRows`). ⚠️ **The funnel is defined against what the log RECORDS, not what its name suggests**: `crm_status_changes` logs transitions only, so all 551 imported deals have zero rows and "entered" is a VISITED-SET union (`from_status`, `to_status`, and the deal's current stage) — a `to_status` count reports an empty funnel for the whole live board; dwell is grouped by **`from_status`** (the stage being LEFT, the opposite key from the naive reading); the log stores NAMES, so a renamed lane orphans its history and orphans are REPORTED in `unmatched`, never dropped; and NULL `closed_at` — every imported closed deal until f4's owner-gated backfill runs — falls outside the trailing window, with the count reported so a 0% win rate is explicable rather than mysterious. The lost-reason breakdown carries a NAMED unattributed bucket: the earlier "complete by construction" claim is FALSE, since the importer bypasses both gates and `lost_reason_id` is `ON DELETE SET NULL`. **No `GROUP BY` is emitted** — the ticket asked for the choice to be stated: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would stop BEING the expression the fixture and the fake read. 47 hermetic cases + 20 vitest + the 14-row shared fixture on both sides, 5 mutants red. **Two findings:** the `entity_type='deal'` mutant initially SURVIVED (the funnel keys through deal ids, so a lead row is excluded twice over) — closed with a row stamped `lead` against a DEAL's id, which is realistic precisely because `entity_id` has **no FK**; and `_crm_fakes`' `lower(col) = :param` reader matched a NULL column, which SQL never does, making the unassigned-owner bucket count rows its own aggregate never summed · **h** stage entry-requirements + rot badges (after f2) · **i** merge/bulk/CSV/saved-views (🟡 spec-thin, audit-narrow first). **Demo path (2026-08-07, spec §9.0): full chain + all gates intact, tickets re-sequenced not thinned; f gained f4 — imported won/lost deals have no `closed_at` (importer never stamps it), backfilled from Zoho `Closing_Date` as a labeled proxy via a direct UPDATE that MUST bypass dirty-marking or ~500 no-op pushes queue for the live tenant.** | +| **WS-26** | **CRM app — native CRM + Zoho retirement** *(minted 2026-08-05)* | `specs/crm_app.md` | ✅ **a + b + c BUILT + DEPLOYED** · ✅ **d read half BUILT + DEPLOYED** · ✅ **D2 = d-email BUILT 2026-08-07** (branch `ws-26d-email-timeline`, merged) · ✅ **D4 = d-write MERGED + DEPLOYED 2026-08-08 (PR #400, no migration; deploy 31217978773 log-verified)** · ✅ **D5 = d-autolead BUILT 2026-08-08 (branch `ws-26d-autolead`, migration 158; flag OFF, NOT flipped, NOT deployed)** · ✅ **D1 = f BUILT 2026-08-07 (branch `ws-26f-pipeline-truth`, NOT run against prod)** · ✅ **D3 = g BUILT 2026-08-07 (branch `ws-26g-reports`, no migration)** · 🟢 **DEMO CRITICAL PATH COMPLETE (owner-directed 2026-08-07, spec §9.0): ~~D1 f~~ (∥ D2 d-email) → ~~D3 g~~ → ~~D4 d-write~~ → ~~D5 d-autolead~~** · 🟡 **h/i/e deferred past the demo; i spec-thin** | Research pass 2026-08-05: `frappe/crm` (AGPL — **concepts only, no code**), `trycompai/crm` (MIT), full-tree Zoho sweep. **Zoho today is a read-only nightly mirror** into the Phase-0 graph tables (`person`/`customer`/`deal`) with no UI, no write path, and **no Leads pull** — so leaving Zoho is import-and-retire, not a live cutover. Spine: Frappe's lead→convert→deal+contact+organization with **statuses-as-data** (color/position/type/probability); trycompai's single activity-spine table + `source` provenance + `last_activity_at` discipline. **BO-10 contribution: WS-26a adds the shared engine seam (`gateway/db.py::get_engine()`, tasks converted as proof) instead of engine 13.** Tickets: **a** schema + feature registration + core API — **BUILT 2026-08-05** (mig `144_crm.sql`, `feature:crm`, `gateway/db.py` seam + tasks converted, `routes/crm/`; **migration 144 applied on prod and `/crm` live as of 2026-08-06**) · **b** **Zoho two-way sync — BUILT 2026-08-05** (branch `ws-26b-zoho-sync`: `list_leads` + `list_deleted` on the read client, the single write client `ingestion/sources/zoho/writer.py` with one grep-asserted caller, mig `145_crm_zoho_sync.sql` (dirty columns + `crm_zoho_tombstones` + `crm_sync_cursors`), `routes/crm/{import_zoho,sync_zoho,broker_handlers}.py`, `crm.zoho_*` broker handlers registered from `main.py`, 80 new hermetic tests). *(Re-scoped 2026-08-05, owner-directed D-CRM-7: "faithful two way sync until we do away with Zoho entirely" — coexistence is bidirectional, not import-once.)* **Measured 2026-08-06: mig 145 is applied on prod and the BACKFILL HAS RUN — 737 orgs / 1,189 contacts / 1,516 leads / 551 deals / 1,909 notes, zero dirty rows, zero unmatched owners; the §7.1 pre-flip curl confirmed the tenant honors RFC-1123 `If-Modified-Since` (304). The PUSH direction has still never run: `CRM_ZOHO_SYNC` ships OFF, nothing has ever written the live Zoho tenant, and enabling the flag or hand-running a push cycle against prod stays OWNER-GATE §6.** WS-1's "no Zoho write path anywhere" clause was corrected in the same change (done-when 6) · **c** UI + the API addendum — **BUILT 2026-08-05** on branch `ws-26c-crm-ui` atop 26a and **merged with b into `ws-26-crm-app` 2026-08-06** (`/crm` app + BFF proxy; the three frontend registration points with `CenterApp` re-typed so `live ⇒ href` is a compile error; `routes/crm/deal_contacts.py` with one-primary-per-deal enforced on the shared `core.link_deal_contact` seam the convert path now also uses — 26b's importer is the one excepted writer and computes `is_primary` in-statement so a backfill can never demote a hand-set primary; `organization_name` on the deal list + board via a derived-table LEFT JOIN; the three review residuals — `?status_id` on a pipeline-less entity → 422, explicit `null` on a defaulted NOT NULL column → 422 not a driver 500, and a hand-edited `lead_name` surviving a name-field PATCH. **Deployed:** migrations 144 and 145 are applied on prod as of 2026-08-06 and `/crm` is live, so live rendering, drag persistence and deep links are owner-verifiable now) · **d** integrations — **audited 2026-08-06 GO-NARROWED and the narrowed slice is BUILT** (branch `ws-26d-agent-crm`): `apps/agents/agent-crm/` (`crm-assistant`, MAF, four READ tools over the existing `/crm` routes carrying the caller's `X-User-Email`, read-only enforced at the transport by a GET-only method allowlist) registered in `_KNOWN_AGENTS` + `_AGENT_REGISTRY` + `agent_registry.json`, plus `"crm"` added to the WhatsApp `_KNOWN_SYSTEMS` allowlist **parse-only** (nothing writes `wa_contacts.entity_ref`, the `crm` context block stays `None`, both pinned by test). **The three held-back items are now DISPATCHABLE — their doc blockers (B3/B4/B5/B7) were closed 2026-08-06 in `crm_app.md` §9.1-§9.3, every anchor read off `origin/main` rather than recalled:** **WS-26d-email** (the timeline join is CALLER-scoped, never record-scoped — it reuses the email app's `_account_scope` predicate, copied into `routes/crm/` rather than imported per D-CRM-4, joins by thread not message, inbound `from_address` only, and needs a new address index at the next free migration number) · **WS-26d-autolead — BUILT 2026-08-08** (branch `ws-26d-autolead`, migration **158** `crm_auto_lead_cursors`). Hook = `routes/email/scheduler_hooks.py::process_new_mail`, the one seam scheduler+manual+webhook all funnel through; the per-message rules loop was considered and REJECTED because a classifier outage there double-fires and history backfills never reach it. ⚠️ **That same seam is ALSO reached by deep resyncs** (2026-08-08 audit blocker G1, closed by PR #402 before the build), which is why the step keeps a per-account THREE-timestamp cursor: `activated_at` (the start of the current ON epoch) makes `received_at > activated_at` the backfill discriminator, `processed_watermark` is the incremental one, and `last_run_at` is the dormancy clock. Without the first, connecting a second mailbox mints a lead per unknown sender across a year of mail, each queued for the live tenant. ⚠️ **Diff review closed two P1s the first cut had, and both needed a running cursor to see:** `activated_at` alone did nothing about an OFF→ON round trip (measured: 27 leads minted for a 27-day OFF window), so a gap in `last_run_at` beyond `REANCHOR_GAP_SECONDS` (3600) re-anchors all three timestamps and mints nothing from the gap; and the watermark advanced over messages that RAISED (measured: 3 candidates, 3 errors, 3 leads lost for good), so it now advances over the successful PREFIX only, with a held cursor reported at WARNING on `sync.auto_lead_stalled` every cycle. **Dormancy reads `last_run_at` and not the watermark deliberately** — the watermark tracks MAIL, so a merely quiet mailbox would be re-anchored and would drop the first message to arrive afterwards, every Monday. Unknown-sender check mirrors `_maybe_block_cold`'s two steps and adds a third (no `crm_contacts`/`crm_leads` row with that `lower(email)`) — **the ticket's original `ON CONFLICT DO NOTHING` could not have fired** (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so dedup is that SELECT guard plus in-batch de-duplication, with the cross-invocation race accepted and the UNIQUE index refused. Colleague suppression is TWO gates: `sender_scope` (which fails SAFE to "external", the wrong direction here) and the normalised internal-domain list. The originating message is logged `type='system'` — outside `sync_zoho`'s `type IN ('note','task')` push predicate — with subject + sender in `meta` and **`body` empty**, so the mail's content never leaves the native CRM even though the lead does. The lead goes through `records.create_record`, never raw SQL. **Built, flag `False`, NOT flipped, NOT deployed; the flip stays §6 (b).** 73 hermetic cases; 13 mutants red and reverted (one of them precision-checked) · **WS-26d-write — BUILT 2026-08-08** (branch `ws-26d-write`, **no migration**: every route the four tools call already existed). `request_confirmation` awaited at the top of each tool before any mutating request is built, fail-closed, and the `non_interactive_default` keyword is asserted ABSENT from the whole module rather than asserted != "approve" — pinning the argument rather than the value means a mutant does not get to pick a spelling the fence has not heard of. `_ALLOWED_METHODS` **widened, never deleted**: `{GET, POST, PATCH}`, still checked inside `_request`, with `DELETE`/`PUT` and any `_delete`/`_put` helper still absent, so the check that used to enforce "read-only" now enforces "never destroys". Path fence extended past `ast.JoinedStr` to `.format`/`%`/`+` (the re-review's P2) and — the part that makes it maintainable — **tested against synthetic sources one per idiom**, so "the fence went blind" is a red test rather than a silent gap. Two supervisor rulings landed as built: `update_deal_status` resolves the stage BY NAME inside the tool against `GET /crm/statuses/deal` (no UUID on the LLM surface; an unknown name returns the real lane names), and a lost-type target requires a `lost_reason` resolved the same way against `GET /crm/lost-reasons` — pre-empting the 422 the "close this as lost" demo beat would otherwise hit — with the vocabulary **only ever read, never created**. `create_lead` takes **no `owner_email` argument at all** (the route derives it from the acting user), deleting an LLM-filled identity field from the surface entirely. ⚠️ **One recorded departure from done-when 1**: the invariant asserted is *no mutation before consent*, not *no HTTP before consent* — two tools must read to describe honestly what they are about to do, and every pre-card call being a GET is itself pinned; the two tools that owe nothing to a pre-read are still held to literally zero calls. `@_annotate_risk` is the shared annotation convention and is NOT enforcement; the Action Broker covers the Zoho push, not the native write — the two fail in OPPOSITE directions and are not interchangeable. 76 new hermetic cases + `test_crm_agent.py` 87 → 143; ten mutants run red and reverted. **Built, not deployed.** The push-queue question is CLOSED — **D-CRM-9 (owner, 2026-08-06): agent-originated writes queue for Zoho exactly like human ones** (🟡 remainder; the flip stays OWNER-GATE) · **e** cutover + retirement inventory + **Zoho refresh-token revoke, which executes part of WS-2's standing P0** (🔴 OWNER-GATE end-to-end). Data-visibility departure recorded: org-visible to `feature:crm` holders in v1, `owner_email` is assignment not ACL (D-CRM-3; workflows v1 is the precedent) — revisit at WS-14 `group:` grants / colleague #1. **Pipeline blueprint added 2026-08-07 (spec §5.1) after the owner's first live board session found lanes out of order and imported stages at 0% probability — root cause: the importer appends unseen Zoho stages past the seeds at probability 0, and 144's seeds renamed Zoho's defaults so name-match missed them; the admin API that could fix it is headless.** New tickets: **f** pipeline truth + settings UI — **BUILT 2026-08-07** (branch `ws-26f-pipeline-truth`, **no migration**: 144 already carried `position`, `probability`, `type`, `closed_at` and `expected_close_date`). `routes/crm/stage_metadata.py` = `POST /crm/import/zoho/stages`, floor `admin:access:manage`, **dry-run by default** and `?apply=true` to write; **>1 pipeline STOPS before the DB is opened** (D-CRM-11); f4's `closed_at` proxy is a direct UPDATE that bypasses `mark_dirty_on_update`, asserted statically against the statement text AND the module's call graph because the shared fake writes only what a SET clause names. Two settings readers on the Zoho read client with `ZohoScopeError`/`ZohoApiVersionError` so **no-scope, no-data and no-such-endpoint are three different reported outcomes** — and no-scope is the *expected* first answer, since the tenant's refresh token was never minted with `ZohoCRM.settings.*`. D-CRM-10's clamp landed in `admin.py::_validate_status` reading the ROW rather than the payload (a PATCH naming only `type`, or only `probability`, contradicts the rule only in combination with what is stored). `?tab=settings` is the headless-API fix and needs no Zoho token at all. **Nothing has been run against the tenant — dry run included.** · **g** forecast & funnel reports off `crm_status_changes` — **BUILT 2026-08-07** (branch `ws-26g-reports`, **no migration**: 144's `crm_status_changes` already carried every column). `routes/crm/reports.py` = four read-only endpoints (`/crm/reports/{pipeline,funnel,win-loss,owners}`) on the shared gated router, plus `?tab=reports`. **`WEIGHTED_SQL` lifted into `core.py`** (pipeline re-exports it) so the forecast formula has ONE definition, and `core.status_wire` absorbed two duplicate status projections instead of gaining a third. **The cross-language parity mechanism is minted here, not inherited** — the `priority.ts ⟷ priority.py` "precedent" is two hand-kept mirrors joined by a comment with no shared fixture anywhere: `tests/fixtures/crm_weighted_parity.json` (new directory) is read by BOTH pytest (through the emitted SQL, whose expression `_crm_fakes._WEIGHTED_SUM_RE` parses out of the statement text) and vitest (through `board.ts::weightedDeal`/`weightedRows`). ⚠️ **The funnel is defined against what the log RECORDS, not what its name suggests**: `crm_status_changes` logs transitions only, so all 551 imported deals have zero rows and "entered" is a VISITED-SET union (`from_status`, `to_status`, and the deal's current stage) — a `to_status` count reports an empty funnel for the whole live board; dwell is grouped by **`from_status`** (the stage being LEFT, the opposite key from the naive reading); the log stores NAMES, so a renamed lane orphans its history and orphans are REPORTED in `unmatched`, never dropped; and NULL `closed_at` — every imported closed deal until f4's owner-gated backfill runs — falls outside the trailing window, with the count reported so a 0% win rate is explicable rather than mysterious. The lost-reason breakdown carries a NAMED unattributed bucket: the earlier "complete by construction" claim is FALSE, since the importer bypasses both gates and `lost_reason_id` is `ON DELETE SET NULL`. **No `GROUP BY` is emitted** — the ticket asked for the choice to be stated: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would stop BEING the expression the fixture and the fake read. 47 hermetic cases + 20 vitest + the 14-row shared fixture on both sides, 5 mutants red. **Two findings:** the `entity_type='deal'` mutant initially SURVIVED (the funnel keys through deal ids, so a lead row is excluded twice over) — closed with a row stamped `lead` against a DEAL's id, which is realistic precisely because `entity_id` has **no FK**; and `_crm_fakes`' `lower(col) = :param` reader matched a NULL column, which SQL never does, making the unassigned-owner bucket count rows its own aggregate never summed · **h** stage entry-requirements + rot badges (after f2) · **i** merge/bulk/CSV/saved-views (🟡 spec-thin, audit-narrow first). **Demo path (2026-08-07, spec §9.0): full chain + all gates intact, tickets re-sequenced not thinned; f gained f4 — imported won/lost deals have no `closed_at` (importer never stamps it), backfilled from Zoho `Closing_Date` as a labeled proxy via a direct UPDATE that MUST bypass dirty-marking or ~500 no-op pushes queue for the live tenant.** | | **WS-27** | **Projects app — native project management + ClickUp retirement** *(minted 2026-08-05)* | `specs/project_management_app.md` | ✅ **a + b + d + e + i BUILT 2026-08-06 · j + k + l + m + n BUILT 2026-08-07** · 🟢 f dispatchable · 🟡 c gated · 🟡 h sequenced | Research pass 2026-08-05: `Paca-AI/paca` v0.11.0 (Apache-2.0 — **patterns adopted, no code translated**; findings + the adopt/adapt/refuse table live in `specs/paca_pm_research_2026-08.md`, reference-only), plus a full-tree ClickUp sweep. **ClickUp today is TWO independent systems** — the Phase-0 graph mirror (read-only, shallow) *and* the per-user Tasks-app connector with a **live broker-gated write path** — so leaving ClickUp is coexistence-sync-then-invert, **not** WS-26's import-and-retire; the constraint-8 inversion is staged and recorded in spec §7. Spine: Paca's two-self-FK hierarchy (departments→projects→subprojects→tasks→subtasks as `pm_projects` + `pm_tasks`, types-as-data with the Epic-root rule), statuses-as-data with a semantic `category` (D-CRM-2 convergence), per-view fractional ordering (`pm_view_task_positions` — what lets People-Center and Center-slice boards order the same task differently), and a single activity spine. **First data-scoped app:** `pm_project_grants` on the shipped `email\|group:\|org` vocabulary (D12; sibling of C1's D13, which is unchanged), 404-not-403, and the full-portfolio view gives D14's zero-consumer `data:org:read` its **first consumer**. **Three owner answers recorded 2026-08-06 as D-PM-8/9/10, and two of them changed the build:** **D-PM-8** no portfolio/program layer — grants are the only grouping axis, a cross-department project simply carries several (a `pm_programs` table stays purely additive if wanted later); **D-PM-9** agent edits to ClickUp-linked tasks are treated **exactly like human edits** (*the agent proposed queueing agent-originated pushes for approval and was overruled*) — so during coexistence a mistaken agent edit reaches the live workspace with no human in between while `ACTION_BROKER_ENFORCE` is off; bounded by attribution (`agent:`), timeline-reversibility, and the fact that the enforce flip converts the whole class to queue-on-approval. Read D-PM-9's Cost paragraph before building WS-27f; **D-PM-10** ClickUp Spaces map to Centers **explicitly**, from agent-proposed suggestions (assignee-overlap → name match → EVAL-LOCKED content classification), owner-confirmed, applied as `group:` grants — and an **unmapped Space still imports in full with no group grant**, staying reachable in `/projects` for `data:org:read` holders and its assignees. This supersedes the "pilot vs all Spaces" framing: scope is now a per-Space decision the plan step surfaces, so a pilot and a full import are one code path. Tickets: **a** schema + `feature:projects` both sides + core API on the `gateway/db.py` seam — **BUILT 2026-08-06** (mig `146_projects.sql`, `routes/projects/` with zero `create_async_engine` calls, 115 hermetic cases + 5 mutants measured red; **not deployed — the migration has not been applied anywhere**) · **b** ClickUp org importer **+ the Space→Center mapping plan** — **BUILT 2026-08-06** (`routes/projects/mapping.py` + `import_clickup.py`; `POST /projects/import/clickup/plan` proposes and writes nothing, `POST /projects/import/clickup` applies the confirmed mapping; 25 hermetic cases, 4 mutants red incl. *applying the suggestion instead of the confirmed mapping*; **neither endpoint has been run — prod execution stays OWNER-GATE, §6**) · **c** two-way coexistence sync — three-way field merge, conflicts logged to the timeline (🟡 **blocked on WS-1's BO-1a + BO-1b, named prerequisites**; enabling push is OWNER-GATE) · **d** UI + Center (app + scope) projections, no forks — **BUILT 2026-08-06** (`src/app/projects/` tree + board + list + task panel + timeline, BFF proxy, nav/access registration, all six Centers linking at the SAME `/projects` path and differing only by `?center=`; 34 vitest cases incl. a registration fence, 6 mutants red) · **authoring landed 2026-08-06** — d shipped a UI that could read and drag but never **create**, so a member could only work with rows a ClickUp import had put there: new department / subproject (from the node, where the parent already is), new task (status not sent — the API picks the project's default), subtask from the panel, and **assignees as chips where an agent and a person share one field** (`lib/assignees.ts`, 17 vitest cases, 7 mutants red). That last one is where D-PM-4 stops being a schema note and is the precondition for WS-27f's dispatch being reachable at all · **e** the personal lens — **BUILT 2026-08-06** and **its shape changed**: `D-PM-6` was revised (owner-directed — *"the personal task manager should be a proper extension … a cohesive whole"*) from a mirror into **one store**. `pm_tasks` is THE task table; private work is a personal project (`pm_projects.personal_owner`); the GTD overlay is **per-member** (`pm_task_personal`, mig `147_projects_personal.sql`) so two assignees can hold different dispositions. Assignment is no longer a sync — the inbox row IS the project row, and completing it there moves the shared status. 31 hermetic cases, 6 mutants red. **Its surface landed the same day** — "My work" above the project tree in the SAME app (`components/MyWork.tsx` + `lib/mywork.ts`, 17 vitest cases, 7 mutants red): capture-first, four work lanes that render even when empty, untriaged counted in the header (the Weekly Review's question, answerable only because dispositions are derived not stored), and a completion checkbox that moves the **shared** status. e had shipped API-only, so the cohesion the revision bought was true in the schema and invisible to a member. One repair it forced: `TaskPanel` read the *selected* project's statuses, wrong for a task opened from My work — now resolved from the task's own root project. **Cost accepted: `gtd_items` becomes legacy and WS-27h retires it** · **f** automation + agent dispatch — **BUILT 2026-08-06** (`routes/projects/automation.py` + `agent_dispatch.py`, the `pm_task` node type in `workflows/engine/`, `PM_EVENT_TOPICS` served by the catalog; 34 hermetic cases, 10 mutants red). Both halves of `workflows_app.md` §13 — **U1** the task-mutation node and **U7** dispatch. The engine imports a transport-free SERVICE, not a route, so an automation's edit is indistinguishable in validation from a human PATCH and lands the same timeline row; **status is named, never keyed** (a graph pinned to one project's status UUID could only automate that project); a `pm_task` node is deliberately **not** write-class (the approval gate is for outward writes — now pinned by a test rather than true by accident); and "already in target state" is asserted to issue **no UPDATE at all**, because `update_row` stamps `updated_at` and a redundant write is invisible in a diff while making a task look freshly touched. Assignment dispatches from an event SINK beside the workflows dispatcher, so a broken agent cannot fail the act of assigning somebody a task, and the handoff activity is committed BEFORE the run starts. **Engine defect found and fixed:** `resolve_value` keeps an unresolvable `{{ref}}` as-is at run time by design and `{{trigger.missing}}` passes the publish gate because its *root* is legal — the literal would have reached Postgres as a would-be uuid and returned "Task not found", pointing the maker at the wrong thing · **i** attachments — **BUILT 2026-08-06** (mig `150_projects_attachments.sql`; 25 cases, 10 mutants red): one file store (`gtd_attachments` reused, upload rules imported) with a thin `pm_task_attachments` join that carries the ACCESS decision, so a file is readable by whoever can see the task rather than only its uploader; no attach-by-id endpoint exists, because naming an arbitrary attachment id would let a caller attach somebody else's private capture to their own task and read it back. **Caught before shipping:** the projects BFF proxy re-serialised every POST as JSON, so a multipart upload would have reached the gateway with NO FILE while still answering 201 · **j–r** the rest of the ClickUp-parity gap, measured against the built tree and sequenced in spec §11 (attachments, notifications/@mentions, filters+saved views, custom fields, tags, bulk edit, recurring, dependency UI, calendar, search) — all 🟢, and **n (bulk edit) gates g — and n is now BUILT (2026-08-07), so that dependency is SATISFIED**: an import that cannot be re-triaged in bulk is one somebody abandons halfway, leaving two live systems, which is the state the retirement exists to end. g itself stays 🔴 OWNER-GATE for its own reasons (§6); this changes the prerequisite, not the gate · **g** cutover + retirement inventory (both ClickUp systems) + token revoke + the root-`AGENTS.md` constraint-8 amendment (🔴 OWNER-GATE end-to-end) · **h** `gtd_items` retirement — the cost D-PM-6's revision accepted: union read, row migration into `pm_tasks` + `pm_task_personal`, then `items.py`'s 27 owner-scoped predicates retire with the table they scope (🟡 after e; the data move is 🔴 OWNER-GATE — it rewrites the owner's live task store).. **j BUILT 2026-08-07** (mig `152_projects_notifications.sql`, `routes/projects/notifications.py`, the header bell + mention picker; 39 hermetic + 27 vitest cases, 10 mutants red): closes §11.2's second gap — *"assignment is silent"*. **Three rules decide who hears**, each the whole reason for a rule: never the actor (a bell that pings you about your own click gets muted, and a muted bell notifies nobody about anything); never an agent (they are handed work by the WS-27f dispatch sink, so a row addressed to one sits unread forever inflating a badge nobody can clear — enforced in Python AND by a CHECK); and **never somebody who cannot open it**, which is the security property: the notification carries the task's TITLE, so delivering one outside the grant closure leaks it and lands them on a 404. That third rule needed `resolve_visibility_for`, which answers for a THIRD PARTY — `resolve_visibility` reads a `UserContext` and the recipient of a mention has no request in flight — by reading the tables `/auth/me` reads and handing them to the **real** `build_access`, so wildcards and allow/deny overrides resolve identically on both paths. Notifications are written **inside the transaction**, not emitted on the bus: `emit` swallows failures by construction so a broken workflow can never fail a task edit, which is right for agent dispatch and wrong here. A mention is an **address, not a name**, because 148 dropped `UNIQUE(name)` — `@Priya` has no answer. **Two bugs found on the way in, both shipping at the time:** every project-task file upload was answering **422** (`ACTIVITY_TYPES` never learned 150's `attachment`; all 25 attachment tests passed because they monkeypatch `record_activity` — the seam under test was mocked out), fixed with two tests that READ the migrations; and `/projects?task=` did nothing, though the People Center has linked there since WS-28b. **WS-27b's UI BUILT 2026-08-07** (`components/ImportClickUp.tsx`, `lib/importPlan.ts`; 18 vitest cases, 3 mutants red): the importer shipped with WS-27b and was **unreachable from the product** — the empty state named "import a ClickUp workspace" and no control anywhere did it, so a new install stayed empty and the only route to real data was curl. Three steps, only the last of which writes: Preview (`/plan`, reads the tenant) → Dry run (`dry_run:true`, exercises the flattening) → Import. **The mapping stays the owner's act (D-PM-10)** and the UI is built so it stays one: the suggestion is pre-filled and shown beside its confidence IN WORDS ("a guess — check it", not `0.45`, because a bare number invites acceptance without looking), and a CONFIRMED mapping always beats a fresh suggestion so a re-run never silently re-maps a Space somebody already ruled on. Unmapped Spaces are a notice rather than a blocker, matching the importer. ⚠️ The gate is unchanged and is now exactly one click: building this was agent-safe, pressing Import is the owner's act, and no agent has run either endpoint against production. **The Tasks-app mirror path BUILT 2026-08-07** (`routes/projects/import_tasks.py`, `POST /import/from-tasks`; 43 hermetic cases, 7 mutants red) — owner-directed: *"just show up all the data that is there in the Tasks app inside the Projects app"*. A SECOND importer rather than a flag: the ClickUp one needs a live token, spends LLM budget and demands a Center mapping BEFORE anything is written, which is backwards for "show me my work today". This reads `gtd_projects`/`gtd_items` — the mirror already on the box — so no API call, no token, no model spend, and it works when the connector is stale. One named department, with the real ClickUp shape beneath it — **Space → Folder → List** rebuilt as projects, each carrying its own `clickup_id`/`clickup_kind`, so promoting a Space node is how the department split happens later. **Verified against a real Postgres** (full migration set + seeded mirror), which found THREE defects the hermetic suite could not: `gtd_projects.space_id` is LOCAL-only so the Space was always NULL; `pm_projects` has no `clickup_snapshot` column, so the first real click would have 500'd (this shipped in #393 and was fixed before anybody pressed it); and the preview under-counted 4 vs 7 because container nodes were only tallied on the write path. The root IS org-granted (same act as `create_node`, narrower than bulk-granting a tenant). Pinned: nothing outside `pm_*` is written; only `source <> 'LOCAL'` rows are read (a personal capture must not be published to a shared board); the provider's own status names are kept; an orphaned task is COUNTED, not dropped. **Mutation found a real gap**: deleting `dry_run` from the write guards left every test green, because on a FIRST dry run `root_id is None` blocks the write anyway — on a SECOND one the department exists and `dry_run` is the only protection. That is the realistic case (preview → import → preview again) and it now has its own test. **k BUILT 2026-08-07** (`routes/projects/filters.py`, `lib/grouping.ts`, `components/FilterBar.tsx`; 34 hermetic + 24 vitest cases, 13 mutants red, and 23 checks against a REAL Postgres) — closes §11.2's third gap, the one whose name was the sentence *"my open bugs in Ops, grouped by assignee"*. **ONE filter builder serves both the list endpoint and saved views**, because a saved view is nothing but a stored set of these filters and two implementations would drift — a *saved* view showing a different set than the same filters typed by hand is the one thing it may not do, so a test compares the two outputs directly. **Every filter is a WHERE clause**: paging happens in SQL, so a filter applied in Python after `LIMIT` returns short pages, and *"page 2 is empty but there are 40 more"* is a bug people work around for months instead of reporting. **`overdue` means past due AND still open** — a finished task with a past due date is done, and permanent red is how a board teaches people to ignore red. **An unknown category is a 422 naming the five real ones**, not an empty board a client reads as "this project is empty". Unknown config **keys** are DROPPED while a bad **value** falls back, because those are different failures: a view is a preference written by an older client, so refusing one over an unrecognised key would make every deploy a migration of everybody's saved views, whereas rendering still has to produce something. On the board, **a task with two assignees appears in BOTH columns** (it is both people's work; picking one hides it from the other, so the header counts tasks not group sizes), **empty status lanes are kept** while every other grouping drops empties (a missing "In progress" column reads as "no such state", not "nothing in progress"), and **dragging is offered only when the columns are statuses** — a drop writes the field the columns represent, and status is the one that is a plain PATCH, so a card that can be dragged into a column which cannot accept it and snaps back is worse than an honestly static column. `toConfig` is deliberately NOT `toQuery`: a query string carries only text so `toQuery` writes `"true"`, and `fromConfig` refuses a string where a toggle belongs, so a view built from query shape would come back with every toggle silently cleared. The project's **order-bearing board is withheld from the chips** — `tree.py` seeds one `board` view per project and it owns every `pm_view_task_positions` row, so offering its ✕ would offer to delete every hand-arranged position; saved views sit at position 300 above the seeded pair and `orderBearingView` is one function used by both the drag handler and the delete guard. **A FIFTH live bug, found the same way as the previous four:** `due_before` was `CAST(:due_before AS timestamptz)` with the raw query-string value — asyncpg infers the parameter's type FROM that cast and then refuses to encode a `str`, so the query never reached Postgres and **`?due_before=…` answered 500** while the hermetic fake, which agrees with whatever SQL it is handed, stayed green. `parse_when` parses on this side and binds a real `datetime`; garbage is a 422 that says what was expected; a naive value is read as UTC rather than inheriting the connection's TimeZone. Two tests — one on the bound value's TYPE, one refusing any `CAST(:param AS timestamp…)` anywhere in the builder — so the next `after=` filter written the obvious way fails in CI instead of in front of a member. **l BUILT 2026-08-07** (mig `155_projects_custom_fields.sql`, `routes/projects/custom_fields.py`, `lib/customFields.ts` + the panel block and the Fields dialog; 47 hermetic + 36 vitest cases, 23 mutants red, 35 checks against a REAL Postgres) — ClickUp's signature feature, and the shape §5's non-goals already recorded as the additive path: **definitions in a table, values denormalised onto the task as JSONB keyed by `field_key`**. NOT a row per (task, field): that is the textbook EAV answer and costs a join per field on every board paint — five fields across two hundred imported tasks is a thousand rows to gather and re-pivot, per render — whereas the JSONB column arrives with the task for free. **The cost is stated rather than discovered**: a value is not referentially tied to its definition, so the DATABASE cannot stop a key no definition owns from being written, and that guarantee moves into Python. Hence the validation IS the feature: an unknown key is a **422 not a silent drop** (a typo that no-ops looks exactly like a save); a patch **MERGES** (a client that knows three of five fields must not wipe the other two — and an older client, or an automation written before a field existed, is precisely that client); an explicit **null CLEARS the key** rather than storing a null, because it is the only way to express "unset this" and a stored null makes "never filled in" and "deliberately emptied" one value in every filter; and **`true` is not the number 1** — `isinstance(True, int)` is True in Python, so the coercers are one-per-type in a dispatch dict specifically so the boolean check can never drift below the number one. **The deliberate departure from Paca:** its research notes record "deleting a definition does not clean task data" as an accepted cost; it is NOT accepted here, because a key left in the JSONB is invisible — no definition means no column, no form row, no filter — until somebody recreates the name and every old value resurfaces carrying the new meaning. The cleared count is reported (R7/R8). Two things a definition may not change once values exist, both because the stored values would stop meaning what they say: **`field_key` is never editable** (it is the identity every value is filed under) and **`field_type` is a 409 naming the count** (text→select cannot re-interpret what is already written); dropping a select option some task holds is refused the same way, adding one is free, and the UI shows the derived key while the name is still being typed since that is the last moment anybody can change it. **Custom fields are REVERTIBLE, which is what makes them first-class:** `patch_task` folds a custom edit into the SAME `field_change` activity under `custom.` rather than inventing an activity type — `record_activity` refuses a type the CHECK does not list, the trap that made every attachment upload answer 422 — and revert restores by **merging onto what the task holds NOW**, never writing back the whole object, since another field may have been edited since and replacing the blob would silently undo that too. **A bug the ticket's own tests caught before it shipped:** `changedValues` compared a form's boolean against a `null` baseline, so a task with an unanswered checkbox sent `open:false` on EVERY save and posted a timeline entry for an edit nobody made — a checkbox has no "unset" state to render, so `false` is its baseline. **And a fence that was quietly a subset check:** `test_projects_routes` asserted a LIST of mounted paths, which catches the module somebody remembered to add a path for; it now also reads the package directory and asserts every module declaring a `@router` route is imported by `__init__.py` — the C1 trap where a missing import mounts nothing while every direct-call test still passes. Verified by deleting the import and watching it fail. **m BUILT 2026-08-07** (mig `156_projects_tags.sql`, `routes/projects/tags.py`, `lib/tags.ts` + the panel picker, the filter row, a `tag` board axis and the Tags dialog; 31 hermetic + 37 vitest cases, 16 mutants red, 30 checks against a REAL Postgres) — the row the research notes left open ON PURPOSE: `paca_pm_research_2026-08.md` row 13 REFUSED Paca's model (*"a bare jsonb string array on tasks. No registry, no colors, no rename/merge — the weakest part of Paca's model"*) and §5 shipped `pm_tasks.tags TEXT[]` in its place with a registry named as additive later. **The array STAYS** — a join table would add a row per tag per task and a join to every board paint, to buy referential integrity this app enforces in one place, whereas the array arrives with the task and its GIN index (146) already answers "tagged X". What the registry buys: **one spelling per tag** (identity is case-INSENSITIVE via a unique index over `lower(name)` so two racing requests cannot create both, display is case-PRESERVING, and the task's array stores the REGISTRY's form — which is what makes "filter by bug finds all of it" true rather than aspirational, and what lets a rename be one statement); **rename**; **merge**; and a **colour**. **Applying an unregistered tag REGISTERS it** — refusing would make tagging a two-step errand (leave the task, create the tag, come back), which is how tagging gets abandoned, and an abandoned tag set is worse than a messy one. **The cost is stated: every typo becomes a tag** — which is exactly why merge is here and is not optional, and why the picker SHOWS the moment of creation rather than minting silently. **A rename onto an existing name is a 409, not a silent merge**: different operations, different outcomes, and one of them destroys a tag — quietly doing the destructive one because the names collided is what stops people using a rename button. **A task carrying BOTH tags ends a merge with the target ONCE** — the case that is easy to get wrong, and getting it wrong leaves a duplicate that renders twice and survives the next merge too; `merged_tags` is pure so that case is asserted directly, and the rewrite runs over affected rows in Python rather than as an `array_replace`, which would leave the duplicate. **TWO tag filters** because both questions get asked and neither answers the other: `tags` is ANY (`&&`) and `tags_all` is ALL (`@>`); collapsing them would silently pick a meaning, and with three tags the answers differ by almost everything. On the board a task with three tags appears in three columns, the same honesty as two assignees appearing in both theirs. **The migration backfills AND rewrites data, deliberately and narrowly:** `tags` has been on `pm_tasks` since 146 and the import path writes them, so an empty registry beside a tagged corpus means the first rename finds nothing; the winning display form is **the spelling people actually use** (most frequent, ties broken deterministically — `min()` alone would canonicalise 400 "Bug" to a single stray "BUG"), and task arrays are then made to agree, only ever swapping one CASING of a tag for another, with the count reported in a NOTICE. **A bug the live run caught in that block:** the canonicalisation used the implicit-comma `FROM pm_tasks t, unnest(t.tags) ... LEFT JOIN` form, where the LEFT JOIN binds only to `unnest(...)` and `t` is not in scope for its ON clause — the migration aborted with "invalid reference to FROM-clause entry for table t". Rewritten as `CROSS JOIN LATERAL … WITH ORDINALITY`, which also fixed a second problem the first version would have shipped: `array_agg(DISTINCT ...)` sorts by its own expression, so every task's tag list would have come back alphabetised. **n BUILT 2026-08-07** (`routes/projects/bulk.py` → `POST /projects/tasks/bulk`, `lib/selection.ts`, `components/BulkBar.tsx` + checkboxes on board and list; 35 hermetic + 32 vitest cases, 16 mutants red, 34 checks against a REAL Postgres; **no migration**) — **the ticket §11.3 names as gating g, so that prerequisite is now satisfied.** It **reuses `automation.apply_task_patch` rather than growing a second writer**: that service exists because WS-27f needed an edit indistinguishable in validation from a human PATCH, and a bulk endpoint with its own field handling would be a third opinion about what a task edit is. Tags go through the same registry the panel uses, because the registry cannot be true if bulk is a second door into the array. **Status is named, never keyed — load-bearing here rather than stylistic**: a selection spans projects and a status id belongs to one root, so `status_id` for fifty tasks across three projects puts two thirds of them in a lane that is not theirs; it is a 422 with its OWN message, because it is the mistake somebody makes by copying a single-task PATCH body and "unknown field" would not explain why the thing that works on one task is refused on fifty. **Assignees and tags are ADD/REMOVE, never SET** — "assign these to Priya" means ALSO Priya, and a replace wipes every individual assignment the fifty tasks already carried; the destructive spelling is absent rather than discouraged. **Shape validated once, outcomes per task**, because those are different failures: an unsettable field is the same mistake for all fifty (422 before any write), while a status name present in one project and absent from another is a fact about THAT task — failing the batch for it makes a mixed selection unusable, which is precisely the selection somebody makes after an import. **An invisible task is SKIPPED, not an error** (R5: per-id reporting says exactly what a per-id 404 says, and aborting would let a caller probe for existence). **One transaction** — a re-triage that half happened is harder to recover from than one that did not, because nobody can tell which half. **Re-asserting a value is not a change** (`moved_people` is pure so the claim is asserted directly): fifty tasks already Priya's would each gain a timeline entry saying nothing, and a count nobody can trust is worse than no count. **ONE notification per person per batch, not one per task** — being handed fifty tasks should ring once and say fifty; fifty bells is a bell people turn off, which is WS-27j's own argument applied to the case that would have broken it. In the browser the selection is **pruned whenever the filter changes** (select forty, narrow to three, press Done must not act on thirty-seven nobody can see), a shift-click ranges over the board's ON-SCREEN order, a two-assignee card drawn in two columns counts once, and the outcome line names every category including the boring ones. **AND IT UNCOVERED A SHIPPED WS-27j BUG** (spec §11.12): `notifications.deliverable` probed only `project_clause('t.root_project_id')` while `core.task_visibility_clause` — whose docstring warns a second implementation "would drift the moment one is edited alone" — carries TWO ways in. So (1) anybody assigned work in a project they hold no grant on was judged undeliverable: they could OPEN the task, the assignment notified nobody, and the response told the assigner they could not see it — the silent assignment WS-27j exists to end, still open for the most common case in a grant-scoped app; and (2) scoping to `root_project_id` ALSO missed a grant made on a SUBPROJECT — the old test asserted that scoping on the argument that "probing `project_id` would miss a grant made on an ancestor", which is BACKWARDS, as a real-Postgres run showed: the closure is recursive and expands DOWNWARD. Fixed to use the shared clause; the test that encoded the wrong reasoning now records why it was wrong. **A fake collision the fix exposed:** `FakeDB` matched the rule-3 probe by substring, and the new clause embeds both `pm_task_assignees` and the closure's `UNION` — exactly the AUDIENCE branch's fingerprint — so `deliverable` got a list of assignees where it expected a visibility answer and four tests failed for a reason unrelated to the code under test. A fake that dispatches on substrings needs fingerprints that are SPECIFIC, not merely present | | **WS-28** | **People Center — directory, org chart, and the assignment seam** *(minted 2026-08-06)* | `specs/people_center_app.md` | ✅ **a + b BUILT 2026-08-06 · b-write BUILT 2026-08-07** · 🟢 c–e dispatchable · 🔴 f owner-gate | Scope owner-set 2026-08-06: **directory, skills, org chart, capacity, seats/roles — exactly what assignment and planning need**; leave/onboarding/hiring are named as later phases so their absence is a decision. **The fact this spec exists to settle:** there are TWO people stores and that is deliberate — `app_user` answers *can they sign in and what may they see*, `gtd_people` answers *who are they and what can they do*, and the directory must include people with **no login** (contractors), which is why the Projects app's assignee is a plain string. They join on lowercased email, and **P-1 fixes that join before it is relied on**: migration 49 made `name` UNIQUE and left `email` unconstrained, so today two rows may share an address and an email→person join is ambiguous. Surfaces: directory (honouring WS-24 N4's HR projection, with a *restricted* empty state distinct from *none*) · person page · org chart from `manager_id` with a Center overlay that **shows** department/group mismatches rather than smoothing them · capability search over stated skills → résumé evidence → the existing `capability_embedding`, which **suggests and never assigns** · seats & roles matrix (read + propose; applying a membership change stays owner-gated per §6 (d)). Closes WS-13's outstanding *People directory read view* item. Tickets **a** key-shape fix (🟢) · **b** directory + person page (🟢) · **c** org chart (🟢) · **d** capability search (🟢, ranking EVAL-LOCKED) · **e** the Projects seams — directory-backed assignee picker listing agents and directory-only people, capacity derived from open assigned tasks (🟢) · **f** seats & roles writes (🔴 OWNER-GATE).. **a BUILT 2026-08-06** (mig `148_people_key_shape.sql` + `scripts/import_hr_people.py`; 22 cases, 11 mutants red, 1 equivalent): `UNIQUE(name)` dropped, partial unique on `lower(email)`, status CHECK. **P-1 did not name its own consequence** — the HR importer upserts `ON CONFLICT (name)` and would have failed outright, so a `source_key` (`:`) carries the upsert instead, backfilled BEFORE the constraint is dropped while `name` is still distinct. **Neither new constraint may block a deploy** (main was bitten twice this month): a duplicate address is quarantined into a new `email_conflict` column with a deterministic winner rather than failing `CREATE UNIQUE INDEX`, and the status CHECK is added `NOT VALID` then validated in a guarded block, so an unanticipated legacy value leaves a NOTICE instead of stopping the deploy. ⚠️ `schema.generated.sql` NOT refreshed — needs a live DB; regenerate on the first deploy that applies 148. **b BUILT 2026-08-06** (mig `149_people.sql`, `routes/people/`, `src/app/people/`; 32 hermetic + 28 vitest cases, 11 mutants red): `/people` directory, person page with all four panels, and the People Center's "Directory & org chart" sub-app flipped live — closing WS-13's outstanding read view. **Its own feature slug**, not `feature:tasks`: a manager who needs the org chart should not be handed the personal GTD task manager to get it. The gate is new but the HR **projection is imported** from `tasks.core` and a test asserts the function's *identity*, since two answers to "may this caller see skills" are two answers waiting to drift. Three filters (the `q` skills clause, `skill`, `has_capacity`) are dropped without `admin:members:read` so search cannot become an oracle for the hidden field — and the response carries `hr_visible` so the UI says "restricted" rather than leaving a blank strip to read as "nobody filled it in". Load is **computed from open assigned tasks** and carries `unestimated`, because a bar built from the estimate sum alone shows somebody holding thirty un-estimated tasks as completely free. The work panel is scoped by the **viewer's** grants and answers `available:false` without `feature:projects`. **Registration is FIVE places, not four** — the fifth is `test_org_access_enforcement.GATED_ROUTERS`, hand-maintained, where an absent router is unchecked rather than passing; also added the named `test_projects_is_registered_on_both_sides` that WS-27a never wrote. **b-write BUILT 2026-08-07** (`people/components/PersonEditor.tsx`, `people/lib/form.ts`, `people/lib/write.ts`; 26 hermetic + 23 vitest cases, 7 mutants red): closes the regression the 2026-08-06 scope narrowing opened — deleting the tasks app's People view took `PersonEditor` with it, and with it the only UI for creating a person, editing skills and uploading a résumé. The GET-only `/api/people` proxy is **unchanged**: the writes go to `/api/tasks/people`, where they have always lived. Controls are absent rather than disabled, driven by a new **`can_manage`** flag on the reads — hide-rather-than-disable is impossible unless the read tells the UI, and the alternative is drawing the button and letting the click find a 403. **Restoring it turned up three ways migration 148 had already broken the write routes**, each a 500 in front of an admin rather than a test failure: the status vocabulary moved (49's `inactive`/`on_leave` vs 148's CHECK) and is now ONE tuple shared by filter, facets, select and validation; `create_person` still refused a duplicate NAME, preserving precisely the behaviour 148 dropped `UNIQUE(name)` to remove; and nothing checked the address 148 made unique, so a duplicate was an `IntegrityError` instead of a 409 naming the other row. **The lesson recorded in spec §10:** a migration that changes a table's shape has to be walked against every route that WRITES it — the read routes were built after 148 and were correct by construction, the write routes predated it and were never revisited | @@ -642,14 +642,24 @@ changes no runtime behaviour at all. The hook is the manual-sync route and the webhook all funnel through — and the flag is read at that CALL SITE, before the CRM step is entered, so the OFF state issues no CRM query; both the runtime regression and an AST assertion that the gate is -lexically outside the step live in `tests/unit/test_crm_auto_lead.py`. ⚠️ Two -things an owner should know before flipping: the first ON-state run per mailbox -only ACTIVATES the cursor (`crm_auto_lead_cursors`, migration 157) and mints -nothing — mail that arrived before that instant is history by construction, which -is what stops a deep resync minting a year of leads — and the accepted residual is -that two concurrent syncs of one account can double-mint one visible, -hand-deletable duplicate (a UNIQUE index on `crm_leads.email` is refused: 1,516 -imported rows may already carry duplicates, the migration-148 shape) · +lexically outside the step live in `tests/unit/test_crm_auto_lead.py`. ⚠️ Three +things an owner should know before flipping: **(1)** the first ON-state run per +mailbox only ACTIVATES the cursor (`crm_auto_lead_cursors`, migration 158 — 157 is +held by open PR #399) and mints nothing — mail that arrived before that instant is +history by construction, which is what stops a deep resync minting a year of leads; +**(2)** the same is true after the flag is turned OFF and back ON, or after the +service is down, for more than an hour: the cursor RE-ANCHORS, the gap's backlog +mints nothing, and the cycle says so at WARNING on `sync.auto_lead_reanchored`. +That guard was added by diff review after an OFF→ON round trip was measured minting +**27 leads for a 27-day OFF window**, each pushing unattended into the live tenant — +so turning the flag off is genuinely a stop, not a pause that accumulates; **(3)** +the accepted residual is that two concurrent syncs of one account can double-mint one +visible, hand-deletable duplicate (a UNIQUE index on `crm_leads.email` is refused: +1,516 imported rows may already carry duplicates, the migration-148 shape). One +operational note: a cycle that logs `sync.auto_lead_stalled` at WARNING every cycle +means a message at the head of the queue cannot be written and the cursor is +deliberately held — that is fail-closed toward the CRM and it needs a human, not a +restart · **(c) the WS-26e cutover + retirement** — the final import + parity check, repointing the graph-mirror consumers (`sales_views.py`, `reconciler.py`), retiring `ingestion/sources/zoho/` + cron + webhook + config (spec §7.4, which diff --git a/apps/services/gateway/AGENTS.md b/apps/services/gateway/AGENTS.md index b34e9cd6a..895a7f78c 100644 --- a/apps/services/gateway/AGENTS.md +++ b/apps/services/gateway/AGENTS.md @@ -54,7 +54,7 @@ webhook receivers, OAuth callbacks, and the Control Plane API. - ⚠️ **`reports.py` (WS-26g) — read-only, and the funnel is defined against what the log RECORDS rather than what its name suggests.** `GET /crm/reports/{pipeline,funnel,win-loss,owners}`; no write, no Zoho call, no flag, no migration. `WEIGHTED_SQL` moved into `core.py` beside `WEIGHTED_TYPES` when this became its second consumer (`pipeline.py` re-exports it, so no caller moved) — a second copy would defeat `_crm_fakes._WEIGHTED_SUM_RE`, which reads the expression OUT of the statement text precisely so a drifted formula changes the tests' answer; `core.status_wire` absorbed `admin`'s and `pipeline`'s duplicate status projections for the same reason. ⚠️ The owner leaderboard's bucket key (`.strip().lower()` in Python) and its aggregate predicate (`lower(trim(owner_email))` in SQL) are one normalisation written twice and must stay byte-consistent: while the SQL lacked `trim()`, a padded address split a bucket the tally had already merged, so the leaderboard under-reported an owner, dropped a deal into no bucket at all, and still said `omitted: 0`. Four properties are load-bearing. **(1)** `crm_status_changes` logs TRANSITIONS only — `create_record` writes no row and the importer writes none — so all 551 imported deals have zero rows and a deal's first stage is never a `to_status`; "entered" is therefore a VISITED-SET union (`from_status`, `to_status`, and the deal's CURRENT stage), and dropping that last term reports an empty funnel for the whole live board. **(2)** Dwell is grouped by **`from_status`**, the stage being LEFT — `to_status` would label every measurement one lane too far on, plausibly. **(3)** The log stores NAMES, not ids, so a lane rename orphans its history; orphans are tallied into `unmatched` and never dropped. `entity_type = 'deal'` filters every such read — and note it is defence-in-depth, not the sole guard, since the funnel also keys through deal ids: the test that makes it load-bearing seeds a row stamped `lead` against a DEAL's id, which is realistic because `entity_id` has **no foreign key** (the log outlives the row on purpose). **(4)** The trailing window is bounded at BOTH ends. NULL `closed_at` — every imported closed deal until WS-26f f4's owner-gated backfill runs — falls outside it (zeros, never "closed today"), with the count reported so a 0% win rate is explicable; and so does a FUTURE `closed_at`, because f4's proxy is Zoho's `Closing_Date`, a forecast date that imported deals routinely carry ahead of today — with a lower bound only, running the repair would have started counting next quarter's deals as closed this quarter and inflating the cycle average by their forward span. The lost-reason breakdown carries a NAMED unattributed bucket (the importer bypasses both gates; `lost_reason_id` is `ON DELETE SET NULL`). **No `GROUP BY` is emitted, deliberately**: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would have to reach it through a join and would stop being the expression the fixture and the fake both read. - **A hand-edited `lead_name` survives a PATCH that moves its inputs** (`core.lead_name_is_derived`): the name is re-derived only while the stored value still equals what the fallback chain would produce. Answered by recomputing rather than by a `lead_name_is_custom` column — a flag has to be maintained by every writer (importer, sync engine, agent tools) and the one that forgets it silently reverts a typed name. - ⚠️ **The timeline's THIRD source is email, and it is the ONE place in this package scoped to the CALLER rather than to the org** (WS-26d-email, spec §9). Everything else here follows D-CRM-3 — org-visible to every `feature:crm` holder, no owner predicate. Email cannot: the CRM is org-visible while a mailbox belongs to one person, so an unscoped join publishes one member's inbox to the whole company. `activities._timeline(entity, record_id, limit, user)` therefore **requires the caller** and all four routes pass it; a route that drops `user` again would compile, return a timeline, and have no identity left to scope by. The predicate is `activities._email_account_scope`, a **verbatim copy** of `routes/email/core.py::_account_scope` (D-CRM-4, the same call `broker_handlers.broker_gate` made — importing another route package's private helper is the coupling this package declined once already). ⚠️ **Two copies is a coincidence; a THIRD copy anywhere means promote it to a shared module instead.** The fragment hardcodes the alias `em`, so the query aliases `email_messages` as `em` and it drops in unchanged (`email/automation/analytics.py` had to `.replace()` it). Other invariants: the unit is the **thread** (`DISTINCT ON (account_id, COALESCE(thread_id, id::text))` — a row-per-message timeline double-counts every conversation, and grouping on a raw nullable `thread_id` folds every un-threaded message in an account into one entry); addresses resolve **once per record**, not per source, because a deal's set already contains its originating lead's and a per-source pass would return every inherited thread twice; `crm_deals` has no `email` column so a deal joins through `lead_id → crm_leads.email` **and** `crm_deal_contacts → crm_contacts.email`, unioned, with the lead's threads labelled `origin="lead"`; **no addresses means no query at all** (an empty `IN ()` is a syntax error, and the failure a fallback would produce is the whole mailbox on a record that names nobody); inbound `from_address` only in v1, and organizations deliberately do **not** join by domain (an `@fracktal.in` match would attach the entire company mailbox to our own org record). Index: `(account_id, LOWER(from_address->>'email'))` on `email_messages` — the two FTS GINs bury the address inside a `to_tsvector` and are usable only via `@@`. ⚠️ **`tests/unit/test_crm_email_timeline.py` carries a MUTATION FENCE**: deleting the `_email_account_scope(…)` call must turn `test_a_holder_with_no_mailbox_sees_no_email` and `test_two_holders_each_see_only_their_own_account` RED. `_crm_fakes.py` grew four readers so it can see that (a scope subquery, a lowercased JSONB address comparison, a composite LEFT JOIN, and `DISTINCT ON` grouping) — before them the fake did not merely ignore the scope, `_PLAIN_EQ` MISREAD the subquery's own `user_id = :uid`. Do not simplify the SQL to suit the fake; extend the fake. - - ⚠️ **`auto_lead.py` (WS-26d-autolead) — the package's second UNATTENDED writer, and the only one reached from another app's hook.** It registers **no routes** and is therefore NOT imported from `__init__.py` (same reason as `broker_handlers.py`); its one entry point `create_leads_from_new_mail(account_id)` is called from `routes/email/scheduler_hooks.py::process_new_mail`. **It lives here rather than in the email package because what it does is write a CRM record** — it owns `crm_auto_lead_cursors`, it goes through `records.create_record`, and its flag is a CRM owner gate. Unlike the timeline join above it **imports** the automation package's PUBLIC identity primitives (`sender_scope` / `resolve_org_domains` / `normalize_domain`) instead of copying them: D-CRM-4 declined to import another package's *private* helper, and a third copy of "is this person a colleague?" is exactly the drift that rule prevents. Six properties are load-bearing. **(1) The flag is read at the CALL SITE, before the step is entered** — `if auto_lead_enabled(): await create_leads_from_new_mail(...)` — so with `CRM_AUTO_LEAD` off no CRM code runs and no CRM query is issued on the mail path; `auto_lead_enabled` is the flag's ONE definition and a gate moved *inside* the step is pinned red by an AST assertion, not only by a runtime sentinel. **(2) TWO cursor predicates, together.** `process_new_mail` is also reached by ~1-year deep resyncs and by a newly connected mailbox's first sync, and neither stamps `rules_held_back_at`, so "everything classified" would mint a lead per unknown sender across a year of mail — each born `zoho_dirty` and queued for the LIVE tenant within one 600s cycle (D-CRM-9), with no confirmation card on a scheduler hook and no delete tool. `received_at > activated_at` (stamped once, never advanced) is the backfill discriminator; `rules_processed_at > processed_watermark` is the incremental cursor. **(3) Dedup is a SELECT guard plus in-batch de-duplication, never `ON CONFLICT`** — `crm_leads` has no unique constraint on email (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so the ticket's original upsert arm could not have fired. The cross-invocation race is ACCEPTED and recorded; **do not "fix" it with a unique index** (1,516 imported rows, the migration-148 shape). **(4) "External" is necessary, not sufficient** — `sender_scope` fails SAFE to `"external"`, which is the wrong direction when the consequence is a lead row for your own CFO in a live Zoho tenant, so the normalised internal-domain list is a second, independent gate. **(5) The lead goes through `records.create_record`, never raw SQL** (`_resolve_status`, the `owner_email` default, `validate_source` and `mark_dirty_on_insert` all live only there, and only the last is visible in the row afterwards), and `lead_name` is left to `compute_lead_name` over a display name STRIPPED before it is split. **(6) The first activity is `type='system'` — outside `sync_zoho.push_activities`' `type IN ('note','task')` predicate — carrying the subject and the sender in `meta` and an EMPTY body.** The step never selects `body_text` or `snippet`: the projection is the privacy boundary (D-CRM-12 applied to what a machine writes). `tests/unit/test_crm_auto_lead.py` (52 cases) carries a seven-mutant fence over exactly those properties, and `_crm_fakes.py` gained a `@>` containment reader for it — without one the "have we ever emailed them" probe was invisible and the fake answered "yes" for every Sent message. + - ⚠️ **`auto_lead.py` (WS-26d-autolead) — the package's second UNATTENDED writer, and the only one reached from another app's hook.** It registers **no routes** and is therefore NOT imported from `__init__.py` (same reason as `broker_handlers.py`); its one entry point `create_leads_from_new_mail(account_id)` is called from `routes/email/scheduler_hooks.py::process_new_mail`. **It lives here rather than in the email package because what it does is write a CRM record** — it owns `crm_auto_lead_cursors`, it goes through `records.create_record`, and its flag is a CRM owner gate. Unlike the timeline join above it **imports** the automation package's PUBLIC identity primitives (`sender_scope` / `resolve_org_domains` / `normalize_domain`) instead of copying them: D-CRM-4 declined to import another package's *private* helper, and a third copy of "is this person a colleague?" is exactly the drift that rule prevents. Six properties are load-bearing. **(1) The flag is read at the CALL SITE, before the step is entered** — `if auto_lead_enabled(): await create_leads_from_new_mail(...)` — so with `CRM_AUTO_LEAD` off no CRM code runs and no CRM query is issued on the mail path; `auto_lead_enabled` is the flag's ONE definition and a gate moved *inside* the step is pinned red by an AST assertion, not only by a runtime sentinel. **(2) THREE cursor facts, three questions.** `process_new_mail` is also reached by ~1-year deep resyncs and by a newly connected mailbox's first sync, and neither stamps `rules_held_back_at`, so "everything classified" would mint a lead per unknown sender across a year of mail — each born `zoho_dirty` and queued for the LIVE tenant within one 600s cycle (D-CRM-9), with no confirmation card on a scheduler hook and no delete tool. `received_at > activated_at` is the backfill discriminator, `rules_processed_at > processed_watermark` is the incremental cursor, and `last_run_at` is the DORMANCY clock. ⚠️ **The third column is not redundant and `activated_at` is not "set once":** the anchor means *the current ON epoch*, because `activated_at` alone did nothing about a flag turned off for four weeks and back on — the first ON cycle minted the whole OFF window (27 leads, measured). A gap in `last_run_at` beyond `REANCHOR_GAP_SECONDS` (3600) re-stamps all three and mints nothing from the gap, at WARNING on `sync.auto_lead_reanchored`. Dormancy reads `last_run_at` rather than the watermark on purpose: the watermark tracks MAIL, so a mailbox merely quiet over a weekend would be re-anchored and would drop the first message to arrive on Monday — the one the feature exists to catch — and a deliberately-held cursor (below) would be silently re-anchored past. **(2b) A failure never advances the cursor past lost work.** The watermark moves over the contiguous PREFIX that wrote its leads and stops at the first that raised; this step opens a second session per lead through `create_record` while holding the batch's own, so pool exhaustion fails many at once and an unconditional advance stepped over all of them (3 leads lost, measured). A held cursor logs `sync.auto_lead_stalled` at WARNING EVERY cycle, because a held cursor and a quiet mailbox both create nothing and only the level tells them apart. A failed first ACTIVITY is the counter-case — counted separately, never holding the cursor, since the lead is already committed and would be skipped on retry. **(3) Dedup is a SELECT guard plus in-batch de-duplication, never `ON CONFLICT`** — `crm_leads` has no unique constraint on email (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so the ticket's original upsert arm could not have fired. The cross-invocation race is ACCEPTED and recorded; **do not "fix" it with a unique index** (1,516 imported rows, the migration-148 shape). **(4) "External" is necessary, not sufficient** — `sender_scope` fails SAFE to `"external"`, which is the wrong direction when the consequence is a lead row for your own CFO in a live Zoho tenant, so the normalised internal-domain list is a second, independent gate that matches SUBDOMAINS too (`cfo@mail.fracktal.in` is the CFO), anchored on a leading dot so `notfracktal.in` is still a prospect. **(5) The lead goes through `records.create_record`, never raw SQL** (`_resolve_status`, the `owner_email` default, `validate_source` and `mark_dirty_on_insert` all live only there, and only the last is visible in the row afterwards), and `lead_name` is left to `compute_lead_name` over a display name STRIPPED before it is split. **(6) The first activity is `type='system'` — outside `sync_zoho.push_activities`' `type IN ('note','task')` predicate — carrying the subject and the sender in `meta` and an EMPTY body.** The step never selects `body_text` or `snippet`: the projection is the privacy boundary (D-CRM-12 applied to what a machine writes). **(7) The Sent probe folds case, unlike `_maybe_block_cold`** — `@>` is case-EXACT, so a reply from `asha@` after we wrote to `Asha@` minted a lead for somebody mid-conversation; this module uses `EXISTS (… jsonb_array_elements … lower(…) = :addr)` and leaves the email package's predicate alone. **(8) Both attacker-controlled strings are clipped** (`MAX_NAME_CHARS`, `MAX_SUBJECT_CHARS`): the display name becomes `lead_name`, which every list, board card and Zoho push then carries. `tests/unit/test_crm_auto_lead.py` (73 cases) carries a THIRTEEN-mutant fence over exactly those properties, and `_crm_fakes.py` gained two readers plus `fail_on(..., after=N)` for it — without the readers the Sent probe was invisible and the fake answered "yes" for every Sent message, and without the offset a prefix-only cursor and an unconditional one are indistinguishable. 14. routes/admin/ -- Org access control `/admin` API + `/auth/me` (spec: ai-company-brain/specs/org_access_control.md, Phase 1): member roster and lifecycle (invite/suspend/remove — soft, because ~every user-scoped table keys people by email — **plus a separate hard delete**, below), role assignment, custom role CRUD, per-user allow/deny overrides, and the feature catalog the admin UI renders from. `GET /auth/me` is deliberately NOT admin-gated — every signed-in member calls it to resolve their own feature/agent access, and it returns resolved OUTCOMES (allowed feature slugs, runnable agent names) rather than raw permission patterns, so the matching rule has exactly one implementation. `GET /admin/members/{email}/access` returns each decision WITH its provenance (which role granted it, which override took it away) — the admin UI shows that verbatim rather than re-deriving it. Invariants enforced in `_common.py`: the org always keeps an owner, nobody assigns a role above their own rank, system roles are immutable, and **nobody locks themselves out**. That fourth one (`assert_not_self_lockout`, `colleague_onboarding.md` §2 Step 5 / N7+N8) is called by `update_member` (PATCH), `remove_member` (DELETE) **and `purge_member` (DELETE …/purge)** — three doors reach the same `is_active = False`, and while the check lived inside DELETE alone the PATCH had none: `PATCH {"status": "suspended"}` on your own row was refused only by `assert_owner_survives` firing coincidentally in a one-owner org, so a second owner opened it. ⚠️ The rule is **"any status that is not `active`"**, never a list of destructive ones — `EffectiveAccess.is_active` is `status == "active"` exactly, so `invited` is a lockout too, and an enumeration would have to remember it. Comparison is case-insensitive and empty-safe on both sides (an IdP that re-cases a UPN must not switch the guard off; a caller with no identity is not everybody). ⚠️ **It and `assert_owner_survives` both answer 409** — a test that asserts the bare status code cannot tell which fired, and for self-suspension the one that fires today on `main` is the wrong one; discriminate on the detail text and on what was written (`tests/unit/test_admin_member_offboarding.py`). **`purge_member` — `DELETE /admin/members/{email}/purge` (N8)** is the hard delete: a SEPARATE route on the same `admin:members:manage`, never a flag on Remove (which would put the irreversible path one typo from the reversible one). Its decision is **purge the person, keep their work** — the `app_user` row, every access grant (`user_role`, `user_permission_override`, `org_group_member`, `chat_session_participant`, `app_grants`, `app_tool_grants`), every credential (`email_accounts`, `wa_accounts`, `task_accounts`), their PRIVATE `chat_session` rows and their `access_request` row go; what they authored and **the audit trail stay** (an audit trail that disappears with the person is not one — `app_audit` already carries a FK-less `app_id` commented "audit survives hard delete"). ⚠️ **Nothing is anonymised, on purpose**: the address is the join key across ~50 tables, so scrubbing `owner_email` would orphan the apps rather than hide the person. ⚠️ **The three credential rows cascade, and the map is `members._CREDENTIAL_CASCADES`** — `email_accounts` takes the whole mirrored mailbox (**17 direct children, 20 with transitives**), `wa_accounts` the whole WhatsApp mirror (**14 / 16**; `wa_media` hangs off `wa_messages`, NOT off the account), `task_accounts` the SYNCED half of `gtd_items` **and `gtd_projects`**; the credential is `NOT NULL` on the row, so it cannot go without it. That map is hand-maintained and says so, and is pinned against `infra/postgres/` by a test that re-derives it — the first version named 15 of the 20 email tables, which on a route whose safety argument is "the admin is told the blast radius before clicking" is the wrong direction of error. ⚠️ **THREE tables are split across both lists, and each predicate is load-bearing:** `chat_session` by `visibility` (private deleted, shared kept — a room cascades `chat_message` and one person's off-boarding must not take a shared transcript), and `gtd_items` + `gtd_projects` by `account_id` (`IS NOT NULL` = the SYNCED mirror, counted and deleted explicitly; `IS NULL` = the LOCAL rows they authored here, kept). ⚠️ **A KEEP clause must exclude everything the delete side CASCADES away, not merely everything it names.** The `tasks` keep clause originally had no `account_id` predicate, so a member with 847 synced tasks was answered `kept: {"tasks": 847}` while all 847 went with `task_accounts` — the response reported a destruction as a survival. `_PURGE_DELETES`/`_PURGE_KEEPS` derive `count_sql` and `delete_sql` from ONE `where` clause so the count and the delete cannot differ, but that is a within-row-spec guarantee and says nothing about a third statement three entries up; one transaction, one commit, and `record_admin_change` fires BEFORE it (`acb_audit` has its own session, so the record of a destruction survives a rollback of it — though `acb_audit/log.py:49` swallows every exception, so a *completed* purge is NOT guaranteed to leave an audit row). Pinned by `tests/unit/test_admin_member_purge.py` — including the structural assertion that no audit table appears on the delete side at all, the exact permission slug on the route (deleting it leaves the `admin:members:read` floor, which `manager` holds), and the cross-table cascade fences built on `tests/unit/_schema_cascade.py`, which derives the FK graph from the numbered migrations. ⚠️ **`_admin_fakes._FakeDB` models no foreign keys and therefore no cascades**, so every cross-table claim here has to be structural; no behavioural case over a seeded fake can make one. Every write calls `invalidate_access` so a change lands immediately instead of after the resolver's 60s TTL. Tables: infra/postgres/130_org_access_control.sql. Same `_common.py`-is-the-leaf layout as routes/apps and routes/tasks — and here the leaf rule is strict: feature modules import from `_common`, **never from each other**. ⚠️ **The `/admin` auth floor is PER-ROUTE, not a package property.** `_common.py` creates the router with **no** `dependencies=`; every route declares `Depends(require_admin_user)` in its own signature. A route added without it inherits no floor at all and is reachable by any authenticated member — the easiest hole to ship in this package. `access_requests.py` — **sign-in requests** (`colleague_onboarding.md` §6 / N6a, migration 143): `/admin` was push-only, so somebody arriving at the front door produced a journald warning nobody read back (53 of them for one address over 18 hours on 2026-08-03/04, and the owner learned out of band). `acb_auth.access.resolve_access` now upserts an `access_request` row when — and ONLY when — `record_request=True`, which exactly one caller passes; `GET /admin/members/requests` + `POST .../{email}/approve|deny` let the owner answer it, both writes on the EXISTING `admin:members:invite` (no new slug — a new slug is nobody's grant until an admin creates it). **Approve provisions AND activates in one action** (`status='active'`, not `'invited'`) because an approval IS the decision to let somebody in and they are already at the door; leaving them `invited` would re-create the two-click trap §2 Step 1b documents. Both provisioning callers go through `_common.provision_member` — ONE path, so invariants 1 and 2 apply to approvals too (it calls `assert_owner_survives` itself, because `set_roles` REPLACES assignments and provisioning the last owner with the default `member` role would otherwise delete the org's only owner grant). ⚠️ **Both writes hold `admin:members:invite`, which is WEAKER than the `admin:members:manage` that suspends or off-boards, so every path by which the weaker one could reverse the stronger is a cross-gate escalation.** Two independent locks, and each is load-bearing for a different sequence: (1) `_load_request(db, email, *, allowed_statuses=…)` — keyword-only, no default — refuses an already-DECIDED row, because decided rows are kept on purpose (dw9) and the tab renders only `pending`, so a decided row is invisible *and* still addressable; approve takes `("pending",)`, deny takes `("pending", "denied")` since re-denying grants nothing, and denying an *approved* request is refused because it could only make the queue contradict the roster. (2) `_common._PROVISION_MEMBER_SQL`'s `ON CONFLICT` arms **name the statuses they rewrite and never negate**: `invited` → the caller's status (the one door to `active`), `removed` → the caller's status **only when it is not `active`** (so invite still returns an off-boarded person as `invited`, byte-for-byte as before, while approve cannot reinstate them — `removed → active` stays `PATCH /admin/members/{email}`), and `active`/`suspended` are never touched. ⚠️ A `<>`/`NOT IN` test against `app_user.status` is the mutation to watch for: it reads as tidier and silently rewrites rows set under a stronger permission. `tests/unit/test_signin_requests.py` pins the SQL **structurally** (`test_provisioning_only_ever_rewrites_a_status_it_names`) — its fake DB re-implements the `ON CONFLICT` arms in Python and a mirror can only agree with itself, so the behavioural cases there cannot see the statement being widened and must not be trusted to. ⚠️ **Lock (2) declines SILENTLY — it just does not rewrite the row — so it is only half an answer, and the other half is `APPROVE_MATRIX`.** Approve used to run its `_decide(…, "approved")` after that quiet decline: HTTP 200, request marked `approved`, `set_roles` re-granting `['member']` to an off-boarded member, and the person gone for good from a tab that renders only `pending` (the resolver's upsert never rewrites `status`). **`access_requests.APPROVE_MATRIX`, read by `_disposition_for` BEFORE anything is written, is the contract:** absent → provision; `invited` → activate + assign the roles; `active` → do nothing, leave their roles alone, resolve the request as `approved` and say so in `ApproveResult.detail`; `suspended`/`removed` → **409, request stays `pending`** so the person stays visible; anything else → refuse (fail closed). The invariant: **approve never rewrites the roles of a member who already exists in a state other than `invited`** — `provision_member` ends in `set_roles`, which REPLACES assignments, and roles are otherwise `admin:members:manage` territory. The matrix is pinned against `members.VALID_STATUSES`, so a fifth member status cannot ship without somebody deciding what approving one means. `_DECIDE_SQL` also binds the read's own status filter into the UPDATE (`AND status = ANY(:allowed) … RETURNING id`) and 409s on zero rows **before `db.commit()`**, so a lost race discards its own provisioning instead of half-applying it; each route must pass `_decide` the same tuple it passed `_load_request` (a test asserts that from the source). 15. routes/workflows/ -- Workflows app `/workflows` API (spec: ai-company-brain/specs/workflows_app.md; RFC: docs/workflow-editor/README.md): workflow CRUD over the React-Flow-native edit-model (`workflows.graph` jsonb, persisted verbatim), publish → compile to an immutable `workflow_versions.serialized` run-model (edit-model ≠ run-model; runs pin versions), run start/history/detail + a per-run SSE event stream (in-process hub in service.py; runs are supervised asyncio tasks — durable queueing is BO‑20), the served node catalog (agents from the live registry, integrations from acb_skills with availability probe, workflow tool registry, ready modules — the palette is never hard-coded, spec D7), Module Studio (workflow_modules CRUD + conversational generate on acb_llm tier routing + AST validate + subprocess test/run), the inbound webhook trigger `POST /workflows/hooks/{hook_token}` (public by token — in PUBLIC_ROUTES + the router's exempt list; optional HMAC `X-CC-Signature`; rate-limited; fires only published workflows with an enabled webhook trigger), and the cron schedule scanner (scheduler.py — apscheduler CronTrigger parsing inside a supervised asyncio loop with CAS claims on `last_fired_at`; started/stopped from main.py lifespan). The engine subpackage (engine/: templating, graph compile/validate, node handlers over injected NodeServices, MAF WorkflowBuilder runner, module AST validator + restricted subprocess runner) is transport-free — no FastAPI/DB imports — so it is unit-testable alone and movable into the orchestrator if isolation later demands. Agent nodes call `orchestrator.executor.run_agent` (source="workflow", MAF batch path — constraint #9); write-class tool nodes dispatch through `action_broker.propose/submit` (fail closed, constraint #4); module code is import-free/pure-transform only (real sandbox is BO‑7). Capability search (search.py): **keyword-only by explicit owner decision** — deterministic token/substring ranking over the live registries (no index table, no embeddings; an embedding-backed variant was built and deliberately removed in favour of BO‑22, the platform-wide semantic-search service, whose ranking backend will swap in behind the same API shape) — `GET /workflows/catalog/search` serves the palette's search box AND the copilot's shortlist from the same ranking. Workflow Copilot (copilot.py): `POST /workflows/{id}/copilot` — chat-to-build; the LLM emits `{reply, graph, new_modules}`; **missing modules are auto-created** (Module Studio AST validation, saved `ready` with `auto_created` provenance, name→id rewired), the graph is validated with one named-issue repair round against the same validators as publish, and the result is returned for CLIENT-side apply — the copilot never writes the workflow row. `_call_copilot` is the stubbing seam for tests. Tables: infra/postgres/132_workflows.sql. Slice 2: **approval node** — an `approval` node pauses the run (engine returns status `paused`; downstream marked `pending`), `service._hold_for_approval` files a `workflow.resume_run` proposal into the EXISTING Action Broker inbox (`pending_actions` → /approvals UI) with everything a resume needs in the `workflow_run_pauses.snapshot`; approving fires `broker_handlers._resume_run_handler` which replays the run with completed nodes' stored outputs (`precomputed` — no repeated side effects) and the gate resolved; a rejected proposal is reconciled lazily on run read (run → `cancelled`). **Event triggers** — `triggers.dispatch_event` starts runs for published workflows whose `kind='event'` binding matches `(source, event_type)` (empty type = all); fed by BOTH `/agent/webhook/{source}` (routes/agent.py calls it after agent routing; response carries `workflow_runs`) and the native ClickUp receiver via `ingestion.event_hooks` (a `post_sync.py`-style sink registry — ingestion never imports upward; main.py registers the dispatcher at startup). Same core-is-the-leaf layout as routes/tasks; ⚠️ `__init__.py` import order is load-bearing (static paths before crud's `/{workflow_id}`; a regression test pins it). Startup: main.py lifespan calls `service.reconcile_orphaned_runs()` BEFORE starting the scheduler — rows still `running` belong to a dead process and are swept to `failed` ("interrupted by a platform restart"); `paused` rows are deliberately untouched (resume rebuilds everything from the pause snapshot), and `runs.py` keeps the per-read lazy patch for reads that race the sweep. Run-history drill-in (spec F9): clicking a history row in the editor's RunConsole fetches the run detail and paints its recorded `node_results` onto the canvas (cleared when a live test run starts). Engine semantics are locked by a CI-blocking golden trajectory eval — `evals/trajectories/test_workflow_engine_trajectory.py`; `skill-eval.yml` triggers on `routes/workflows/**` so engine edits re-run the gate. **Publish authority** (spec Q3, migration 133): `POST /{id}/publish`, `/versions/{v}/rollback`, and `/disable` require the `workflows:publish` capability on top of the router's `feature:workflows` gate — they are the acts that ARM triggers to run unattended. Drafting, validate, Test runs, duplicate, and the copilot stay open to the feature (a draft fires no triggers and its writes are still broker-held). `/auth/me` returns a resolved `capabilities` list so the editor can grey out Publish with a reason instead of a bare 403 — the browser must never re-derive wildcard matching (`permissions` holds raw patterns; an owner has `*`). **Wait node** (F3 logic vocabulary): `{"seconds": N}`, ≤`WAIT_INLINE_MAX_SECONDS` (60) sleeps inline inside the run; longer pauses the run exactly like an approval but with `reason='wait'` + a `resume_at` deadline in the pause snapshot and NO broker proposal (nobody decides anything) — `scheduler.scan_due_waits()` runs in the same loop as cron triggers and hands matured pauses to the SAME `service.resume_run`, which routes by pause reason (`elapsed_waits` vs `resolved_approvals`, so an elapsed wait can never clear an approval downstream). A resumed wait must never sleep again: the handler only sleeps when the duration is inline-short. Lifecycle extras: `POST /{id}/duplicate` (crud.py — copies graph/variables/triggers into a fresh DRAFT; the hook token is ALWAYS regenerated, it is a credential) and `POST /{id}/versions/{v}/rollback` (publish.py — republishes version v's immutable snapshot as a NEW version; deliberately does not re-validate as a gate since rollback is incident response — catalog drift comes back as non-blocking `warnings`, and the draft edit-model is never clobbered). **Automation health** (spec R2, migration 134): every terminal run calls `service.evaluate_automation_health()`, which disables a published workflow after `AUTO_DISABLE_AFTER` (5) consecutive failures **from `UNATTENDED_TRIGGERS` only** (`schedule`/`webhook`/`event` — a maker's Test runs and agent `api` calls must never disable production). The streak is derived from `workflow_runs`, never a counter column, and is scoped to runs after `workflows.health_since`, which publish/rollback/enable each re-stamp — without that window a re-enabled workflow would re-disable on its next failure, since the failures that tripped the policy are still the newest rows. The disable is a CAS on `status='published'` so concurrent failing runs produce exactly one disable; `disabled_reason`/`disabled_at` are written the same way for the human Disable path, so the gallery answers "why is this off?" identically. Notification is in-product (persisted reason → gallery badge + editor banner, `workflows.auto_disabled` log, activity-feed `disabled` event); outward notification would be an outward write and belongs on the broker path. `POST /{id}/enable` (publish.py, same `workflows:publish` gate) is the way back: it re-arms the EXISTING live version rather than minting one, 409s if the workflow was never published, and is idempotent when already live. `_execute_run`'s `trigger_kind` is a REQUIRED keyword — a dropped kwarg would make the whole policy silently inert. **Trigger durability** (spec §3.3a): schedules are DB rows, not OS cron and not an APScheduler process — `CronTrigger` is a parser only. ⚠️ `compute_due_fire` only looks FORWARD, so a trigger with `last_fired_at IS NULL` yields no tick; `_claim_baseline` arms it on first sight instead of firing (a cron says *when*, not *how far back*). Without that step a new schedule never fires **at all** — it produced no tick, so it never got a baseline, so it produced no tick. `config.timezone` is an IANA wall clock (default UTC) validated at save with the cron, so a 9am job stays 9am across DST; the zone is passed to `CronTrigger.from_crontab`, and instants stay UTC-aware throughout. `update_workflow` rewrites trigger rows wholesale but CARRIES `last_fired_at` across for unchanged schedules (`_trigger_identity` = kind + cron@timezone) — otherwise every canvas save re-armed the cron and lost the already-fired-this-tick guarantee. Because the CAS claim commits BEFORE `start_run`, a claimed tick can never be re-offered: every path out of that block calls `service.record_skipped_run()`, which writes a terminal `cancelled` run row (cancelled, not failed — being busy must not feed the R2 auto-disable policy). **Hook URL**: `core.hook_url()` builds it from `settings.public_api_base_url` and `get_workflow` returns `hook_url`/`hook_path`; the browser must NEVER assemble one from `window.location`, because the control-plane `/api` proxy re-serializes JSON (breaking sender HMAC) and drops non-JSON bodies. The Next route `api/workflows/hooks/[token]/route.ts` is a raw-bytes passthrough that attaches no internal bearer. **Typed tool arguments** (`engine/tool_args.py` — n8n's typed-node-parameters pattern, Sim's `subBlocks`): a tool's `args_schema` value is a mini-language `type[?][|description]` over the closed set `{string,number,boolean,object,array}`; an unknown type degrades to `string` rather than raising (one bad declaration must not take the whole catalog down). It is parsed in ONE place and consumed in three — the catalog serves `args[]` (parsed) so the browser never re-implements the grammar, `validate_graph(tool_schemas=…)` blocks publish on a missing/unknown/mistyped argument (`tool_args` issue code), and `execute_tool` re-checks at run time because a draft Test, a copilot graph, or an older published version can all reach a handler that publish never saw. `{{refs}}` satisfy required checks and are exempt from type checks on both sides — they resolve at run time. Type checking is deliberately lenient (only container-vs-scalar category errors) so the messages that fire are worth reading. `tests/unit/test_workflows_tool_contract.py` holds each declaration to its handler by AST-scanning for `args.get("x")`/`args["x"]` — the drift it hunts is a handler growing an input the schema never declares (`_broker_write`'s `target_field` is dynamic, so it has its own explicit test). **Golden workflow fixtures** (`evals/trajectories/workflows/*.json` + `test_workflow_fixtures.py`): whole workflows paired with an expected outcome, one generic runner; `expect.publishable: false` fixtures pin the publish gates. Tool schemas and destructive actions come from the REAL registry so fixtures break when the shipped catalog changes; fixtures assert which seams were crossed (`agent_calls`/`tool_calls`/`tool_args`), because "succeeded" while silently never calling the integration is the failure mode they exist to catch. 16. agents.json -- Dynamic agent registry (persisted alongside pyproject.toml) diff --git a/apps/services/gateway/gateway/routes/crm/auto_lead.py b/apps/services/gateway/gateway/routes/crm/auto_lead.py index 6d7e58d35..cc445ca99 100644 --- a/apps/services/gateway/gateway/routes/crm/auto_lead.py +++ b/apps/services/gateway/gateway/routes/crm/auto_lead.py @@ -20,25 +20,37 @@ the automation package. A third copy of it here would be the drift the rule exists to prevent. -Four properties are load-bearing, and each one is a way this feature can do +Five properties are load-bearing, and each one is a way this feature can do real damage rather than merely be wrong: 1. **The flag is read BEFORE the step is entered.** ``CRM_AUTO_LEAD`` ships OFF and :func:`auto_lead_enabled` is its single definition, but the *call site* - is what is guarded — with the flag off nothing here runs and no CRM query is - issued on the mail path at all. A short-circuit *inside* this module would - satisfy a careless test and still open a database session on every sync - cycle of every mailbox. - -2. **Two cursor predicates, together.** ``process_new_mail`` is also reached by - deep resyncs and by the first-ever sync of a newly connected mailbox, and - neither marks the mail it classifies as history. ``received_at > - activated_at`` is therefore the backfill discriminator (mail that ARRIVED - before auto-lead was first active on the account mints nothing, whenever it - is classified), and ``rules_processed_at > processed_watermark`` is the - incremental cursor. Without the first, connecting a second mailbox mints a - lead per unknown sender in a year of mail — each born ``zoho_dirty`` and - queued for the live Zoho tenant within one 600s cycle. + is what is guarded — with the flag off this module opens no session, issues + no query and creates nothing, and ``process_new_mail`` does not even import + :func:`create_leads_from_new_mail`. (It does import *this module* to read + the predicate; that is one ``sys.modules`` lookup after boot, because + ``routes/crm`` is mounted by ``main.py`` regardless. One definition of what + the flag means is worth more than saving it — two places that must agree + about a flag is how a loop runs with the flag off.) + +2. **THREE cursor facts, and they answer three different questions.** + ``process_new_mail`` is also reached by deep resyncs and by the first-ever + sync of a newly connected mailbox, and neither marks the mail it classifies + as history. + + * ``activated_at`` — the start of the CURRENT ON epoch. + ``received_at > activated_at`` is the backfill discriminator: mail that + ARRIVED before this epoch began mints nothing, whenever it is classified. + * ``processed_watermark`` — the incremental cursor over + ``rules_processed_at``. + * ``last_run_at`` — when this step last RAN. It is what detects dormancy, + and it is a separate column because the watermark cannot answer that + question: a quiet mailbox has a watermark hours old while the step has + been running faithfully every cycle. See :func:`_reanchor_if_dormant`. + + Without the first, connecting a second mailbox mints a lead per unknown + sender in a year of mail — each born ``zoho_dirty`` and queued for the live + Zoho tenant within one 600s cycle. 3. **"Unknown" is three questions, and "external" is only necessary.** :func:`_is_unknown_sender` mirrors ``senders._maybe_block_cold``'s two steps @@ -47,7 +59,8 @@ Separately, :func:`_is_external_sender` runs TWO gates, because ``sender_scope`` fails SAFE to ``"external"`` — the wrong direction here. A lead row for your own CFO, pushed into the live Zoho tenant, is what the - second gate exists to prevent. + second gate exists to prevent, and it is SUFFIX-aware: a colleague on + ``mail.fracktal.in`` is a colleague. 4. **The first activity is metadata, never content** (D-CRM-12 applied to what a machine writes). ``type='system'`` — deliberately outside the Zoho push @@ -57,6 +70,15 @@ body is not, and no snippet of it is either. This module never reads ``body_text`` or ``snippet``. +5. **The watermark advances over the successful PREFIX only.** A failure is not + a reason to skip work: the step opens a second session per lead (through + ``create_record``) while holding the batch's own, so pool exhaustion fails + many candidates at once, and an unconditional advance would step over every + one of them permanently. On the first lead-write failure the cursor stops + moving and the cycle logs ``sync.auto_lead_stalled`` at WARNING every cycle + until a human looks. That trades silent loss for a visible stall on a poison + head message, deliberately: fail closed toward the CRM. + **One race is accepted and recorded** (spec §9): two concurrent ``process_new_mail`` invocations for one account can read the same watermark and double-mint. The cost is one visible, hand-deletable duplicate lead. The @@ -69,6 +91,7 @@ from __future__ import annotations import json +from datetime import UTC, datetime from typing import Any from acb_auth import UserContext, UserRole @@ -100,6 +123,18 @@ #: reads as "covered everything". MAX_CANDIDATES_PER_CYCLE = 200 +#: How long a gap in this step's own RUNS means the ON epoch ended. +#: Six scheduler periods at the 600s sync interval, so an ordinary slow cycle, +#: a restart or a single missed poll can never trip it. +#: +#: ⚠️ This is the OFF→ON guard and it is not optional. ``activated_at`` alone +#: stops a deep resync, but it does nothing about a flag that was on, turned +#: off for four weeks, and turned back on: the cursor still carries day-1's +#: anchor, so the first ON cycle would mint the entire OFF window in one batch +#: — measured at 27 leads for a 27-day window, each pushing unattended into the +#: live tenant. Re-anchoring makes the anchor mean "the current ON epoch". +REANCHOR_GAP_SECONDS = 3600 + #: The activity ``type`` the originating message is logged as. **Not 'note'.** #: ``sync_zoho.push_activities`` pushes ``type IN ('note', 'task')`` only, so #: 'system' is how this row stays inside the native CRM — the mail's subject @@ -118,6 +153,17 @@ INBOX_FOLDER = "inbox" SENT_FOLDER = "sent" +#: Ceilings on the two attacker-controlled strings that reach TEXT columns — +#: the sender's display name (which becomes ``first_name``/``last_name`` and +#: therefore ``lead_name``) and the subject line. Nothing upstream bounds +#: either: a display name is whatever the sending server put in the header, and +#: it lands in a column every CRM list, board card and Zoho push then carries. +MAX_NAME_CHARS = 120 +MAX_SUBJECT_CHARS = 500 +#: Appended when either ceiling bites, so a clipped value reads as clipped +#: rather than as the sender's actual name. +CLIP_MARKER = "…" + #: The candidate predicate, written ONCE and shared by the fetch and the #: overflow count. Two copies would let the count answer a different question #: from the batch — and the count is the number an operator reads to decide @@ -134,15 +180,38 @@ #: ⚠️ The projection is the privacy boundary: ``body_text`` and ``snippet`` are #: absent on purpose and must stay absent. Nothing downstream can leak a body #: it was never handed. +#: +#: ``ORDER BY rules_processed_at, id`` — the ``id`` is a TIEBREAK, and without +#: it the ordering of rows sharing a timestamp is undefined, so the cap could +#: cut a tie group in a different place than the watermark assumes and lose the +#: rows in between. See :func:`_drop_boundary_group`. _CANDIDATE_SQL = ( "SELECT id, subject, from_address, received_at, thread_id, " "internet_message_id, rules_processed_at " f"FROM email_messages WHERE {_CANDIDATE_WHERE} " - "ORDER BY rules_processed_at LIMIT :limit" + "ORDER BY rules_processed_at, id LIMIT :limit" ) _CANDIDATE_COUNT_SQL = f"SELECT COUNT(*) FROM email_messages WHERE {_CANDIDATE_WHERE}" +#: "Have we ever emailed them" — ``_maybe_block_cold``'s second step, with the +#: one difference this module needs. +#: +#: ⚠️ That helper asks it with jsonb containment (``to_addresses @> :json``), +#: which Postgres evaluates EXACTLY: an owner who wrote to +#: ``Asha@AcmeRobotics.com`` has not, as far as ``@>`` is concerned, emailed +#: ``asha@acmerobotics.com`` — and she replies in lower case, so the reply +#: minted a lead for somebody already in the middle of a conversation. Here the +#: probe folds case explicitly. ``_maybe_block_cold`` is left alone: it is the +#: email package's predicate and its blast radius is the cold-email blocker, +#: not this. +_EVER_EMAILED_SQL = ( + "SELECT 1 FROM email_messages " + "WHERE account_id = :account_id AND LOWER(folder) = :folder " + "AND EXISTS (SELECT 1 FROM jsonb_array_elements(to_addresses) recipient " + "WHERE lower(recipient->>'email') = :address) LIMIT 1" +) + def auto_lead_enabled() -> bool: """``CRM_AUTO_LEAD`` — the single definition of what the flag means. @@ -164,12 +233,15 @@ def _new_stats() -> dict[str, int]: return { "candidates": 0, "overflow": 0, + "boundary_deferred": 0, "created": 0, "skipped_internal": 0, "skipped_known": 0, "skipped_unusable": 0, "deduped_in_batch": 0, "errors": 0, + "activity_errors": 0, + "reanchored": 0, } @@ -190,33 +262,73 @@ async def create_leads_from_new_mail(account_id: str) -> dict[str, int]: cursor = await _load_or_activate_cursor(db, account_id) if cursor is None: # pragma: no cover — the row was just written return stats - candidates = await _load_candidates(db, account_id, cursor) - stats["candidates"] = len(candidates) - if not candidates: + if await _reanchor_if_dormant(db, account_id, cursor): + stats["reanchored"] = 1 _emit(account_id, stats) return stats - stats["overflow"] = await _count_overflow(db, account_id, cursor, - len(candidates)) - domains = await _internal_domains(db, account_id, account.email_address) - seen: set[str] = set() - for message in candidates: - await _consider(db, account, message, domains, seen, stats) - # Advanced over everything CONSIDERED, including the messages that - # minted nothing and the ones that raised: this is a best-effort - # enrichment step, not a queue, and a poison message that held the - # cursor still would re-fail on every cycle forever. The errors are - # counted in the line below instead. - await _advance_watermark( - db, account_id, - max(message.rules_processed_at for message in candidates), - ) - await db.commit() + await _run_batch(db, account, cursor, stats) _emit(account_id, stats) return stats finally: await db.close() +async def _run_batch( + db: Any, account: Any, cursor: Any, stats: dict[str, int], +) -> None: + """One cycle's candidates, considered in order, cursor stamped once.""" + account_id = str(account.id) + candidates, next_stamp = await _load_candidates(db, account_id, cursor) + capped = next_stamp is not None + if capped and candidates[-1].rules_processed_at == next_stamp: + # …and ONLY then. A cap that fell between two timestamp groups cut + # nothing in half, so deferring its last group would shrink the batch + # by one message every single cycle for no reason. + candidates = _drop_boundary_group(candidates, stats) + stats["candidates"] = len(candidates) + if not candidates: + # A cap that dropped a whole batch into the boundary group is a stall, + # not a quiet cycle, and the two must not read the same in the log. + if stats["boundary_deferred"]: + _log.warning("sync.auto_lead_stalled", account_id=account_id, + reason="boundary_group_fills_the_cap", + boundary_deferred=stats["boundary_deferred"]) + await _stamp_cursor(db, account_id, run_at=now()) + await db.commit() + return + if capped: + stats["overflow"] = await _count_overflow( + db, account_id, cursor, len(candidates), + ) + domains = await _internal_domains(db, account_id, account.email_address) + seen: set[str] = set() + + #: The stamp of the last message in the contiguous prefix that did NOT + #: fail to write its lead. Everything after the first failure is + #: deliberately left for the next cycle: re-considering a message whose + #: lead already exists costs one indexed SELECT (step 3 of + #: `_is_unknown_sender` finds it and skips), and that is far cheaper than + #: the alternative, which is stepping the cursor over work that was lost. + advance_to: Any = None + blocked = False + for message in candidates: + failed = await _consider(db, account, message, domains, seen, stats) + if failed: + blocked = True + elif not blocked: + advance_to = message.rules_processed_at + + await _stamp_cursor(db, account_id, watermark=advance_to, run_at=now()) + await db.commit() + if advance_to is None and stats["errors"]: + # WARNING, and it repeats every cycle: an INFO line saying "created 0" + # is what a stuck head message looked like before, which is to say it + # looked like a quiet mailbox. + _log.warning("sync.auto_lead_stalled", account_id=account_id, + reason="lead_write_failed_on_the_first_candidate", + candidates=stats["candidates"], errors=stats["errors"]) + + # ── The account, and the cursor that decides what is history ──────────────── async def _load_account(db: Any, account_id: str) -> Any | None: @@ -238,7 +350,7 @@ async def _load_account(db: Any, account_id: str) -> Any | None: async def _read_cursor(db: Any, account_id: str) -> Any | None: return (await db.execute(text( - "SELECT account_id, activated_at, processed_watermark " + "SELECT account_id, activated_at, processed_watermark, last_run_at " "FROM crm_auto_lead_cursors WHERE account_id = :account_id" ), {"account_id": account_id})).fetchone() @@ -246,12 +358,12 @@ async def _read_cursor(db: Any, account_id: str) -> Any | None: async def _load_or_activate_cursor(db: Any, account_id: str) -> Any | None: """Read the account's cursor, creating it on the first ON-state run. - Activation stamps ``activated_at`` and ``processed_watermark`` to the SAME - instant, so the activating cycle itself mints nothing: everything already - in the mailbox arrived before auto-lead existed for this account, which is - precisely the deep-resync case. ``ON CONFLICT DO NOTHING`` + a re-read - means a concurrent activation is one row and one activation instant, not a - primary-key error on the mail path. + Activation stamps all three timestamps to the SAME instant, so the + activating cycle itself mints nothing: everything already in the mailbox + arrived before auto-lead existed for this account, which is precisely the + deep-resync case. ``ON CONFLICT DO NOTHING`` + a re-read means a concurrent + activation is one row and one activation instant, not a primary-key error + on the mail path. """ row = await _read_cursor(db, account_id) if row is not None: @@ -259,15 +371,95 @@ async def _load_or_activate_cursor(db: Any, account_id: str) -> Any | None: at = now() await db.execute(text( "INSERT INTO crm_auto_lead_cursors " - "(account_id, activated_at, processed_watermark) " - "VALUES (:account_id, :activated_at, :processed_watermark) " + "(account_id, activated_at, processed_watermark, last_run_at) " + "VALUES (:account_id, :activated_at, :processed_watermark, :last_run_at) " "ON CONFLICT (account_id) DO NOTHING" - ), {"account_id": account_id, "activated_at": at, "processed_watermark": at}) + ), {"account_id": account_id, "activated_at": at, + "processed_watermark": at, "last_run_at": at}) await db.commit() _log.info("crm.auto_lead_activated", account_id=account_id) return await _read_cursor(db, account_id) +def _aware(value: Any) -> datetime: + """A stored instant as tz-aware UTC. + + ``TIMESTAMPTZ`` comes back aware from asyncpg; this exists so that a naive + value from any other source cannot raise mid-cycle on the subtraction and + take the whole step down through the hook's ``except``. + """ + if not isinstance(value, datetime): + return now() + return value if value.tzinfo is not None else value.replace(tzinfo=UTC) + + +async def _reanchor_if_dormant(db: Any, account_id: str, cursor: Any) -> bool: + """Did this step stop running? Then the ON epoch ended; start a new one. + + ``activated_at`` guards a deep RESYNC. It does nothing about a flag that + was on, turned off for four weeks and turned back on — the cursor still + carries the old anchor, so the first ON cycle mints the whole OFF window + at once, unattended, into a live tenant. Re-anchoring is what makes + ``activated_at`` mean *the current ON epoch* rather than *the first time + anyone ever enabled this*. + + ⚠️ **Dormancy is measured on ``last_run_at``, not on + ``processed_watermark``** — a deliberate departure from the shape first + prescribed, for two reasons that both bite in production: + + * the watermark tracks MAIL, not runs. A mailbox that is simply quiet over + a weekend has a watermark 60 hours old while this step has run faithfully + every 600s, so a watermark-based test re-anchors it and the first message + to arrive on Monday — the one this whole feature exists to catch — falls + before the new anchor and mints nothing. Every Monday. + * a genuinely poison head message holds the watermark still ON PURPOSE + (property 5). A watermark-based test would then re-anchor after an hour + and skip the very backlog the stall was protecting, quietly undoing the + stall it was supposed to make visible. + + Both directions fail closed: an OFF window and a real outage each skip + their backlog. A missed lead is hand-creatable and visible in the mailbox; + 27 unattended pushes into the live Zoho tenant are neither. + """ + gap = (now() - _aware(cursor.last_run_at)).total_seconds() + if gap <= REANCHOR_GAP_SECONDS: + return False + at = now() + await db.execute(text( + "UPDATE crm_auto_lead_cursors " + "SET activated_at = :activated_at, " + "processed_watermark = :processed_watermark, " + "last_run_at = :last_run_at, updated_at = now() " + "WHERE account_id = :account_id" + ), {"account_id": account_id, "activated_at": at, + "processed_watermark": at, "last_run_at": at}) + await db.commit() + _log.warning("sync.auto_lead_reanchored", account_id=account_id, + gap_seconds=int(gap), threshold_seconds=REANCHOR_GAP_SECONDS) + return True + + +async def _stamp_cursor( + db: Any, account_id: str, *, run_at: Any, watermark: Any = None, +) -> None: + """Record that the step ran, and how far it got. + + ``last_run_at`` moves on EVERY cycle — that is what stops a quiet mailbox + looking dormant. ``processed_watermark`` moves only when there is a + successful prefix to move it over, which is what stops a failure being + treated as work done. + """ + assignments = ["last_run_at = :last_run_at", "updated_at = now()"] + params: dict[str, Any] = {"account_id": account_id, "last_run_at": run_at} + if watermark is not None: + assignments.insert(0, "processed_watermark = :processed_watermark") + params["processed_watermark"] = watermark + await db.execute(text( + f"UPDATE crm_auto_lead_cursors SET {', '.join(assignments)} " + "WHERE account_id = :account_id" + ), params) + + def _cursor_params(account_id: str, cursor: Any) -> dict[str, Any]: return { "account_id": account_id, @@ -277,17 +469,54 @@ def _cursor_params(account_id: str, cursor: Any) -> dict[str, Any]: } -async def _load_candidates(db: Any, account_id: str, cursor: Any) -> list[Any]: +async def _load_candidates( + db: Any, account_id: str, cursor: Any, +) -> tuple[list[Any], Any]: """Classified inbox mail this account has not been considered for yet. Both cursor predicates apply. Ordered by ``rules_processed_at`` ascending so the batch's maximum IS the new watermark — ordering by ``received_at`` would advance the cursor past messages the cap left behind. + + Fetches one MORE than the cap and returns that extra row's stamp as the + second element (``None`` when the batch was short). Peeking one row is what + turns "there is more" from an inference off a full page into a fact, and it + is also the only way to know whether the cap fell *inside* a group of rows + sharing a timestamp — see the caller. """ - return (await db.execute( + rows = list((await db.execute( text(_CANDIDATE_SQL), - {**_cursor_params(account_id, cursor), "limit": MAX_CANDIDATES_PER_CYCLE}, - )).fetchall() + {**_cursor_params(account_id, cursor), + "limit": MAX_CANDIDATES_PER_CYCLE + 1}, + )).fetchall()) + if len(rows) > MAX_CANDIDATES_PER_CYCLE: + return (rows[:MAX_CANDIDATES_PER_CYCLE], + rows[MAX_CANDIDATES_PER_CYCLE].rules_processed_at) + return rows, None + + +def _drop_boundary_group( + candidates: list[Any], stats: dict[str, int], +) -> list[Any]: + """Drop the trailing rows that share the cap's last timestamp. + + The watermark is a timestamp, so advancing it to a stamp the cap cut + *through* would step over the rest of that group forever. Dropping the + whole boundary group means the next cycle sees it complete; the rows come + back deduped, so re-considering them is free. + + ⚠️ Theoretically this can empty the batch — if every row in a full cap + shares one stamp, nothing is left and the cursor cannot move. That is + reported as a stall rather than as a quiet cycle. It cannot happen in + production: ``rules_processed_at`` is stamped per message inside the rules + runner's own loop, one transaction per message, so 200 messages carrying a + single identical instant would require 200 messages to be stamped by one + statement — which no writer of that column does. + """ + boundary = candidates[-1].rules_processed_at + kept = [row for row in candidates if row.rules_processed_at != boundary] + stats["boundary_deferred"] = len(candidates) - len(kept) + return kept async def _count_overflow( @@ -295,26 +524,16 @@ async def _count_overflow( ) -> int: """How many candidates the cap left for the next cycle. - Only asked when the batch came back full — a short batch is the whole - remainder by definition, and a COUNT on every quiet cycle is a scan + Only asked when the fetch came back over the cap — a short batch is the + whole remainder by definition, and a COUNT on every quiet cycle is a scan nobody reads. """ - if taken < MAX_CANDIDATES_PER_CYCLE: - return 0 total = (await db.execute( text(_CANDIDATE_COUNT_SQL), _cursor_params(account_id, cursor), )).scalar() return max(0, int(total or 0) - taken) -async def _advance_watermark(db: Any, account_id: str, watermark: Any) -> None: - await db.execute(text( - "UPDATE crm_auto_lead_cursors " - "SET processed_watermark = :processed_watermark, updated_at = now() " - "WHERE account_id = :account_id" - ), {"account_id": account_id, "processed_watermark": watermark}) - - # ── Who the sender is ─────────────────────────────────────────────────────── async def _internal_domains( @@ -334,6 +553,21 @@ async def _internal_domains( return frozenset(domain for domain in domains if domain) +def _is_internal_domain(domain: str, internal_domains: frozenset[str]) -> bool: + """Is this domain ours, or a SUBDOMAIN of one of ours? + + Exact matching alone let ``cfo@mail.fracktal.in`` through while + ``cfo@fracktal.in`` was caught — and mail from a company's own + ``mail.``/``corp.``/regional subdomains is routine. The suffix test is + anchored on a leading dot so ``notfracktal.in`` is not a subdomain of + ``fracktal.in``, which is the mistake a bare ``endswith`` makes. + """ + return any( + domain == internal or domain.endswith(f".{internal}") + for internal in internal_domains + ) + + def _is_external_sender( address: str, account_address: str | None, internal_domains: frozenset[str], ) -> bool: @@ -345,12 +579,11 @@ def _is_external_sender( **Gate 2** is the configured internal-domain list. It is not a restatement of gate 1: ``sender_scope`` fails SAFE to ``"external"`` on an unparseable address (the wrong direction when the consequence is a lead row in a live - Zoho tenant), and the extra ``org_domains`` reach it through a different - normalisation than ``resolve_org_domains`` applies — so a colleague on the - company's SECOND domain, configured by somebody who pasted an address - rather than a bare host, is external to gate 1 and internal to gate 2. - Routing the configured list through gate 2 only means there is exactly one - normalisation of it here rather than two that can disagree. + Zoho tenant), the extra ``org_domains`` reach it through a different + normalisation than ``resolve_org_domains`` applies, and it matches domains + exactly where this one matches subdomains too. Routing the configured list + through gate 2 only means there is exactly one normalisation of it here + rather than two that can disagree. An address with no domain at all fails gate 2: we do not mint a lead from something we could not parse. @@ -358,7 +591,7 @@ def _is_external_sender( if sender_scope(address, account_address or "") != "external": return False domain = normalize_domain(address) - return bool(domain) and domain not in internal_domains + return bool(domain) and not _is_internal_domain(domain, internal_domains) async def _is_unknown_sender(db: Any, account_id: str, address: str) -> bool: @@ -367,12 +600,13 @@ async def _is_unknown_sender(db: Any, account_id: str, address: str) -> bool: 1. the cold-sender memo: this account has already decided something about this address (flagged cold, or whitelisted). Either way it is not a stranger who just wrote in. - 2. have we ever emailed them. ⚠️ ``@>`` is exact, as Postgres is, so a - recipient stored with different casing escapes this probe — which is - part of why it is not the only step. + 2. have we ever emailed them — case-insensitively, unlike the containment + form the cold blocker uses. See :data:`_EVER_EMAILED_SQL`. 3. ours: no CRM row already carries the address. A contact or a lead means the company already knows this person, and a second lead for them is - the duplicate a human then has to merge. + the duplicate a human then has to merge. This step is also what makes + re-considering a message free, which is what lets the watermark hold + still after a failure without costing anything. """ memo = (await db.execute(text( "SELECT status FROM email_cold_senders " @@ -380,13 +614,8 @@ async def _is_unknown_sender(db: Any, account_id: str, address: str) -> bool: ), {"account_id": account_id, "address": address})).fetchone() if memo: return False - replied = (await db.execute(text( - "SELECT 1 FROM email_messages " - "WHERE account_id = :account_id AND LOWER(folder) = :folder " - "AND to_addresses @> :recipient LIMIT 1" - ), { - "account_id": account_id, "folder": SENT_FOLDER, - "recipient": json.dumps([{"email": address}]), + replied = (await db.execute(text(_EVER_EMAILED_SQL), { + "account_id": account_id, "folder": SENT_FOLDER, "address": address, })).fetchone() if replied: return False @@ -401,12 +630,23 @@ async def _is_unknown_sender(db: Any, account_id: str, address: str) -> bool: # ── One candidate ─────────────────────────────────────────────────────────── +def _clip(value: str | None, limit: int) -> str | None: + """Bound one attacker-controlled string, marking it when it bites.""" + if value is None: + return None + text_value = str(value) + if len(text_value) <= limit: + return text_value + return text_value[: limit - len(CLIP_MARKER)] + CLIP_MARKER + + def _sender(message: Any) -> tuple[str, str]: """``(address, display name)`` off the message's ``from_address`` JSONB. The driver hands back a dict or the raw text depending on how the row was read; both shapes appear in this codebase, so both are handled here rather - than at four call sites. + than at four call sites. The display name is CLIPPED here, once, so every + consumer of it downstream is bounded by construction. """ raw = getattr(message, "from_address", None) if not isinstance(raw, dict): @@ -418,7 +658,7 @@ def _sender(message: Any) -> tuple[str, str]: raw = {} return ( str(raw.get("email") or "").strip().lower(), - str(raw.get("name") or "").strip(), + _clip(str(raw.get("name") or "").strip(), MAX_NAME_CHARS) or "", ) @@ -463,55 +703,77 @@ def _principal(owner_email: str) -> UserContext: async def _consider( db: Any, account: Any, message: Any, internal_domains: frozenset[str], seen: set[str], stats: dict[str, int], -) -> None: - """Decide about ONE candidate message, and count the decision.""" +) -> bool: + """Decide about ONE candidate message, and count the decision. + + Returns True when the LEAD write failed — the only outcome that must hold + the watermark. A failure to write the lead's first activity does not: the + lead is already committed, so re-considering the message next cycle finds + it at step 3 and skips, meaning the activity would never be retried and + the cursor would stall forever on work that cannot be redone. + """ address, display_name = _sender(message) if not address: stats["skipped_unusable"] += 1 - return + return False if address in seen: # In-batch de-duplication. Not an optimisation: the SELECT guard in # `_is_unknown_sender` reads a row `create_record` has committed on # ANOTHER session, so without this a sender who wrote twice in one # batch is a coin flip between one lead and two. stats["deduped_in_batch"] += 1 - return + return False seen.add(address) if not _is_external_sender(address, account.email_address, internal_domains): stats["skipped_internal"] += 1 - return + return False if not await _is_unknown_sender(db, str(account.id), address): stats["skipped_known"] += 1 - return + return False try: - async with savepoint(db): - await _mint_lead(db, account, message, address, display_name) + lead = await _create_lead(account, address, display_name) except Exception as exc: stats["errors"] += 1 _log.warning("crm.auto_lead_message_failed", account_id=str(account.id), message_id=str(getattr(message, "id", "")), error=str(exc)[:200]) - return + return True + # Counted the moment the lead is COMMITTED, not after its activity: the + # row exists and will push to Zoho either way, and a log line reading + # `created=0` beside a lead that is already queued is the report that + # sends somebody looking in the wrong place. stats["created"] += 1 + try: + async with savepoint(db): + await _log_origin_activity( + db, lead, account, message, address, display_name, + ) + except Exception as exc: + stats["activity_errors"] += 1 + _log.warning("crm.auto_lead_activity_failed", + account_id=str(account.id), + lead_id=str(lead.get("id", "")), + message_id=str(getattr(message, "id", "")), + error=str(exc)[:200]) + return False -async def _mint_lead( - db: Any, account: Any, message: Any, address: str, display_name: str, -) -> None: - """Create the lead, then log the originating message against it. +async def _create_lead( + account: Any, address: str, display_name: str, +) -> dict[str, Any]: + """Create the lead through the CRM's own service write path. - The lead goes through ``records.create_record`` — never a raw INSERT. - That path is where ``_resolve_status`` (the NOT NULL ``status_id``), the - ``owner_email`` default, ``validate_source`` and, through ``insert_row``, + ``records.create_record`` — never a raw INSERT. That path is where + ``_resolve_status`` (the NOT NULL ``status_id``), the ``owner_email`` + default, ``validate_source`` and, through ``insert_row``, ``mark_dirty_on_insert`` all live; raw SQL silently loses all four, and only the last of them is visible in the row afterwards. - ``create_record`` opens and commits its OWN session (it is the same - function ``POST /crm/leads`` calls), so the lead is committed before the - activity is written. A failure between the two therefore leaves a lead with - an empty timeline — logged, counted, and the lesser of the two evils: the - alternative is a second, divergent write path for the record itself. + It opens and commits its OWN session (it is the same function + ``POST /crm/leads`` calls), which is also why the caller counts a created + lead before it writes the activity: by the time this returns, the row is + committed and queued for Zoho. """ first_name, last_name = _split_display_name(display_name) # Absent, never explicitly null: ``clean_payload`` is ``exclude_unset``, so @@ -524,10 +786,9 @@ async def _mint_lead( fields["first_name"] = first_name if last_name: fields["last_name"] = last_name - lead = await create_record( + return await create_record( LEADS, LeadIn(**fields), _principal(str(account.user_id)), ) - await _log_origin_activity(db, lead, account, message, address, display_name) async def _log_origin_activity( @@ -546,7 +807,7 @@ async def _log_origin_activity( """ await insert_row(db, "crm_activities", { "type": ACTIVITY_TYPE, - "subject": getattr(message, "subject", None), + "subject": _clip(getattr(message, "subject", None), MAX_SUBJECT_CHARS), "body": None, "occurred_at": getattr(message, "received_at", None), "meta": { diff --git a/apps/services/gateway/gateway/routes/email/scheduler_hooks.py b/apps/services/gateway/gateway/routes/email/scheduler_hooks.py index fb82d310c..95d7f864f 100644 --- a/apps/services/gateway/gateway/routes/email/scheduler_hooks.py +++ b/apps/services/gateway/gateway/routes/email/scheduler_hooks.py @@ -120,18 +120,28 @@ async def process_new_mail(account_id: str) -> None: error=str(exc)[:200]) try: # WS-26d-autolead. ⚠️ The flag is checked HERE, before the step is - # entered — never inside it. With CRM_AUTO_LEAD off no CRM code runs - # and no CRM query is issued on the mail path at all; a gate that - # lived inside `create_leads_from_new_mail` would open a database + # entered — never inside it. With CRM_AUTO_LEAD off the step is not + # imported, not called, opens no session and issues no query; a gate + # that lived inside `create_leads_from_new_mail` would open a database # session on every sync cycle of every mailbox to discover it had - # nothing to do. `auto_lead_enabled` is the flag's ONE definition and - # is imported rather than restated for the same reason. - from gateway.routes.crm.auto_lead import ( - auto_lead_enabled, - create_leads_from_new_mail, - ) + # nothing to do. + # + # The predicate itself is imported above the gate, and that is + # deliberate: `auto_lead_enabled` is the flag's ONE definition, and + # reading `settings.crm_auto_lead` here instead would make two places + # responsible for agreeing what the flag means — which is how a loop + # ends up running with its flag off (the `sync_enabled` precedent). + # The cost is one `sys.modules` lookup, since `routes/crm` is mounted + # by `main.py` at boot regardless of this flag. + # + # ⚠️ Divergence from the five steps above, on purpose: the import sits + # INSIDE the try. A `routes/crm` module that fails to import must be + # logged like any other CRM failure, not raised out of the mail path. + from gateway.routes.crm.auto_lead import auto_lead_enabled if auto_lead_enabled(): + from gateway.routes.crm.auto_lead import create_leads_from_new_mail + await create_leads_from_new_mail(account_id) except Exception as exc: # noqa: BLE001 _log.warning("sync.auto_lead_failed", account_id=account_id, diff --git a/infra/AGENTS.md b/infra/AGENTS.md index da0f83ed8..37231ccdb 100644 --- a/infra/AGENTS.md +++ b/infra/AGENTS.md @@ -5,7 +5,7 @@ Docker Compose, Postgres schema, LiteLLM tier config. LLM routing is via the gat ## Key Files - docker-compose.yml -- core services (Postgres 16 + pgvector, Redis 7) -- postgres/ -- schema files (00-10) + 09_app_user.sql (NextAuth users) + 11_integration_credentials.sql (unified credential store) + 130_org_access_control.sql (organization, membership lifecycle on app_user, org_role/org_role_permission/user_role, user_permission_override, feature_catalog — spec: ai-company-brain/specs/org_access_control.md) + 131_integration_memory_permissions.sql (additive: grants `integrations:use:*` + org-memory permissions to the seeded roles; `member` reads org memory but does not write it) + 143_access_request.sql (the sign-in queue — one row per address that authenticated with no `app_user` row, unique on `lower(email)`; `status` carries a `CHECK (pending|approved|denied)` because the vocabulary is load-bearing for ACCESS — approve acts only on `pending`, so a typo'd status would fall out of both the queue and the decided record; deliberately a SEPARATE table and not a fifth `app_user.status`, because an `app_user` row IS the org's member record and a stranger who merely knocked must not acquire one that a future join can surface — spec: ai-company-brain/specs/colleague_onboarding.md §6) + 144_crm.sql (the native CRM spine — organizations/contacts/leads/deals, **statuses as data** rather than enums, one `crm_activities` timeline whose four target FKs are all nullable under a CHECK requiring at least one, and a `crm_status_changes` dwell log; also seeds the `crm` feature_catalog row. Contains the schema's **one FK cycle** — `crm_leads.converted_deal_id` ⟷ `crm_deals.lead_id` — closed by a guarded `DO $$` on `pg_constraint`, because `ALTER TABLE … ADD CONSTRAINT` has no `IF NOT EXISTS`. Idempotency is pinned STATICALLY by `tests/unit/test_crm_migration.py`, which reads the file as text: the unit suite runs no database, so an idempotency claim that holds only by inspection is unenforceable. Spec: ai-company-brain/specs/crm_app.md §3) + 145_crm_zoho_sync.sql (what the two-way Zoho sync needs: `zoho_dirty`/`zoho_synced_at` on the four CRM record tables — and **only** those four, since pipeline vocabulary flows down-only and an activity's push signal is its NULL `zoho_id` — plus `crm_zoho_tombstones` (FK-less on purpose: the row it describes is gone by the time anyone reads it) and `crm_sync_cursors` (`module` PK, so the pull cannot silently rewind). Idempotent via `ADD COLUMN IF NOT EXISTS`, which — unlike `ADD CONSTRAINT` — Postgres supports directly, so no guarded `DO $$` is needed. Pinned by the same static `tests/unit/test_crm_migration.py`, which finds BOTH CRM migrations by CONTENT rather than by number. Spec: crm_app.md §7.1) + 157_crm_auto_lead_cursor.sql (WS-26d-autolead: `crm_auto_lead_cursors`, one row per `email_accounts` id, carrying `activated_at` — stamped ONCE on the first ON-state run and **never advanced**, because `received_at > activated_at` is the only thing that tells a deep resync's year-old backlog apart from new mail on the shared `process_new_mail` hook — and `processed_watermark`, the incremental cursor. Both NOT NULL: a NULL cursor is a predicate that matches nothing, which reads exactly like a working feature. ⚠️ **It deliberately adds NO unique index on `crm_leads.email`**: the cross-invocation double-mint race is accepted (one visible, hand-deletable duplicate) because a UNIQUE constraint on a column where 1,516 imported rows may already carry duplicates is a deploy-blocking migration of exactly the shape 148 had to defuse. Inert until `CRM_AUTO_LEAD` is flipped, which is OWNER-GATE. Spec: crm_app.md §9 WS-26d-autolead). ⚠️ `CREATE TABLE IF NOT EXISTS` means a column or constraint added to an ALREADY-APPLIED migration file is silently skipped on that deployment; 143, 144 and 145 have never been applied anywhere (merging 143 is the OWNER-GATE), which is the only reason they are still editable in place. +- postgres/ -- schema files (00-10) + 09_app_user.sql (NextAuth users) + 11_integration_credentials.sql (unified credential store) + 130_org_access_control.sql (organization, membership lifecycle on app_user, org_role/org_role_permission/user_role, user_permission_override, feature_catalog — spec: ai-company-brain/specs/org_access_control.md) + 131_integration_memory_permissions.sql (additive: grants `integrations:use:*` + org-memory permissions to the seeded roles; `member` reads org memory but does not write it) + 143_access_request.sql (the sign-in queue — one row per address that authenticated with no `app_user` row, unique on `lower(email)`; `status` carries a `CHECK (pending|approved|denied)` because the vocabulary is load-bearing for ACCESS — approve acts only on `pending`, so a typo'd status would fall out of both the queue and the decided record; deliberately a SEPARATE table and not a fifth `app_user.status`, because an `app_user` row IS the org's member record and a stranger who merely knocked must not acquire one that a future join can surface — spec: ai-company-brain/specs/colleague_onboarding.md §6) + 144_crm.sql (the native CRM spine — organizations/contacts/leads/deals, **statuses as data** rather than enums, one `crm_activities` timeline whose four target FKs are all nullable under a CHECK requiring at least one, and a `crm_status_changes` dwell log; also seeds the `crm` feature_catalog row. Contains the schema's **one FK cycle** — `crm_leads.converted_deal_id` ⟷ `crm_deals.lead_id` — closed by a guarded `DO $$` on `pg_constraint`, because `ALTER TABLE … ADD CONSTRAINT` has no `IF NOT EXISTS`. Idempotency is pinned STATICALLY by `tests/unit/test_crm_migration.py`, which reads the file as text: the unit suite runs no database, so an idempotency claim that holds only by inspection is unenforceable. Spec: ai-company-brain/specs/crm_app.md §3) + 145_crm_zoho_sync.sql (what the two-way Zoho sync needs: `zoho_dirty`/`zoho_synced_at` on the four CRM record tables — and **only** those four, since pipeline vocabulary flows down-only and an activity's push signal is its NULL `zoho_id` — plus `crm_zoho_tombstones` (FK-less on purpose: the row it describes is gone by the time anyone reads it) and `crm_sync_cursors` (`module` PK, so the pull cannot silently rewind). Idempotent via `ADD COLUMN IF NOT EXISTS`, which — unlike `ADD CONSTRAINT` — Postgres supports directly, so no guarded `DO $$` is needed. Pinned by the same static `tests/unit/test_crm_migration.py`, which finds BOTH CRM migrations by CONTENT rather than by number. Spec: crm_app.md §7.1) + 158_crm_auto_lead_cursor.sql (WS-26d-autolead: `crm_auto_lead_cursors`, one row per `email_accounts` id, carrying THREE timestamps because they answer three different questions: `activated_at` (the start of the current ON epoch — `received_at > activated_at` is what tells a deep resync's year-old backlog apart from new mail on the shared `process_new_mail` hook), `processed_watermark` (the incremental cursor, advanced over a batch's successful PREFIX only), and `last_run_at` (the dormancy clock, stamped every cycle — a gap beyond an hour re-anchors the epoch so a flag turned off for weeks and back on mints nothing from the gap). ⚠️ `last_run_at` is deliberately NOT derived from the watermark: the watermark tracks MAIL, so a merely quiet mailbox would read as dormant and lose the first message to arrive afterwards. All three NOT NULL: a NULL cursor is a predicate that matches nothing, which reads exactly like a working feature. ⚠️ **It deliberately adds NO unique index on `crm_leads.email`**: the cross-invocation double-mint race is accepted (one visible, hand-deletable duplicate) because a UNIQUE constraint on a column where 1,516 imported rows may already carry duplicates is a deploy-blocking migration of exactly the shape 148 had to defuse. Inert until `CRM_AUTO_LEAD` is flipped, which is OWNER-GATE. Spec: crm_app.md §9 WS-26d-autolead). ⚠️ `CREATE TABLE IF NOT EXISTS` means a column or constraint added to an ALREADY-APPLIED migration file is silently skipped on that deployment; 143, 144 and 145 have never been applied anywhere (merging 143 is the OWNER-GATE), which is the only reason they are still editable in place. ## Conventions - Postgres migrations are numbered SQL files diff --git a/infra/postgres/157_crm_auto_lead_cursor.sql b/infra/postgres/157_crm_auto_lead_cursor.sql deleted file mode 100644 index cc859b841..000000000 --- a/infra/postgres/157_crm_auto_lead_cursor.sql +++ /dev/null @@ -1,82 +0,0 @@ --- 157_crm_auto_lead_cursor.sql — WS-26d-autolead --- --- What: one row per email account recording when CRM auto-lead first became --- active on that mailbox, and how far the step has processed. --- Why: the auto-lead step hangs off `process_new_mail`, and that hook is --- reached by DEEP RESYNCS as well as by new mail. `resync_account` runs --- a ~1-year all-folder backfill and then fires the hook; a first-ever --- sync of a newly connected mailbox is deep by the same heuristic; and --- neither path stamps `rules_held_back_at` (its only writer is --- `_backfill_and_clean_job`, which does not go through this hook). A --- candidate query of "everything classified" would therefore mint a lead --- per unknown external sender across a YEAR of mail the moment a second --- mailbox connects — each born `zoho_dirty`, each queued for the live --- Zoho tenant within one 600s cycle (D-CRM-9), with no confirmation card --- anywhere on a scheduler hook and no delete tool to take them back. --- --- Two timestamps, because "is this message history?" and "have I already --- looked at this message?" are different questions and one column cannot --- answer both: --- --- activated_at set ONCE, on the step's first ON-state run for --- the account, and NEVER advanced. The backfill --- discriminator is `received_at > activated_at`: --- mail that ARRIVED before auto-lead was first --- active mints nothing, no matter when a resync --- gets around to classifying it. A moving cursor --- cannot express that — it would let a resync --- re-present year-old mail as newly processed. --- --- processed_watermark the incremental cursor, compared against --- `rules_processed_at` and advanced only after a --- batch has been written. It is what makes a --- re-run of the same sync consider nothing. --- --- Both predicates apply together. `processed_watermark` starts equal to --- `activated_at`, so the activating run itself mints nothing. --- --- ⚠️ There is deliberately NO unique index on `crm_leads.email` to go --- with this. Two concurrent `process_new_mail` invocations for one --- account can read the same watermark and double-mint; the cost is one --- visible, hand-deletable duplicate lead, and the alternative — a UNIQUE --- constraint minted on a column where 1,516 imported rows may already --- carry duplicates — is a deploy-blocking constraint of exactly the shape --- migration 148 had to defuse. The accepted race is recorded in --- `crm_app.md` §9 WS-26d-autolead. Do not "fix" it with that index. --- --- Spec: ai-company-brain/specs/crm_app.md §9 WS-26d-autolead (the cursor --- paragraph) · D-CRM-9. --- Depends on: 17_email_accounts.sql (email_accounts, the FK target) and --- 144_crm.sql (the CRM spine this cursor guards writes into). --- --- Idempotent: CREATE TABLE / CREATE INDEX IF NOT EXISTS only. No seed, no --- ALTER, nothing dropped. - -BEGIN; - -CREATE TABLE IF NOT EXISTS crm_auto_lead_cursors ( - -- The mailbox, not the CRM record: the step is per account because - -- `process_new_mail` is. CASCADE because a disconnected mailbox's cursor - -- describes nothing — the leads it already minted are CRM rows and are - -- untouched by this. - account_id UUID PRIMARY KEY - REFERENCES email_accounts (id) ON DELETE CASCADE, - - -- Written once, by the first ON-state run. Never advanced. See above. - activated_at TIMESTAMPTZ NOT NULL, - - -- Advanced to MAX(rules_processed_at) of each committed batch. - processed_watermark TIMESTAMPTZ NOT NULL, - - created_at TIMESTAMPTZ NOT NULL DEFAULT now(), - updated_at TIMESTAMPTZ NOT NULL DEFAULT now() -); - --- The candidate query reads this row by primary key, so no second index is --- needed here. This one supports the operator question the log line raises — --- "which mailboxes has auto-lead ever been active on?" — without a seq scan --- growing with the number of connected accounts. -CREATE INDEX IF NOT EXISTS idx_crm_auto_lead_cursors_activated_at - ON crm_auto_lead_cursors (activated_at); - -COMMIT; diff --git a/infra/postgres/158_crm_auto_lead_cursor.sql b/infra/postgres/158_crm_auto_lead_cursor.sql new file mode 100644 index 000000000..d6a752664 --- /dev/null +++ b/infra/postgres/158_crm_auto_lead_cursor.sql @@ -0,0 +1,105 @@ +-- 158_crm_auto_lead_cursor.sql — WS-26d-autolead +-- +-- ⚠️ Numbered 158, not 157: open PR #399 (`157_projects_recurrence.sql`) holds +-- 157. Two migrations sharing a number replay in filename order against the +-- wrong schema, so the ladder carries a deliberate reservation gap at 157 +-- until that PR lands. `tests/unit/test_crm_auto_lead.py` finds this file by +-- CONTENT rather than by number, so a further renumber in review is free. +-- +-- What: one row per email account recording the current auto-lead ON epoch for +-- that mailbox, how far the step has processed, and when it last ran. +-- Why: the auto-lead step hangs off `process_new_mail`, and that hook is +-- reached by DEEP RESYNCS as well as by new mail. `resync_account` runs +-- a ~1-year all-folder backfill and then fires the hook; a first-ever +-- sync of a newly connected mailbox is deep by the same heuristic; and +-- neither path stamps `rules_held_back_at` (its only writer is +-- `_backfill_and_clean_job`, which does not go through this hook). A +-- candidate query of "everything classified" would therefore mint a lead +-- per unknown external sender across a YEAR of mail the moment a second +-- mailbox connects — each born `zoho_dirty`, each queued for the live +-- Zoho tenant within one 600s cycle (D-CRM-9), with no confirmation card +-- anywhere on a scheduler hook and no delete tool to take them back. +-- +-- THREE columns, because that is three different questions and no one +-- column answers more than one of them: +-- +-- activated_at the start of the CURRENT ON epoch. The backfill +-- discriminator is `received_at > activated_at`: +-- mail that ARRIVED before this epoch began mints +-- nothing, no matter when a resync gets around to +-- classifying it. +-- +-- processed_watermark the incremental cursor, compared against +-- `rules_processed_at`. It advances only over the +-- contiguous prefix of a batch that actually +-- wrote its leads, so a failure is never mistaken +-- for work done. +-- +-- last_run_at when the step last RAN. This is what detects +-- dormancy — an OFF window, or an outage — and it +-- has to be its own column: the watermark tracks +-- MAIL, so a mailbox that is merely quiet over a +-- weekend has a 60-hour-old watermark while the +-- step has run faithfully every 600s. Re-anchoring +-- such an account would drop Monday's first +-- message, which is exactly the message this +-- feature exists to catch. (It also keeps a +-- deliberate stall on a poison message stalled, +-- instead of quietly re-anchoring past it.) +-- +-- When `now() - last_run_at` exceeds the step's REANCHOR_GAP_SECONDS, +-- all three are re-stamped to now: the ON epoch restarts and the gap's +-- backlog mints nothing. Fail-closed in both directions — a missed lead +-- is hand-creatable and visible in the mailbox; 27 unattended pushes +-- into a live tenant are neither. +-- +-- ⚠️ There is deliberately NO unique index on `crm_leads.email` to go +-- with this. Two concurrent `process_new_mail` invocations for one +-- account can read the same watermark and double-mint; the cost is one +-- visible, hand-deletable duplicate lead, and the alternative — a UNIQUE +-- constraint minted on a column where 1,516 imported rows may already +-- carry duplicates — is a deploy-blocking constraint of exactly the shape +-- migration 148 had to defuse. The accepted race is recorded in +-- `crm_app.md` §9 WS-26d-autolead. Do not "fix" it with that index. +-- +-- Spec: ai-company-brain/specs/crm_app.md §9 WS-26d-autolead (the cursor +-- paragraph) · D-CRM-9. +-- Depends on: 17_email_accounts.sql (email_accounts, the FK target) and +-- 144_crm.sql (the CRM spine this cursor guards writes into). +-- +-- Idempotent: CREATE TABLE / CREATE INDEX IF NOT EXISTS only. No seed, no +-- ALTER, nothing dropped. + +BEGIN; + +CREATE TABLE IF NOT EXISTS crm_auto_lead_cursors ( + -- The mailbox, not the CRM record: the step is per account because + -- `process_new_mail` is. CASCADE because a disconnected mailbox's cursor + -- describes nothing — the leads it already minted are CRM rows and are + -- untouched by this. + account_id UUID PRIMARY KEY + REFERENCES email_accounts (id) ON DELETE CASCADE, + + -- The start of the current ON epoch. Re-stamped on re-anchor, never + -- advanced by ordinary progress. See above. + activated_at TIMESTAMPTZ NOT NULL, + + -- Advanced to MAX(rules_processed_at) of each batch's successful prefix. + processed_watermark TIMESTAMPTZ NOT NULL, + + -- Stamped at the end of EVERY cycle, including the ones that considered + -- nothing and the ones that stalled. The dormancy clock. + last_run_at TIMESTAMPTZ NOT NULL, + + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT now() +); + +-- The candidate query reads this row by primary key, so no second index is +-- needed here. This one supports the operator question the log line raises — +-- "which mailboxes has auto-lead run on, and when?" — without a seq scan +-- growing with the number of connected accounts. +CREATE INDEX IF NOT EXISTS idx_crm_auto_lead_cursors_last_run_at + ON crm_auto_lead_cursors (last_run_at); + +COMMIT; diff --git a/tests/unit/_crm_fakes.py b/tests/unit/_crm_fakes.py index 4d709c46e..6d15dd178 100644 --- a/tests/unit/_crm_fakes.py +++ b/tests/unit/_crm_fakes.py @@ -165,6 +165,29 @@ #: reason `_IN_LITERALS` exists: a fake that cannot see a predicate agrees with #: the bug that deletes it. _JSONB_CONTAINS = re.compile(r"(?:(\w+)\.)?(\w+)\s*@>\s*:(\w+)", re.I) +#: ``EXISTS (SELECT 1 FROM jsonb_array_elements(to_addresses) recipient +#: WHERE lower(recipient->>'email') = :address)`` — the CASE-FOLDING form of +#: the same "have we ever emailed them" question, which WS-26d-autolead asks +#: because `@>` is case-exact and a reply from `asha@` after we wrote to +#: `Asha@` is the same person. +#: +#: ⚠️ **Reading it is not the hard part; STRIPPING it is.** Its inner +#: comparison is literally `lower(recipient->>'email') = :address`, which +#: `_JSONB_LOWER_CMP` below matches — and that reader would then filter the +#: OUTER `email_messages` rows on a `recipient` column they do not have, +#: answering "no rows" for a correct query. Same trap `_ACCOUNT_SCOPE` +#: documents, one subquery shape further on. Evaluated and removed FIRST. +#: +#: The element alias is a backreference, so a statement whose WHERE clause +#: compares a DIFFERENT alias than the one `jsonb_array_elements` declared +#: does not match here and fails the "could not read the WHERE clause" guard +#: rather than being quietly accepted. +_JSONB_ANY_LOWER = re.compile( + r"EXISTS\s*\(\s*SELECT\s+1\s+FROM\s+jsonb_array_elements\(\s*" + r"(?:(\w+)\.)?(\w+)\s*\)\s+(\w+)\s+WHERE\s+lower\(\s*\3->>'(\w+)'\s*\)" + r"\s*=\s*:(\w+)\s*\)", + re.I, +) #: ``LEFT JOIN email_thread_status ts ON ts.account_id = em.account_id AND #: ts.thread_id = em.thread_id`` — a COMPOSITE key. `_LEFT_JOIN` above demands #: literally ``ON .id = base.`` and matches nothing here. @@ -474,14 +497,20 @@ def begin_nested(self) -> _FakeSavepoint: """ return _FakeSavepoint(self) - def fail_on(self, needle: str, *, times: int = 1) -> None: - """Make the next ``times`` statements containing *needle* raise. + def fail_on(self, needle: str, *, times: int = 1, after: int = 0) -> None: + """Make ``times`` statements containing *needle* raise, skipping the + first ``after`` matches. Simulates a driver-level statement error — a `numeric field overflow`, a CHECK violation — which is the class of failure a plain ``try/except`` around a record cannot actually contain in Postgres. + + ``after`` exists because *where* in a batch the failure lands is the + property under test in WS-26d-autolead: a cursor that advances over the + successful PREFIX behaves identically to one that advances + unconditionally when the very first record is the one that fails. """ - self._failures.append([needle, times]) + self._failures.append([needle, times, after]) async def execute(self, sql: Any, params: dict | None = None) -> _Result: statement = " ".join(str(sql).split()) @@ -490,6 +519,9 @@ async def execute(self, sql: Any, params: dict | None = None) -> _Result: self.calls.append((statement, args)) for entry in self._failures: if entry[0] in statement and entry[1] > 0: + if entry[2] > 0: + entry[2] -= 1 + continue entry[1] -= 1 raise RuntimeError( f"fake driver error on statement containing {entry[0]!r}" @@ -747,6 +779,21 @@ def _email_predicates( allowed &= {str(args.get(aid))} rows = [row for row in rows if str(row.get(column)) in allowed] where = where.replace(scope.group(0), "") + # FIRST, and stripped before anything else looks at the clause — its + # inner `lower(recipient->>'email') = :address` is exactly what + # `_JSONB_LOWER_CMP` below matches, and that reader would filter the + # OUTER rows on a `recipient` column they do not have. + for _alias, column, _element, key, param in _JSONB_ANY_LOWER.findall(where): + seen = True + wanted = str(args.get(param) or "").lower() + rows = [ + row for row in rows + if any( + str((element or {}).get(key) or "").lower() == wanted + for element in (row.get(column) or []) + ) + ] + where = _JSONB_ANY_LOWER.sub("", where) for _alias, column, key, single, listed in _JSONB_LOWER_CMP.findall(where): seen = True names = ( diff --git a/tests/unit/test_crm_auto_lead.py b/tests/unit/test_crm_auto_lead.py index 2d5b195a8..ab2c834ce 100644 --- a/tests/unit/test_crm_auto_lead.py +++ b/tests/unit/test_crm_auto_lead.py @@ -104,6 +104,38 @@ def on(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr(auto_lead, "auto_lead_enabled", lambda: True) +class _Log: + """A structlog stand-in that records (level, key, kwargs). + + The stall and the re-anchor are *log lines by design* — WARNING, repeating + every cycle, because the state they report is invisible in the row counts + (a held cursor and a quiet mailbox both create nothing). A test that only + checked the counters would let the line be deleted. + """ + + def __init__(self) -> None: + self.lines: list[tuple[str, str, dict[str, Any]]] = [] + + def info(self, key: str, **kwargs: Any) -> None: + self.lines.append(("info", key, kwargs)) + + def warning(self, key: str, **kwargs: Any) -> None: + self.lines.append(("warning", key, kwargs)) + + def at(self, level: str, key: str) -> list[dict[str, Any]]: + return [kw for lvl, k, kw in self.lines if lvl == level and k == key] + + def keys(self) -> list[str]: + return [key for _lvl, key, _kw in self.lines] + + +@pytest.fixture +def log(monkeypatch: pytest.MonkeyPatch) -> _Log: + recorder = _Log() + monkeypatch.setattr(auto_lead, "_log", recorder) + return recorder + + @pytest.fixture def quiet_pipeline(monkeypatch: pytest.MonkeyPatch) -> None: """Silence the five mail steps ``process_new_mail`` runs before ours. @@ -145,11 +177,18 @@ def _seed_status(db: FakeCrmDB) -> Any: def _seed_cursor( db: FakeCrmDB, *, activated_at: datetime = ACTIVATED, - watermark: datetime | None = None, + watermark: datetime | None = None, last_run_at: datetime | None = None, ) -> Any: + """An account auto-lead is already active on. + + ``last_run_at`` defaults to NOW, not to ``activated_at``: the ordinary + state of a running account is "the step ran a moment ago", and seeding it + stale is how a test asks for the dormancy path. + """ return db.seed("crm_auto_lead_cursors", account_id=ACCOUNT_ID, activated_at=activated_at, - processed_watermark=watermark or activated_at) + processed_watermark=watermark or activated_at, + last_run_at=last_run_at or datetime.now(UTC)) #: "you did not say" — distinct from an explicit ``None``, which is how a test @@ -324,6 +363,39 @@ def test_dw2_the_gate_is_lexically_outside_the_step() -> None: assert total == 1 +def test_dw2_the_step_is_not_even_imported_when_the_flag_is_off() -> None: + """The import of the STEP lives inside the guarded branch too. + + Not cosmetic: `from gateway.routes.crm.auto_lead import + create_leads_from_new_mail` above the gate would run on every mail cycle + of every mailbox with the flag off. What must stay above the gate is the + predicate — `auto_lead_enabled` is the flag's one definition, and reading + `settings.crm_auto_lead` here instead would make two places responsible + for agreeing what the flag means. + """ + function = _process_new_mail_ast() + branches = [ + branch for branch in ast.walk(function) + if isinstance(branch, ast.If) + and _calls_named(branch.test, "auto_lead_enabled") + ] + assert branches, "the flag gate is gone" + imported_inside = { + alias.name + for branch in branches + for node in ast.walk(branch) + if isinstance(node, ast.ImportFrom) + for alias in node.names + } + assert "create_leads_from_new_mail" in imported_inside, ( + "the step is imported above the flag gate — routes/crm loads on every " + "mail cycle even with CRM_AUTO_LEAD off" + ) + assert "auto_lead_enabled" not in imported_inside, ( + "the predicate cannot be imported inside its own gate" + ) + + def test_dw2_a_crm_failure_never_breaks_mail_sync( on: None, quiet_pipeline: None, monkeypatch: pytest.MonkeyPatch, ) -> None: @@ -933,13 +1005,12 @@ async def test_a_missing_account_row_is_skipped( assert db.rows("crm_auto_lead_cursors") == [] -async def test_one_bad_message_does_not_lose_the_batch( +async def test_a_failed_activity_write_does_not_lose_the_batch( db: FakeCrmDB, on: None, ) -> None: """The WS-26b lesson: Postgres aborts the TRANSACTION on a statement error, so a bare per-record try/except loses every row after the bad one. - The savepoint is what makes "one bad message" true, and the error is - counted rather than swallowed.""" + The savepoint is what makes "one bad activity" true.""" _ready(db) _message(db, address="first@elsewhere.com", processed_at=_at(5, 9)) _message(db, address="second@elsewhere.com", processed_at=_at(5, 10)) @@ -947,12 +1018,58 @@ async def test_one_bad_message_does_not_lose_the_batch( stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) - assert stats["errors"] == 1 - assert stats["created"] == 1 + # Both leads exist, and BOTH are counted created — see the test below for + # why that is the point rather than a rounding error. + assert stats["created"] == 2 + assert stats["activity_errors"] == 1 + assert stats["errors"] == 0 + assert len(_leads(db)) == 2 + assert len(_activities(db)) == 1 assert db.savepoints == 2 assert db.savepoint_rollbacks == 1 +async def test_a_lead_whose_activity_failed_is_still_counted_created( + db: FakeCrmDB, on: None, log: _Log, +) -> None: + """`created` counts COMMITTED LEADS, not completed pairs. + + The lead is committed on its own session before the activity is attempted, + so by the time the activity fails the row exists and is already + `zoho_dirty` — it WILL push to the live tenant. A cycle that logged + `created=0` next to that row would send whoever read it looking in the + wrong place entirely. + """ + _ready(db) + _message(db) + db.fail_on("INSERT INTO crm_activities", times=1) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["created"] == 1 + assert stats["activity_errors"] == 1 + assert len(_leads(db)) == 1 + assert _leads(db)[0]["zoho_dirty"] is True + failures = log.at("warning", "crm.auto_lead_activity_failed") + assert len(failures) == 1 + assert failures[0]["lead_id"] == _leads(db)[0]["id"] + + +async def test_the_activity_failure_does_not_hold_the_cursor( + db: FakeCrmDB, on: None, +) -> None: + """It must not: the lead is committed, so the next cycle finds it at step 3 + and skips — the activity would never be retried and the cursor would stall + forever on work that cannot be redone.""" + _ready(db) + _message(db, processed_at=_at(5, 9)) + db.fail_on("INSERT INTO crm_activities", times=1) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert db.rows("crm_auto_lead_cursors")[0]["processed_watermark"] == _at(5, 9) + + async def test_the_watermark_advances_over_messages_that_minted_nothing( db: FakeCrmDB, on: None, ) -> None: @@ -991,6 +1108,355 @@ def test_the_module_registers_no_routes() -> None: assert "auto_lead" not in package +# ── done-when 8: an OFF→ON round trip mints nothing from the OFF window ───── + +async def test_dw8_an_off_then_on_round_trip_mints_nothing_from_the_gap( + db: FakeCrmDB, on: None, log: _Log, +) -> None: + """The hole ``activated_at`` alone does NOT cover. + + Flag on day 1 (cursor anchored), off days 2-29, on again day 30. Nothing + about the cursor changed while the flag was off, so the anchor still says + day 1 and every message in the OFF window passes both predicates — the + first ON cycle mints the entire four-week backlog in one batch, each lead + pushing unattended into the live tenant. Re-anchoring on dormancy is what + makes ``activated_at`` mean *the current ON epoch*. + """ + _seed_account(db) + _seed_status(db) + _seed_cursor(db, activated_at=_at(1), watermark=_at(1), + last_run_at=_at(1, 10)) # the last cycle before the flag went off + for index in range(27): + _message(db, address=f"stranger{index}@elsewhere.com", + received_at=_at(2) + timedelta(days=index), + processed_at=_at(2) + timedelta(days=index)) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["reanchored"] == 1 + assert stats["candidates"] == 0 + assert stats["created"] == 0 + assert _leads(db) == [] + cursor = db.rows("crm_auto_lead_cursors")[0] + assert cursor["activated_at"] == cursor["processed_watermark"] + assert cursor["activated_at"] > _at(1) # a NEW epoch, not the old anchor + reanchors = log.at("warning", "sync.auto_lead_reanchored") + assert len(reanchors) == 1 + assert reanchors[0]["gap_seconds"] > auto_lead.REANCHOR_GAP_SECONDS + + +async def test_dw8_mail_arriving_after_the_reanchor_mints_normally( + db: FakeCrmDB, on: None, +) -> None: + """The control: re-anchoring must start an epoch, not end the feature.""" + _seed_account(db) + _seed_status(db) + _seed_cursor(db, activated_at=_at(1), watermark=_at(1), + last_run_at=_at(1, 10)) + _message(db, address="old@elsewhere.com", received_at=_at(2)) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + assert _leads(db) == [] + + # Now a message that arrives after the new anchor. + later = datetime.now(UTC) + timedelta(minutes=5) + _message(db, address="new@elsewhere.com", received_at=later, + processed_at=later) + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["created"] == 1 + assert [row["email"] for row in _leads(db)] == ["new@elsewhere.com"] + + +async def test_dw8_a_quiet_but_running_mailbox_is_never_reanchored( + db: FakeCrmDB, on: None, log: _Log, +) -> None: + """Why dormancy reads ``last_run_at`` and not ``processed_watermark``. + + A mailbox with no classified inbox mail over a weekend has a watermark 60 + hours old while this step has run faithfully every 600s. Anchoring the + dormancy test on the watermark would re-anchor it and drop the first + message to arrive on Monday morning — which is precisely the message this + feature exists to catch, dropped every Monday. ``last_run_at`` moves on + every cycle, so "quiet" and "not running" stay different facts. + """ + _seed_account(db) + _seed_status(db) + _seed_cursor(db, activated_at=_at(1), watermark=_at(1), + last_run_at=datetime.now(UTC) - timedelta(seconds=300)) + monday = datetime.now(UTC) + timedelta(seconds=1) + _message(db, address="customer@elsewhere.com", received_at=monday, + processed_at=monday) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["reanchored"] == 0 + assert log.at("warning", "sync.auto_lead_reanchored") == [] + assert stats["created"] == 1 + + +async def test_dw8_every_cycle_stamps_last_run_at( + db: FakeCrmDB, on: None, +) -> None: + """Including the ones that considered nothing — otherwise a mailbox with + no new mail drifts into looking dormant purely by being calm.""" + _seed_account(db) + _seed_status(db) + stale = datetime.now(UTC) - timedelta(seconds=300) + _seed_cursor(db, last_run_at=stale) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert db.rows("crm_auto_lead_cursors")[0]["last_run_at"] > stale + + +# ── done-when 9: a failure never advances the cursor past lost work ───────── + +async def test_dw9_the_watermark_advances_over_the_successful_prefix_only( + db: FakeCrmDB, on: None, log: _Log, +) -> None: + """[ok, raise, ok] — the shape a pool exhaustion produces. + + This step opens a SECOND session per lead (``create_record``) while + holding the batch's own, so a pool that runs out fails several candidates + at once. An unconditional advance would step the cursor over every one of + them permanently: the reviewer measured 3 candidates, 3 errors, watermark + advanced, three leads lost for good. + """ + _ready(db) + _message(db, address="first@elsewhere.com", processed_at=_at(5, 9)) + _message(db, address="second@elsewhere.com", processed_at=_at(5, 10)) + _message(db, address="third@elsewhere.com", processed_at=_at(5, 11)) + # `after=1` so the poison lands on the SECOND candidate: a prefix rule and + # an unconditional advance are indistinguishable when the FIRST record is + # the one that fails. + db.fail_on("INSERT INTO crm_leads", times=1, after=1) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["candidates"] == 3 + assert stats["errors"] == 1 + # Held at message ONE's stamp — not message three's. + assert db.rows("crm_auto_lead_cursors")[0]["processed_watermark"] == _at(5, 9) + + +async def test_dw9_the_next_cycle_reconsiders_everything_after_the_failure( + db: FakeCrmDB, on: None, +) -> None: + """The other half: holding the cursor is only useful if the work comes + back. Message 3 succeeded in cycle 1 and is re-considered in cycle 2 — + that is free, because step 3 of the unknown-sender check finds the lead it + already created and skips.""" + _ready(db) + _message(db, address="first@elsewhere.com", processed_at=_at(5, 9)) + _message(db, address="second@elsewhere.com", processed_at=_at(5, 10)) + _message(db, address="third@elsewhere.com", processed_at=_at(5, 11)) + db.fail_on("INSERT INTO crm_leads", times=1, after=1) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + second = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert second["candidates"] == 2 # messages 2 and 3, again + assert second["created"] == 1 # message 2, finally + assert second["skipped_known"] == 1 # message 3, already a lead + assert sorted(row["email"] for row in _leads(db)) == [ + "first@elsewhere.com", "second@elsewhere.com", "third@elsewhere.com", + ] + + +async def test_dw9_a_stuck_head_message_is_logged_at_warning_every_cycle( + db: FakeCrmDB, on: None, log: _Log, +) -> None: + """A held cursor and a quiet mailbox both create nothing, so the counters + cannot tell them apart. The WARNING is the whole difference, and it + repeats — an INFO line saying `created=0` is what a stuck head message + looked like before.""" + _ready(db) + _message(db, address="poison@elsewhere.com", processed_at=_at(5, 9)) + db.fail_on("INSERT INTO crm_leads", times=2) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + stalls = log.at("warning", "sync.auto_lead_stalled") + assert len(stalls) == 2, ( + "the stall must be reported on EVERY cycle it persists, not once" + ) + assert stalls[0]["errors"] == 1 + assert db.rows("crm_auto_lead_cursors")[0]["processed_watermark"] == ACTIVATED + + +async def test_dw9_a_healthy_cycle_logs_no_stall( + db: FakeCrmDB, on: None, log: _Log, +) -> None: + _ready(db) + _message(db) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert log.at("warning", "sync.auto_lead_stalled") == [] + + +# ── The Sent probe folds case (P2-4) ─────────────────────────────────────── + +async def test_a_reply_from_someone_we_emailed_in_another_case_mints_nothing( + db: FakeCrmDB, on: None, +) -> None: + """The owner wrote to ``Asha@AcmeRobotics.com``; she replies from + ``asha@acmerobotics.com``. Postgres's ``@>`` is case-EXACT, so the + containment form of this probe says we have never emailed her and mints a + lead for somebody already mid-conversation. This module folds case + explicitly; ``_maybe_block_cold`` is left alone, being the email package's + predicate with its own blast radius.""" + _ready(db) + db.seed("email_messages", account_id=ACCOUNT_ID, folder="SENT", + from_address={"email": OWNER}, + to_addresses=[{"email": "Asha@AcmeRobotics.com"}], + rules_processed_at=None, rules_held_back_at=None) + _message(db, address="asha@acmerobotics.com") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_known"] == 1 + assert _leads(db) == [] + + +async def test_the_case_folding_probe_still_distinguishes_recipients( + db: FakeCrmDB, on: None, +) -> None: + """Folding case must not fold everything: mail to somebody else is still + mail to somebody else.""" + _ready(db) + db.seed("email_messages", account_id=ACCOUNT_ID, folder="SENT", + from_address={"email": OWNER}, + to_addresses=[{"email": "Someone.Else@Elsewhere.com"}], + rules_processed_at=None, rules_held_back_at=None) + _message(db) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["created"] == 1 + + +# ── Subdomain colleagues (P2-6) ──────────────────────────────────────────── + +async def test_a_colleague_on_a_subdomain_creates_nothing( + db: FakeCrmDB, on: None, +) -> None: + """``cfo@mail.fracktal.in`` is the CFO. Exact matching alone let him + through while ``cfo@fracktal.in`` was caught, and a company's own + ``mail.``/``corp.``/regional subdomains are routine.""" + _ready(db) + _message(db, address="cfo@mail.fracktal.in", name="Our CFO") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["skipped_internal"] == 1 + assert _leads(db) == [] + + +async def test_a_lookalike_domain_is_not_a_subdomain( + db: FakeCrmDB, on: None, +) -> None: + """The mistake a bare ``endswith`` makes: ``notfracktal.in`` is a + different company, and refusing its leads would be the suffix test doing + real damage in the other direction.""" + _ready(db) + _message(db, address="sales@notfracktal.in", name="Not Us") + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["created"] == 1 + + +# ── The cap's tie boundary (P2-5) ────────────────────────────────────────── + +def test_the_candidate_order_carries_a_tiebreak() -> None: + """Asserted as SQL TEXT, deliberately: the shared fake orders on the first + key only, so the tiebreak is not observable through it. Without ``, id`` + the order of rows sharing a timestamp is undefined and the cap could cut a + tie group in a different place on each read.""" + assert "ORDER BY rules_processed_at, id" in auto_lead._CANDIDATE_SQL + + +async def test_a_tie_group_split_by_the_cap_is_deferred_whole( + db: FakeCrmDB, on: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + """The watermark is a TIMESTAMP, so advancing it to a stamp the cap cut + through would step over the rest of that group forever.""" + monkeypatch.setattr(auto_lead, "MAX_CANDIDATES_PER_CYCLE", 3) + _ready(db) + for index, stamp in enumerate( + [_at(5, 9), _at(5, 10), _at(5, 11), _at(5, 11), _at(5, 12)] + ): + _message(db, address=f"stranger{index}@elsewhere.com", + received_at=_at(5), processed_at=stamp) + + first = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert first["boundary_deferred"] == 1 + assert first["candidates"] == 2 + assert db.rows("crm_auto_lead_cursors")[0]["processed_watermark"] == _at(5, 10) + + second = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert second["candidates"] == 3 # the whole tie group, plus 5 + assert len(_leads(db)) == 5 + + +async def test_a_cap_that_falls_between_groups_defers_nothing( + db: FakeCrmDB, on: None, monkeypatch: pytest.MonkeyPatch, +) -> None: + """The control, and the reason the drop is conditional: deferring the last + group unconditionally would shrink every capped batch by one message for + no reason at all.""" + monkeypatch.setattr(auto_lead, "MAX_CANDIDATES_PER_CYCLE", 3) + _ready(db) + for index in range(5): + _message(db, address=f"stranger{index}@elsewhere.com", + received_at=_at(5), + processed_at=_at(5, 9) + timedelta(minutes=index)) + + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert stats["boundary_deferred"] == 0 + assert stats["candidates"] == 3 + + +# ── Attacker-controlled text is bounded ──────────────────────────────────── + +async def test_a_huge_display_name_and_subject_are_clipped( + db: FakeCrmDB, on: None, +) -> None: + """Nothing upstream bounds either: a display name is whatever the sending + server put in the header, and it lands in a column every CRM list, board + card and Zoho push then carries.""" + _ready(db) + _message(db, name="Asha " + ("x" * 5000), subject="Q: " + ("y" * 5000)) + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + lead = _leads(db)[0] + assert len(lead["last_name"]) <= auto_lead.MAX_NAME_CHARS + assert len(lead["lead_name"]) <= auto_lead.MAX_NAME_CHARS + len("Asha ") + assert lead["last_name"].endswith(auto_lead.CLIP_MARKER) + activity = _activities(db)[0] + assert len(activity["subject"]) == auto_lead.MAX_SUBJECT_CHARS + assert activity["subject"].endswith(auto_lead.CLIP_MARKER) + + +async def test_an_ordinary_name_and_subject_are_untouched( + db: FakeCrmDB, on: None, +) -> None: + _ready(db) + _message(db, name="Asha Menon", subject="Quote for 40 printers") + + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert _leads(db)[0]["lead_name"] == "Asha Menon" + assert auto_lead.CLIP_MARKER not in _activities(db)[0]["subject"] + + # ── The migration, read as text ───────────────────────────────────────────── def _cursor_migration() -> Path: @@ -1023,18 +1489,37 @@ def bare(migration: str) -> str: return "\n".join(re.sub(r"--.*$", "", line) for line in migration.splitlines()) -def test_the_migration_takes_the_next_free_number() -> None: - numbers = sorted( +def test_the_migration_number_is_unique() -> None: + """The property that actually protects the ladder, and the ONLY one this + branch can hold. + + Two migrations sharing a number replay in filename order against the wrong + schema — that is the failure, and it is asserted. Contiguity is + deliberately NOT asserted: open PR #399 holds 157, so this file took 158 + and the ladder carries a reservation gap until that PR lands. (Nor is "the + highest number in the repo" asserted — `test_crm_migration.py` records + that version going red the moment any later workstream's migration landed, + and a red unit test silently blocks deploy.) + """ + numbers = [ int(path.name.split("_", 1)[0]) for path in MIGRATIONS.glob("*.sql") if path.name.split("_", 1)[0].isdigit() - ) + ] mine = int(_cursor_migration().name.split("_", 1)[0]) assert numbers.count(mine) == 1, ( f"two migrations share number {mine} — the ladder replays them in " "filename order, so one runs against the wrong schema" ) - assert mine - 1 in numbers, "the ladder has a gap immediately below it" + + +def test_the_migration_number_records_why_it_skipped_one() -> None: + """A gap in the ladder is a thing a future reader will trip over, so the + file says who holds the number it skipped.""" + header = _cursor_migration().read_text(encoding="utf-8")[:1200] + assert "#399" in header, ( + "the migration skips a number without naming the PR that holds it" + ) def test_the_migration_header_says_what_why_and_what_it_depends_on() -> None: @@ -1065,11 +1550,14 @@ def test_the_migration_drops_or_truncates_nothing(bare: str) -> None: assert not re.search(r"\b(DROP|TRUNCATE|DELETE\s+FROM)\b", bare, re.I) -def test_the_cursor_carries_both_timestamps_not_null(bare: str) -> None: - for column in ("activated_at", "processed_watermark"): +def test_the_cursor_carries_all_three_timestamps_not_null(bare: str) -> None: + """Three columns because that is three questions: which ON epoch is this + (`activated_at`), how far has the step got (`processed_watermark`), and did + it run at all (`last_run_at`). A NULL on any of them is a predicate that + matches nothing, which reads exactly like a working feature.""" + for column in ("activated_at", "processed_watermark", "last_run_at"): assert re.search(rf"{column}\s+TIMESTAMPTZ\s+NOT NULL", bare), ( - f"{column} must be NOT NULL — a NULL cursor is a predicate that " - "matches nothing, which reads exactly like a working feature" + f"{column} must exist and be NOT NULL" ) From 646ccd2f8ed19deb44da2a237d38617c906c0822 Mon Sep 17 00:00:00 2001 From: Vijay Raghav Varada Date: Sat, 8 Aug 2026 04:30:23 +0530 Subject: [PATCH 3/4] =?UTF-8?q?fix(WS-26d-autolead):=20the=20re-anchor=20a?= =?UTF-8?q?te=20the=20message=20that=20woke=20it=20=E2=80=94=20clamp,=20do?= =?UTF-8?q?n't=20reset?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit My own P1, introduced by the previous round's fix, and it was worse than the bug it fixed: instead of over-minting once after a four-week outage, it silently dropped a lead every single night. The premise I built that fix on was false. I assumed this step runs once per scheduler period, so a stale last_run_at meant "the service was down". It does not run per period: email_ingestion/scheduler.py:463-472 reads `synced` off the sync result and fires the hook ONLY when a sync actually persisted mail. A mailbox with no new mail does not run this step at all, so last_run_at freezes on a quiet mailbox exactly like the watermark does — which means the cycle that trips the dormancy test is ALWAYS the cycle carrying the message that woke it. Measured consequence: no mail 22:00 to 07:30, a cold prospect writes at 07:30:50, the sync persists it, the step runs for the first time in nine hours at 07:31:05, gap > 3600 so it re-anchors everything to 07:31:05 and returns early — and the triggering message's received_at is fifteen seconds the wrong side of the anchor its own arrival created. Excluded forever. Every night, every weekend, on exactly the message this feature exists to catch. The fix is to clamp rather than reset, and to never discard the triggering batch: activated_at moves forward to now - REANCHOR_GAP_SECONDS instead of to now, and the cycle proceeds into the batch normally. The candidate WHERE then does the work by itself — received_at > activated_at excludes the OFF window while admitting everything from the last hour, which is where the waking message always is. The watermark is deliberately untouched on re-anchor: OFF-window mail is already excluded by the anchor whatever its rules_processed_at says, and moving it would skip the triggering batch a second way. Documented residual, now asserted deliberately rather than discovered later: mail received in the final hour before the flag comes back on IS minted, up to one capped batch of it. That is the price of never dropping the waking message, and it is bounded. §6(b) tells an owner about it before they flip, with the advice to flip at a quiet moment if the tail matters. The guarding test was green on fiction — it seeded last_run_at = now - 300s directly, a state the scheduler cannot produce, so it could not see any of this. Tests now drive a movable clock and express the lull the way the scheduler does: by NOT CALLING the step. The Monday-morning case runs an evening cycle, advances nine and a half hours with no invocation at all, then delivers the 07:30 message and asserts it is minted in the same cycle that re-anchors. Also corrected the docstrings that carried the false premise — the constant's comment claimed "six scheduler periods", which the wiring does not provide, and the third-column rationale claimed the step "runs faithfully every 600s" on a quiet mailbox, which it does not. last_run_at is still the right clock, but for a narrower reason than I wrote: both it and the watermark freeze on a mailbox receiving nothing, and what separates them is a cycle that ran and found no candidates — which is also the state a deliberate stall holds the watermark in. Migration hardening: CREATE TABLE IF NOT EXISTS is a no-op against a database that already has the table, so it cannot add a column to one. A scratch DB that applied this file while it was still the two-column version would keep the old shape and fail every cycle. Added the guarded ADD COLUMN IF NOT EXISTS, a COALESCE backfill (SET NOT NULL refuses otherwise), and SET NOT NULL — all no-ops on a fresh database. Keeping _JSONB_CONTAINS in _crm_fakes: it is what makes mutant k expressible, and that mutant is the only thing standing between the case-folding Sent probe and a silent revert to case-exact containment. 73 -> 75 tests; 13 -> 15 mutants red and reverted, the two new ones being the reset-instead-of-clamp and the early-return that discarded the batch. R4 in the same change: the cursor paragraph's re-anchor block, done-when 8, as-built decision 6 (which now names its own earlier false premise rather than quietly replacing it), work_plan's WS-26 row and the §6(b) owner note, both AGENTS.md. Still BUILT, NOT FLIPPED, NOT DEPLOYED. Co-Authored-By: Claude Fable 5 --- ai-company-brain/specs/crm_app.md | 109 +++++---- ai-company-brain/work_plan.md | 11 +- apps/services/gateway/AGENTS.md | 2 +- .../gateway/gateway/routes/crm/auto_lead.py | 146 ++++++++---- infra/AGENTS.md | 2 +- infra/postgres/158_crm_auto_lead_cursor.sql | 22 ++ tests/unit/test_crm_auto_lead.py | 215 +++++++++++++----- 7 files changed, 360 insertions(+), 147 deletions(-) diff --git a/ai-company-brain/specs/crm_app.md b/ai-company-brain/specs/crm_app.md index 498f1a065..8a59cdde8 100644 --- a/ai-company-brain/specs/crm_app.md +++ b/ai-company-brain/specs/crm_app.md @@ -61,12 +61,15 @@ > re-anchor need. **Nothing has been flipped and nothing has been deployed** — > the flip stays OWNER-GATE (`work_plan.md` §6 (b)), and while the flag is off > this changes no runtime behaviour at all. Tests: -> `tests/unit/test_crm_auto_lead.py` (73 cases); thirteen mutants run red and -> reverted. ⚠️ **One diff-review round landed on this branch** and closed two -> P1s that only a running cursor would have shown: an OFF→ON round trip minted -> the whole OFF window (27 leads measured), and a single failure advanced the -> cursor over every candidate behind it (3 leads lost, measured). See the -> ticket's done-when 8 and 9. +> `tests/unit/test_crm_auto_lead.py` (75 cases); fifteen mutants run red and +> reverted. ⚠️ **Two review rounds landed on this branch.** The first closed two P1s +> that only a running cursor would have shown: an OFF→ON round trip minted the +> whole OFF window (27 leads measured), and a single failure advanced the +> cursor over every candidate behind it (3 leads lost, measured). The second +> closed a P1 the FIRST FIX introduced — the re-anchor reset the anchor to +> `now` and returned early, which discarded the very message that woke the +> step, every night and every weekend; it now clamps to `now - 1h` and runs the +> batch. See the ticket's done-when 8 and 9. > · **WS-26e: 🟡 SPEC, nothing built.** > **26f** — 🟢 **MERGED + DEPLOYED 2026-08-07 (PR #391), NOT RUN against the tenant.** f1 > `POST /crm/import/zoho/stages` (`routes/crm/stage_metadata.py`, floor @@ -1457,7 +1460,7 @@ Frontend: extend the existing CRM vitest for the third `kind`. > import sits INSIDE the `try`, so a `routes/crm` module that fails to import > is logged on `sync.auto_lead_failed` like any other CRM failure rather than > raised out of the mail path. -> * `tests/unit/test_crm_auto_lead.py` — **73 cases**, each done-when named in +> * `tests/unit/test_crm_auto_lead.py` — **75 cases**, each done-when named in > a test. `tests/unit/_crm_fakes.py` gained two readers and one capability: > `@>` jsonb containment and the case-folding > `EXISTS (… jsonb_array_elements …)` form (each needed because a probe the @@ -1466,13 +1469,15 @@ Frontend: extend the existing CRM vitest for the third `kind`. > reader misreads its inner comparison), plus `fail_on(..., after=N)`, > because *where* in a batch a failure lands is the whole property under > test in done-when 9. -> * **Thirteen mutants run red and were reverted** (seven pre-review, six more -> for the repair round): the flag check · the `received_at > activated_at` +> * **Fifteen mutants run red and were reverted** (seven pre-review, six for +> the repair round, two more for the delta re-review): the flag check · the `received_at > activated_at` > predicate · the watermark advance · the internal-domain second gate · > `type='system'` · the service write path · in-batch dedup · the dormancy > re-anchor · the prefix-only advance · the stall WARNING (demoted to INFO) · > the case-folding Sent probe · suffix-aware internal domains · the -> `last_run_at` stamp on a quiet cycle. The Sent-probe mutant is additionally +> `last_run_at` stamp on a quiet cycle · the anchor CLAMP (reset to `now`) · +> the early return that discarded the triggering batch. The Sent-probe mutant +> is additionally > checked for PRECISION: reverting it must NOT redden the lower-case > already-emailed case, or the mutant broke the probe rather than narrowing > it. @@ -1508,21 +1513,22 @@ Frontend: extend the existing CRM vitest for the third `kind`. > even activated.** `actor()` would attribute the lead to `"anonymous"`, > and a lead that is nobody's follow-up and that the `owner` filter cannot > match is worse than no lead. -> 6. ⚠️ **Dormancy is measured on a THIRD column, `last_run_at`, not on -> `processed_watermark`** — a deliberate departure from the shape the P1-1 -> ruling prescribed, because the watermark tracks MAIL rather than runs and -> the literal version has two production failures. (a) A mailbox that is -> merely quiet over a weekend carries a 60-hour-old watermark while the step -> has run every 600s, so it would be re-anchored and **Monday's first -> message — the one this feature exists to catch — would fall before the new -> anchor and mint nothing. Every Monday.** (b) A poison head message holds -> the watermark still ON PURPOSE (decision 3); a watermark-based test would -> re-anchor past it after an hour and silently undo the stall that was -> supposed to stay visible. `last_run_at` is stamped on every cycle -> including empty and stalled ones, so "quiet" and "not running" stay -> different facts. The ruling's constant (3600s), log key -> (`sync.auto_lead_reanchored`) and behaviour on a real OFF window or outage -> are unchanged. +> 6. ⚠️ **The re-anchor CLAMPS `activated_at` to `now - REANCHOR_GAP_SECONDS` +> and runs the batch anyway** — it does not stamp `now`, and it does not +> return early. Corrected in the delta re-review after the first fix shipped +> a P1 regression of its own: this step is invoked only when a sync +> PERSISTED mail (`email_ingestion/scheduler.py:463-472`), never once per +> period, so the cycle that trips dormancy is always the cycle carrying the +> message that woke it — and resetting the anchor excluded that message +> permanently, every night and every weekend. **The premise the earlier +> version of this note gave for the third column was false** (it claimed the +> step runs every 600s on a quiet mailbox; it does not run at all). +> `last_run_at` is still the right clock, for the narrower reason recorded in +> the cursor paragraph, and it is stamped on every cycle including empty and +> stalled ones. The ruling's constant (3600s), log key +> (`sync.auto_lead_reanchored`) and fail-closed behaviour on a genuine OFF +> window are unchanged; the accepted residual — the last gap-width of the +> window is minted — is recorded there and in §6 (b). > 7. **Both attacker-controlled strings are clipped** (`MAX_NAME_CHARS` 120, > `MAX_SUBJECT_CHARS` 500, with a `…` marker). Nothing upstream bounds > either: a display name is whatever the sending server put in the header, @@ -1579,10 +1585,13 @@ The step therefore keeps a **per-account three-timestamp cursor** in a new table (`rules_processed_at IS NOT NULL`, `rules_held_back_at IS NULL`) with `rules_processed_at > processed_watermark`. - `last_run_at` — when the step last RAN, stamped on **every** cycle including the ones - that considered nothing. This is the dormancy clock, and it is a third column rather - than a reading of the second because the watermark tracks MAIL: a mailbox that is - merely quiet over a weekend has a 60-hour-old watermark while the step has run - faithfully every cycle. + that considered nothing. This is the dormancy clock. ⚠️ It is a third column rather + than a reading of the second for a narrower reason than first claimed: BOTH freeze on + a mailbox that receives nothing, because the hook does not fire at all without new + mail. What separates them is a cycle that ran and found no *candidates* — mail outside + the inbox, held back, or predating the anchor — which advances `last_run_at` and not + the watermark; and that is also the state a deliberate stall holds the watermark in, + so keying dormancy on the watermark would re-anchor past a stall and quietly undo it. **Re-anchoring, and why "set ONCE" was wrong (2026-08-08 diff review, P1-1).** `activated_at` alone guards a deep resync and does nothing about a flag that was on, @@ -1590,11 +1599,31 @@ turned OFF for four weeks, and turned back on: the cursor still carries the old so the first ON cycle mints the whole OFF window in one batch — **measured at 27 leads for a 27-day window**, each pushing unattended into the live tenant. So when an ON-state cycle finds `now - last_run_at > REANCHOR_GAP_SECONDS` (a named constant, -3600 — six scheduler periods), it **re-stamps all three timestamps to now**, mints -nothing from the gap, and logs `sync.auto_lead_reanchored` with `gap_seconds`. -Fail-closed in both directions — an OFF window and a real outage each skip their -backlog. A missed lead is hand-creatable and visible in the mailbox; 27 unattended -pushes into the live tenant are neither. +3600), it **clamps `activated_at` forward to `now - REANCHOR_GAP_SECONDS`**, logs +`sync.auto_lead_reanchored` with `gap_seconds`, and **runs the batch normally**. + +⚠️ **CLAMP, never reset — and never discard the triggering batch** (2026-08-08 delta +re-review; the first fix stamped `now` and returned early, which was a P1 regression in +its own right). **This step is not invoked once per scheduler period.** +`email_ingestion/scheduler.py:463-472` reads `synced` off the sync result and fires the +hook **only when mail was actually persisted**, so a mailbox with no new mail does not +run this step at all — which means the cycle that trips the dormancy test is *always* +the cycle carrying the message that woke it. Measured: no mail 22:00→07:30, a cold +prospect writes at 07:30:50, the sync persists it, the step runs at 07:31:05, and a +reset-to-`now` anchor excluded that message **permanently — every night and every +weekend**. Clamping keeps the fail-closed property that matters (anything received +longer ago than the gap width stays excluded, so a real OFF window or multi-day outage +still mints nothing from its backlog) while admitting everything received inside the +last hour, which is where the triggering message always is. The watermark is NOT +touched on re-anchor: OFF-window mail is excluded by the anchor predicate whatever its +`rules_processed_at` says, and moving it would skip the triggering batch a second way. + +**Documented residual, accepted:** mail received in the final `REANCHOR_GAP_SECONDS` of +an OFF window IS minted when the flag comes back on. It is bounded — one gap width, and +at most one capped batch of it — and it is the deliberate price of never dropping the +message that woke the step. A named test asserts it as the bound rather than leaving it +to be discovered. Fail-closed otherwise: a missed lead is hand-creatable and visible in +the mailbox; 27 unattended pushes into the live tenant are neither. **A failure never advances the cursor past lost work (2026-08-08 diff review, P1-2).** The watermark advances to the max `rules_processed_at` of the **contiguous prefix of @@ -1696,10 +1725,14 @@ real prospect's leads, which is the same damage in the other direction. the test seeds a year-old classified backlog and runs the ON-state hook against it. 8. **An OFF→ON round trip mints nothing from the OFF window** (added 2026-08-08, P1-1): the hook run against an account whose cursor was anchored weeks ago and whose - `last_run_at` is stale creates zero leads and re-anchors the cursor — seed a stale - cursor plus a backlog. Its control is named too: a mailbox that is merely QUIET but - still running is never re-anchored, because re-anchoring it would drop the first - message to arrive after the quiet spell — the one this feature exists to catch. + `last_run_at` is stale creates zero leads and re-anchors the cursor. ⚠️ Its + counterpart is equally named and equally load-bearing: **the message that WOKE the + step is minted, not discarded** — an overnight lull expressed the way the scheduler + produces it (the step simply not called), then a message arrives and IS turned into a + lead in the same cycle that re-anchors. Neither test may seed `last_run_at` by hand: + a stale value the scheduler cannot produce is a test green on fiction, which is + exactly how the first fix shipped its regression. The residual is asserted too — mail + from the last gap-width of the OFF window is minted, deliberately and boundedly. 9. **A failure never advances the cursor past lost work** (added 2026-08-08, P1-2): a batch of `[ok, raise, ok]` leaves the watermark at the FIRST message's stamp, the next cycle re-considers messages 2 and 3, and a cycle whose watermark did not move diff --git a/ai-company-brain/work_plan.md b/ai-company-brain/work_plan.md index 18db7d39f..786ee97d8 100644 --- a/ai-company-brain/work_plan.md +++ b/ai-company-brain/work_plan.md @@ -147,7 +147,7 @@ looks. Full statements live in `FOUNDATION_BUILDOUT_CHECKLIST.md`. | WS-20 | **WhatsApp activation + remainder** (search UI 🟢 AGENT-SAFE; OCR needs a vision-tier decision; Odoo/Zoho-bound items blocked) | `whatsapp_message_manager.md` §11 (header fixed 2026-08-01) | 🟡 owner | **OWNER-GATE:** Meta env/app review, enrichment cost flags. | | WS-21 | **Calendar F2/F3** (`gtd_time_blocks`, email windows, mobile timeline, external sync) | `calendar_focus_os.md` **§9** (canonical for all F2/F3 acceptance; **§5** canonical for `gtd_time_blocks`) + `calendar_timeboxing.md` **§13** (canonical for P4) — both rewritten 2026-08-03 | 🟡 partial | **Re-audited 2026-08-03 → GO-NARROWED.** P3 roll-over was already shipped (released-to-unscheduled, mig 78 + `start_auto_rollover`). ~~"ideal week"~~ **struck — substantially shipped** (mig 98 + settings round-trip + editor + grid render + packer honouring + 2 unit tests); only the unused-focus-window / template-adherence gap remains (§9.6). **Breaks-in-the-packer SHIPPED 2026-07-23** (`80722e17`, mig **97**) as *packer geometry* — a widened buffer plus lunch protection, **a gap, not a `kind='break'` row**, which is exactly why F2 survives (§5 residual 4, now closed). **The 2026-08-01 acceptance was satisfiable by doing nothing** — 2 of its 3 `gtd_time_blocks` clauses were already green against shipped code; they are deleted and replaced with four that all fail today. **`gtd_time_blocks` is 4 slices, not 1 PR** (§9.1 S1–S4): the "non-breaking `TimeBlock[]` swap" claim was **FALSE** — the measured blast radius is 17 TS files + 3 gateway modules + `apps/skills/skill-task-gtd/` + `apps/agents/agent-task-manager/`. **Focus Shield is AGENT-SAFE, not owner-gated** (§9.5) — it needs a design, not a credential; do not dispatch on §4.1 prose alone. **Top-5 outcomes (Horizons) — DO NOT DISPATCH:** it collides with WS-18; §4 assigns it here, and WS-18's title keeps it struck. **Verify by naming test files — never `pytest tests/unit -k calendar`**: `-k` still collects the whole directory, and whole-directory collection hangs on the Windows box. **Dispatchable today:** §9.1 S1 · the ritual-stamp localStorage residue (§9.1 done-when 4, independently shippable) · §9.6 · §9.7. **OWNER-GATE:** external sync (§9.11 / timeboxing §13 P4) needs Google Calendar and/or Microsoft Graph OAuth client credentials provisioned on the VPS. | | WS-22 | **draw.io** (all 13 tickets open, nothing built) | `drawio_integration.md` | 🟡 owner | Best acceptance structure in the corpus; needs an owner and re-verified anchors (~5 weeks stale). ST-DRW-02 is a decision gate. | -| **WS-26** | **CRM app — native CRM + Zoho retirement** *(minted 2026-08-05)* | `specs/crm_app.md` | ✅ **a + b + c BUILT + DEPLOYED** · ✅ **d read half BUILT + DEPLOYED** · ✅ **D2 = d-email BUILT 2026-08-07** (branch `ws-26d-email-timeline`, merged) · ✅ **D4 = d-write MERGED + DEPLOYED 2026-08-08 (PR #400, no migration; deploy 31217978773 log-verified)** · ✅ **D5 = d-autolead BUILT 2026-08-08 (branch `ws-26d-autolead`, migration 158; flag OFF, NOT flipped, NOT deployed)** · ✅ **D1 = f BUILT 2026-08-07 (branch `ws-26f-pipeline-truth`, NOT run against prod)** · ✅ **D3 = g BUILT 2026-08-07 (branch `ws-26g-reports`, no migration)** · 🟢 **DEMO CRITICAL PATH COMPLETE (owner-directed 2026-08-07, spec §9.0): ~~D1 f~~ (∥ D2 d-email) → ~~D3 g~~ → ~~D4 d-write~~ → ~~D5 d-autolead~~** · 🟡 **h/i/e deferred past the demo; i spec-thin** | Research pass 2026-08-05: `frappe/crm` (AGPL — **concepts only, no code**), `trycompai/crm` (MIT), full-tree Zoho sweep. **Zoho today is a read-only nightly mirror** into the Phase-0 graph tables (`person`/`customer`/`deal`) with no UI, no write path, and **no Leads pull** — so leaving Zoho is import-and-retire, not a live cutover. Spine: Frappe's lead→convert→deal+contact+organization with **statuses-as-data** (color/position/type/probability); trycompai's single activity-spine table + `source` provenance + `last_activity_at` discipline. **BO-10 contribution: WS-26a adds the shared engine seam (`gateway/db.py::get_engine()`, tasks converted as proof) instead of engine 13.** Tickets: **a** schema + feature registration + core API — **BUILT 2026-08-05** (mig `144_crm.sql`, `feature:crm`, `gateway/db.py` seam + tasks converted, `routes/crm/`; **migration 144 applied on prod and `/crm` live as of 2026-08-06**) · **b** **Zoho two-way sync — BUILT 2026-08-05** (branch `ws-26b-zoho-sync`: `list_leads` + `list_deleted` on the read client, the single write client `ingestion/sources/zoho/writer.py` with one grep-asserted caller, mig `145_crm_zoho_sync.sql` (dirty columns + `crm_zoho_tombstones` + `crm_sync_cursors`), `routes/crm/{import_zoho,sync_zoho,broker_handlers}.py`, `crm.zoho_*` broker handlers registered from `main.py`, 80 new hermetic tests). *(Re-scoped 2026-08-05, owner-directed D-CRM-7: "faithful two way sync until we do away with Zoho entirely" — coexistence is bidirectional, not import-once.)* **Measured 2026-08-06: mig 145 is applied on prod and the BACKFILL HAS RUN — 737 orgs / 1,189 contacts / 1,516 leads / 551 deals / 1,909 notes, zero dirty rows, zero unmatched owners; the §7.1 pre-flip curl confirmed the tenant honors RFC-1123 `If-Modified-Since` (304). The PUSH direction has still never run: `CRM_ZOHO_SYNC` ships OFF, nothing has ever written the live Zoho tenant, and enabling the flag or hand-running a push cycle against prod stays OWNER-GATE §6.** WS-1's "no Zoho write path anywhere" clause was corrected in the same change (done-when 6) · **c** UI + the API addendum — **BUILT 2026-08-05** on branch `ws-26c-crm-ui` atop 26a and **merged with b into `ws-26-crm-app` 2026-08-06** (`/crm` app + BFF proxy; the three frontend registration points with `CenterApp` re-typed so `live ⇒ href` is a compile error; `routes/crm/deal_contacts.py` with one-primary-per-deal enforced on the shared `core.link_deal_contact` seam the convert path now also uses — 26b's importer is the one excepted writer and computes `is_primary` in-statement so a backfill can never demote a hand-set primary; `organization_name` on the deal list + board via a derived-table LEFT JOIN; the three review residuals — `?status_id` on a pipeline-less entity → 422, explicit `null` on a defaulted NOT NULL column → 422 not a driver 500, and a hand-edited `lead_name` surviving a name-field PATCH. **Deployed:** migrations 144 and 145 are applied on prod as of 2026-08-06 and `/crm` is live, so live rendering, drag persistence and deep links are owner-verifiable now) · **d** integrations — **audited 2026-08-06 GO-NARROWED and the narrowed slice is BUILT** (branch `ws-26d-agent-crm`): `apps/agents/agent-crm/` (`crm-assistant`, MAF, four READ tools over the existing `/crm` routes carrying the caller's `X-User-Email`, read-only enforced at the transport by a GET-only method allowlist) registered in `_KNOWN_AGENTS` + `_AGENT_REGISTRY` + `agent_registry.json`, plus `"crm"` added to the WhatsApp `_KNOWN_SYSTEMS` allowlist **parse-only** (nothing writes `wa_contacts.entity_ref`, the `crm` context block stays `None`, both pinned by test). **The three held-back items are now DISPATCHABLE — their doc blockers (B3/B4/B5/B7) were closed 2026-08-06 in `crm_app.md` §9.1-§9.3, every anchor read off `origin/main` rather than recalled:** **WS-26d-email** (the timeline join is CALLER-scoped, never record-scoped — it reuses the email app's `_account_scope` predicate, copied into `routes/crm/` rather than imported per D-CRM-4, joins by thread not message, inbound `from_address` only, and needs a new address index at the next free migration number) · **WS-26d-autolead — BUILT 2026-08-08** (branch `ws-26d-autolead`, migration **158** `crm_auto_lead_cursors`). Hook = `routes/email/scheduler_hooks.py::process_new_mail`, the one seam scheduler+manual+webhook all funnel through; the per-message rules loop was considered and REJECTED because a classifier outage there double-fires and history backfills never reach it. ⚠️ **That same seam is ALSO reached by deep resyncs** (2026-08-08 audit blocker G1, closed by PR #402 before the build), which is why the step keeps a per-account THREE-timestamp cursor: `activated_at` (the start of the current ON epoch) makes `received_at > activated_at` the backfill discriminator, `processed_watermark` is the incremental one, and `last_run_at` is the dormancy clock. Without the first, connecting a second mailbox mints a lead per unknown sender across a year of mail, each queued for the live tenant. ⚠️ **Diff review closed two P1s the first cut had, and both needed a running cursor to see:** `activated_at` alone did nothing about an OFF→ON round trip (measured: 27 leads minted for a 27-day OFF window), so a gap in `last_run_at` beyond `REANCHOR_GAP_SECONDS` (3600) re-anchors all three timestamps and mints nothing from the gap; and the watermark advanced over messages that RAISED (measured: 3 candidates, 3 errors, 3 leads lost for good), so it now advances over the successful PREFIX only, with a held cursor reported at WARNING on `sync.auto_lead_stalled` every cycle. **Dormancy reads `last_run_at` and not the watermark deliberately** — the watermark tracks MAIL, so a merely quiet mailbox would be re-anchored and would drop the first message to arrive afterwards, every Monday. Unknown-sender check mirrors `_maybe_block_cold`'s two steps and adds a third (no `crm_contacts`/`crm_leads` row with that `lower(email)`) — **the ticket's original `ON CONFLICT DO NOTHING` could not have fired** (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so dedup is that SELECT guard plus in-batch de-duplication, with the cross-invocation race accepted and the UNIQUE index refused. Colleague suppression is TWO gates: `sender_scope` (which fails SAFE to "external", the wrong direction here) and the normalised internal-domain list. The originating message is logged `type='system'` — outside `sync_zoho`'s `type IN ('note','task')` push predicate — with subject + sender in `meta` and **`body` empty**, so the mail's content never leaves the native CRM even though the lead does. The lead goes through `records.create_record`, never raw SQL. **Built, flag `False`, NOT flipped, NOT deployed; the flip stays §6 (b).** 73 hermetic cases; 13 mutants red and reverted (one of them precision-checked) · **WS-26d-write — BUILT 2026-08-08** (branch `ws-26d-write`, **no migration**: every route the four tools call already existed). `request_confirmation` awaited at the top of each tool before any mutating request is built, fail-closed, and the `non_interactive_default` keyword is asserted ABSENT from the whole module rather than asserted != "approve" — pinning the argument rather than the value means a mutant does not get to pick a spelling the fence has not heard of. `_ALLOWED_METHODS` **widened, never deleted**: `{GET, POST, PATCH}`, still checked inside `_request`, with `DELETE`/`PUT` and any `_delete`/`_put` helper still absent, so the check that used to enforce "read-only" now enforces "never destroys". Path fence extended past `ast.JoinedStr` to `.format`/`%`/`+` (the re-review's P2) and — the part that makes it maintainable — **tested against synthetic sources one per idiom**, so "the fence went blind" is a red test rather than a silent gap. Two supervisor rulings landed as built: `update_deal_status` resolves the stage BY NAME inside the tool against `GET /crm/statuses/deal` (no UUID on the LLM surface; an unknown name returns the real lane names), and a lost-type target requires a `lost_reason` resolved the same way against `GET /crm/lost-reasons` — pre-empting the 422 the "close this as lost" demo beat would otherwise hit — with the vocabulary **only ever read, never created**. `create_lead` takes **no `owner_email` argument at all** (the route derives it from the acting user), deleting an LLM-filled identity field from the surface entirely. ⚠️ **One recorded departure from done-when 1**: the invariant asserted is *no mutation before consent*, not *no HTTP before consent* — two tools must read to describe honestly what they are about to do, and every pre-card call being a GET is itself pinned; the two tools that owe nothing to a pre-read are still held to literally zero calls. `@_annotate_risk` is the shared annotation convention and is NOT enforcement; the Action Broker covers the Zoho push, not the native write — the two fail in OPPOSITE directions and are not interchangeable. 76 new hermetic cases + `test_crm_agent.py` 87 → 143; ten mutants run red and reverted. **Built, not deployed.** The push-queue question is CLOSED — **D-CRM-9 (owner, 2026-08-06): agent-originated writes queue for Zoho exactly like human ones** (🟡 remainder; the flip stays OWNER-GATE) · **e** cutover + retirement inventory + **Zoho refresh-token revoke, which executes part of WS-2's standing P0** (🔴 OWNER-GATE end-to-end). Data-visibility departure recorded: org-visible to `feature:crm` holders in v1, `owner_email` is assignment not ACL (D-CRM-3; workflows v1 is the precedent) — revisit at WS-14 `group:` grants / colleague #1. **Pipeline blueprint added 2026-08-07 (spec §5.1) after the owner's first live board session found lanes out of order and imported stages at 0% probability — root cause: the importer appends unseen Zoho stages past the seeds at probability 0, and 144's seeds renamed Zoho's defaults so name-match missed them; the admin API that could fix it is headless.** New tickets: **f** pipeline truth + settings UI — **BUILT 2026-08-07** (branch `ws-26f-pipeline-truth`, **no migration**: 144 already carried `position`, `probability`, `type`, `closed_at` and `expected_close_date`). `routes/crm/stage_metadata.py` = `POST /crm/import/zoho/stages`, floor `admin:access:manage`, **dry-run by default** and `?apply=true` to write; **>1 pipeline STOPS before the DB is opened** (D-CRM-11); f4's `closed_at` proxy is a direct UPDATE that bypasses `mark_dirty_on_update`, asserted statically against the statement text AND the module's call graph because the shared fake writes only what a SET clause names. Two settings readers on the Zoho read client with `ZohoScopeError`/`ZohoApiVersionError` so **no-scope, no-data and no-such-endpoint are three different reported outcomes** — and no-scope is the *expected* first answer, since the tenant's refresh token was never minted with `ZohoCRM.settings.*`. D-CRM-10's clamp landed in `admin.py::_validate_status` reading the ROW rather than the payload (a PATCH naming only `type`, or only `probability`, contradicts the rule only in combination with what is stored). `?tab=settings` is the headless-API fix and needs no Zoho token at all. **Nothing has been run against the tenant — dry run included.** · **g** forecast & funnel reports off `crm_status_changes` — **BUILT 2026-08-07** (branch `ws-26g-reports`, **no migration**: 144's `crm_status_changes` already carried every column). `routes/crm/reports.py` = four read-only endpoints (`/crm/reports/{pipeline,funnel,win-loss,owners}`) on the shared gated router, plus `?tab=reports`. **`WEIGHTED_SQL` lifted into `core.py`** (pipeline re-exports it) so the forecast formula has ONE definition, and `core.status_wire` absorbed two duplicate status projections instead of gaining a third. **The cross-language parity mechanism is minted here, not inherited** — the `priority.ts ⟷ priority.py` "precedent" is two hand-kept mirrors joined by a comment with no shared fixture anywhere: `tests/fixtures/crm_weighted_parity.json` (new directory) is read by BOTH pytest (through the emitted SQL, whose expression `_crm_fakes._WEIGHTED_SUM_RE` parses out of the statement text) and vitest (through `board.ts::weightedDeal`/`weightedRows`). ⚠️ **The funnel is defined against what the log RECORDS, not what its name suggests**: `crm_status_changes` logs transitions only, so all 551 imported deals have zero rows and "entered" is a VISITED-SET union (`from_status`, `to_status`, and the deal's current stage) — a `to_status` count reports an empty funnel for the whole live board; dwell is grouped by **`from_status`** (the stage being LEFT, the opposite key from the naive reading); the log stores NAMES, so a renamed lane orphans its history and orphans are REPORTED in `unmatched`, never dropped; and NULL `closed_at` — every imported closed deal until f4's owner-gated backfill runs — falls outside the trailing window, with the count reported so a 0% win rate is explicable rather than mysterious. The lost-reason breakdown carries a NAMED unattributed bucket: the earlier "complete by construction" claim is FALSE, since the importer bypasses both gates and `lost_reason_id` is `ON DELETE SET NULL`. **No `GROUP BY` is emitted** — the ticket asked for the choice to be stated: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would stop BEING the expression the fixture and the fake read. 47 hermetic cases + 20 vitest + the 14-row shared fixture on both sides, 5 mutants red. **Two findings:** the `entity_type='deal'` mutant initially SURVIVED (the funnel keys through deal ids, so a lead row is excluded twice over) — closed with a row stamped `lead` against a DEAL's id, which is realistic precisely because `entity_id` has **no FK**; and `_crm_fakes`' `lower(col) = :param` reader matched a NULL column, which SQL never does, making the unassigned-owner bucket count rows its own aggregate never summed · **h** stage entry-requirements + rot badges (after f2) · **i** merge/bulk/CSV/saved-views (🟡 spec-thin, audit-narrow first). **Demo path (2026-08-07, spec §9.0): full chain + all gates intact, tickets re-sequenced not thinned; f gained f4 — imported won/lost deals have no `closed_at` (importer never stamps it), backfilled from Zoho `Closing_Date` as a labeled proxy via a direct UPDATE that MUST bypass dirty-marking or ~500 no-op pushes queue for the live tenant.** | +| **WS-26** | **CRM app — native CRM + Zoho retirement** *(minted 2026-08-05)* | `specs/crm_app.md` | ✅ **a + b + c BUILT + DEPLOYED** · ✅ **d read half BUILT + DEPLOYED** · ✅ **D2 = d-email BUILT 2026-08-07** (branch `ws-26d-email-timeline`, merged) · ✅ **D4 = d-write MERGED + DEPLOYED 2026-08-08 (PR #400, no migration; deploy 31217978773 log-verified)** · ✅ **D5 = d-autolead BUILT 2026-08-08 (branch `ws-26d-autolead`, migration 158; flag OFF, NOT flipped, NOT deployed)** · ✅ **D1 = f BUILT 2026-08-07 (branch `ws-26f-pipeline-truth`, NOT run against prod)** · ✅ **D3 = g BUILT 2026-08-07 (branch `ws-26g-reports`, no migration)** · 🟢 **DEMO CRITICAL PATH COMPLETE (owner-directed 2026-08-07, spec §9.0): ~~D1 f~~ (∥ D2 d-email) → ~~D3 g~~ → ~~D4 d-write~~ → ~~D5 d-autolead~~** · 🟡 **h/i/e deferred past the demo; i spec-thin** | Research pass 2026-08-05: `frappe/crm` (AGPL — **concepts only, no code**), `trycompai/crm` (MIT), full-tree Zoho sweep. **Zoho today is a read-only nightly mirror** into the Phase-0 graph tables (`person`/`customer`/`deal`) with no UI, no write path, and **no Leads pull** — so leaving Zoho is import-and-retire, not a live cutover. Spine: Frappe's lead→convert→deal+contact+organization with **statuses-as-data** (color/position/type/probability); trycompai's single activity-spine table + `source` provenance + `last_activity_at` discipline. **BO-10 contribution: WS-26a adds the shared engine seam (`gateway/db.py::get_engine()`, tasks converted as proof) instead of engine 13.** Tickets: **a** schema + feature registration + core API — **BUILT 2026-08-05** (mig `144_crm.sql`, `feature:crm`, `gateway/db.py` seam + tasks converted, `routes/crm/`; **migration 144 applied on prod and `/crm` live as of 2026-08-06**) · **b** **Zoho two-way sync — BUILT 2026-08-05** (branch `ws-26b-zoho-sync`: `list_leads` + `list_deleted` on the read client, the single write client `ingestion/sources/zoho/writer.py` with one grep-asserted caller, mig `145_crm_zoho_sync.sql` (dirty columns + `crm_zoho_tombstones` + `crm_sync_cursors`), `routes/crm/{import_zoho,sync_zoho,broker_handlers}.py`, `crm.zoho_*` broker handlers registered from `main.py`, 80 new hermetic tests). *(Re-scoped 2026-08-05, owner-directed D-CRM-7: "faithful two way sync until we do away with Zoho entirely" — coexistence is bidirectional, not import-once.)* **Measured 2026-08-06: mig 145 is applied on prod and the BACKFILL HAS RUN — 737 orgs / 1,189 contacts / 1,516 leads / 551 deals / 1,909 notes, zero dirty rows, zero unmatched owners; the §7.1 pre-flip curl confirmed the tenant honors RFC-1123 `If-Modified-Since` (304). The PUSH direction has still never run: `CRM_ZOHO_SYNC` ships OFF, nothing has ever written the live Zoho tenant, and enabling the flag or hand-running a push cycle against prod stays OWNER-GATE §6.** WS-1's "no Zoho write path anywhere" clause was corrected in the same change (done-when 6) · **c** UI + the API addendum — **BUILT 2026-08-05** on branch `ws-26c-crm-ui` atop 26a and **merged with b into `ws-26-crm-app` 2026-08-06** (`/crm` app + BFF proxy; the three frontend registration points with `CenterApp` re-typed so `live ⇒ href` is a compile error; `routes/crm/deal_contacts.py` with one-primary-per-deal enforced on the shared `core.link_deal_contact` seam the convert path now also uses — 26b's importer is the one excepted writer and computes `is_primary` in-statement so a backfill can never demote a hand-set primary; `organization_name` on the deal list + board via a derived-table LEFT JOIN; the three review residuals — `?status_id` on a pipeline-less entity → 422, explicit `null` on a defaulted NOT NULL column → 422 not a driver 500, and a hand-edited `lead_name` surviving a name-field PATCH. **Deployed:** migrations 144 and 145 are applied on prod as of 2026-08-06 and `/crm` is live, so live rendering, drag persistence and deep links are owner-verifiable now) · **d** integrations — **audited 2026-08-06 GO-NARROWED and the narrowed slice is BUILT** (branch `ws-26d-agent-crm`): `apps/agents/agent-crm/` (`crm-assistant`, MAF, four READ tools over the existing `/crm` routes carrying the caller's `X-User-Email`, read-only enforced at the transport by a GET-only method allowlist) registered in `_KNOWN_AGENTS` + `_AGENT_REGISTRY` + `agent_registry.json`, plus `"crm"` added to the WhatsApp `_KNOWN_SYSTEMS` allowlist **parse-only** (nothing writes `wa_contacts.entity_ref`, the `crm` context block stays `None`, both pinned by test). **The three held-back items are now DISPATCHABLE — their doc blockers (B3/B4/B5/B7) were closed 2026-08-06 in `crm_app.md` §9.1-§9.3, every anchor read off `origin/main` rather than recalled:** **WS-26d-email** (the timeline join is CALLER-scoped, never record-scoped — it reuses the email app's `_account_scope` predicate, copied into `routes/crm/` rather than imported per D-CRM-4, joins by thread not message, inbound `from_address` only, and needs a new address index at the next free migration number) · **WS-26d-autolead — BUILT 2026-08-08** (branch `ws-26d-autolead`, migration **158** `crm_auto_lead_cursors`). Hook = `routes/email/scheduler_hooks.py::process_new_mail`, the one seam scheduler+manual+webhook all funnel through; the per-message rules loop was considered and REJECTED because a classifier outage there double-fires and history backfills never reach it. ⚠️ **That same seam is ALSO reached by deep resyncs** (2026-08-08 audit blocker G1, closed by PR #402 before the build), which is why the step keeps a per-account THREE-timestamp cursor: `activated_at` (the start of the current ON epoch) makes `received_at > activated_at` the backfill discriminator, `processed_watermark` is the incremental one, and `last_run_at` is the dormancy clock. Without the first, connecting a second mailbox mints a lead per unknown sender across a year of mail, each queued for the live tenant. ⚠️ **Diff review closed two P1s the first cut had, and both needed a running cursor to see:** `activated_at` alone did nothing about an OFF→ON round trip (measured: 27 leads minted for a 27-day OFF window), so a gap in `last_run_at` beyond `REANCHOR_GAP_SECONDS` (3600) CLAMPS the epoch to `now - 1h` and runs the batch anyway — **clamp, not reset**, because a delta re-review measured the reset version discarding the very message that woke the step (this hook fires only when a sync PERSISTED mail, `email_ingestion/scheduler.py:463-472`, so the cycle that detects the gap always carries it), with the bounded residual that the window's last hour is minted; and the watermark advanced over messages that RAISED (measured: 3 candidates, 3 errors, 3 leads lost for good), so it now advances over the successful PREFIX only, with a held cursor reported at WARNING on `sync.auto_lead_stalled` every cycle. **Dormancy reads `last_run_at` and not the watermark deliberately**, though for a narrower reason than first recorded: both freeze on a mailbox receiving nothing (the hook does not fire without new mail), and what separates them is a cycle that ran and found no candidates — which is also the state a deliberate stall holds the watermark in. Unknown-sender check mirrors `_maybe_block_cold`'s two steps and adds a third (no `crm_contacts`/`crm_leads` row with that `lower(email)`) — **the ticket's original `ON CONFLICT DO NOTHING` could not have fired** (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so dedup is that SELECT guard plus in-batch de-duplication, with the cross-invocation race accepted and the UNIQUE index refused. Colleague suppression is TWO gates: `sender_scope` (which fails SAFE to "external", the wrong direction here) and the normalised internal-domain list. The originating message is logged `type='system'` — outside `sync_zoho`'s `type IN ('note','task')` push predicate — with subject + sender in `meta` and **`body` empty**, so the mail's content never leaves the native CRM even though the lead does. The lead goes through `records.create_record`, never raw SQL. **Built, flag `False`, NOT flipped, NOT deployed; the flip stays §6 (b).** 75 hermetic cases; 15 mutants red and reverted (one of them precision-checked) · **WS-26d-write — BUILT 2026-08-08** (branch `ws-26d-write`, **no migration**: every route the four tools call already existed). `request_confirmation` awaited at the top of each tool before any mutating request is built, fail-closed, and the `non_interactive_default` keyword is asserted ABSENT from the whole module rather than asserted != "approve" — pinning the argument rather than the value means a mutant does not get to pick a spelling the fence has not heard of. `_ALLOWED_METHODS` **widened, never deleted**: `{GET, POST, PATCH}`, still checked inside `_request`, with `DELETE`/`PUT` and any `_delete`/`_put` helper still absent, so the check that used to enforce "read-only" now enforces "never destroys". Path fence extended past `ast.JoinedStr` to `.format`/`%`/`+` (the re-review's P2) and — the part that makes it maintainable — **tested against synthetic sources one per idiom**, so "the fence went blind" is a red test rather than a silent gap. Two supervisor rulings landed as built: `update_deal_status` resolves the stage BY NAME inside the tool against `GET /crm/statuses/deal` (no UUID on the LLM surface; an unknown name returns the real lane names), and a lost-type target requires a `lost_reason` resolved the same way against `GET /crm/lost-reasons` — pre-empting the 422 the "close this as lost" demo beat would otherwise hit — with the vocabulary **only ever read, never created**. `create_lead` takes **no `owner_email` argument at all** (the route derives it from the acting user), deleting an LLM-filled identity field from the surface entirely. ⚠️ **One recorded departure from done-when 1**: the invariant asserted is *no mutation before consent*, not *no HTTP before consent* — two tools must read to describe honestly what they are about to do, and every pre-card call being a GET is itself pinned; the two tools that owe nothing to a pre-read are still held to literally zero calls. `@_annotate_risk` is the shared annotation convention and is NOT enforcement; the Action Broker covers the Zoho push, not the native write — the two fail in OPPOSITE directions and are not interchangeable. 76 new hermetic cases + `test_crm_agent.py` 87 → 143; ten mutants run red and reverted. **Built, not deployed.** The push-queue question is CLOSED — **D-CRM-9 (owner, 2026-08-06): agent-originated writes queue for Zoho exactly like human ones** (🟡 remainder; the flip stays OWNER-GATE) · **e** cutover + retirement inventory + **Zoho refresh-token revoke, which executes part of WS-2's standing P0** (🔴 OWNER-GATE end-to-end). Data-visibility departure recorded: org-visible to `feature:crm` holders in v1, `owner_email` is assignment not ACL (D-CRM-3; workflows v1 is the precedent) — revisit at WS-14 `group:` grants / colleague #1. **Pipeline blueprint added 2026-08-07 (spec §5.1) after the owner's first live board session found lanes out of order and imported stages at 0% probability — root cause: the importer appends unseen Zoho stages past the seeds at probability 0, and 144's seeds renamed Zoho's defaults so name-match missed them; the admin API that could fix it is headless.** New tickets: **f** pipeline truth + settings UI — **BUILT 2026-08-07** (branch `ws-26f-pipeline-truth`, **no migration**: 144 already carried `position`, `probability`, `type`, `closed_at` and `expected_close_date`). `routes/crm/stage_metadata.py` = `POST /crm/import/zoho/stages`, floor `admin:access:manage`, **dry-run by default** and `?apply=true` to write; **>1 pipeline STOPS before the DB is opened** (D-CRM-11); f4's `closed_at` proxy is a direct UPDATE that bypasses `mark_dirty_on_update`, asserted statically against the statement text AND the module's call graph because the shared fake writes only what a SET clause names. Two settings readers on the Zoho read client with `ZohoScopeError`/`ZohoApiVersionError` so **no-scope, no-data and no-such-endpoint are three different reported outcomes** — and no-scope is the *expected* first answer, since the tenant's refresh token was never minted with `ZohoCRM.settings.*`. D-CRM-10's clamp landed in `admin.py::_validate_status` reading the ROW rather than the payload (a PATCH naming only `type`, or only `probability`, contradicts the rule only in combination with what is stored). `?tab=settings` is the headless-API fix and needs no Zoho token at all. **Nothing has been run against the tenant — dry run included.** · **g** forecast & funnel reports off `crm_status_changes` — **BUILT 2026-08-07** (branch `ws-26g-reports`, **no migration**: 144's `crm_status_changes` already carried every column). `routes/crm/reports.py` = four read-only endpoints (`/crm/reports/{pipeline,funnel,win-loss,owners}`) on the shared gated router, plus `?tab=reports`. **`WEIGHTED_SQL` lifted into `core.py`** (pipeline re-exports it) so the forecast formula has ONE definition, and `core.status_wire` absorbed two duplicate status projections instead of gaining a third. **The cross-language parity mechanism is minted here, not inherited** — the `priority.ts ⟷ priority.py` "precedent" is two hand-kept mirrors joined by a comment with no shared fixture anywhere: `tests/fixtures/crm_weighted_parity.json` (new directory) is read by BOTH pytest (through the emitted SQL, whose expression `_crm_fakes._WEIGHTED_SUM_RE` parses out of the statement text) and vitest (through `board.ts::weightedDeal`/`weightedRows`). ⚠️ **The funnel is defined against what the log RECORDS, not what its name suggests**: `crm_status_changes` logs transitions only, so all 551 imported deals have zero rows and "entered" is a VISITED-SET union (`from_status`, `to_status`, and the deal's current stage) — a `to_status` count reports an empty funnel for the whole live board; dwell is grouped by **`from_status`** (the stage being LEFT, the opposite key from the naive reading); the log stores NAMES, so a renamed lane orphans its history and orphans are REPORTED in `unmatched`, never dropped; and NULL `closed_at` — every imported closed deal until f4's owner-gated backfill runs — falls outside the trailing window, with the count reported so a 0% win rate is explicable rather than mysterious. The lost-reason breakdown carries a NAMED unattributed bucket: the earlier "complete by construction" claim is FALSE, since the importer bypasses both gates and `lost_reason_id` is `ON DELETE SET NULL`. **No `GROUP BY` is emitted** — the ticket asked for the choice to be stated: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would stop BEING the expression the fixture and the fake read. 47 hermetic cases + 20 vitest + the 14-row shared fixture on both sides, 5 mutants red. **Two findings:** the `entity_type='deal'` mutant initially SURVIVED (the funnel keys through deal ids, so a lead row is excluded twice over) — closed with a row stamped `lead` against a DEAL's id, which is realistic precisely because `entity_id` has **no FK**; and `_crm_fakes`' `lower(col) = :param` reader matched a NULL column, which SQL never does, making the unassigned-owner bucket count rows its own aggregate never summed · **h** stage entry-requirements + rot badges (after f2) · **i** merge/bulk/CSV/saved-views (🟡 spec-thin, audit-narrow first). **Demo path (2026-08-07, spec §9.0): full chain + all gates intact, tickets re-sequenced not thinned; f gained f4 — imported won/lost deals have no `closed_at` (importer never stamps it), backfilled from Zoho `Closing_Date` as a labeled proxy via a direct UPDATE that MUST bypass dirty-marking or ~500 no-op pushes queue for the live tenant.** | | **WS-27** | **Projects app — native project management + ClickUp retirement** *(minted 2026-08-05)* | `specs/project_management_app.md` | ✅ **a + b + d + e + i BUILT 2026-08-06 · j + k + l + m + n BUILT 2026-08-07** · 🟢 f dispatchable · 🟡 c gated · 🟡 h sequenced | Research pass 2026-08-05: `Paca-AI/paca` v0.11.0 (Apache-2.0 — **patterns adopted, no code translated**; findings + the adopt/adapt/refuse table live in `specs/paca_pm_research_2026-08.md`, reference-only), plus a full-tree ClickUp sweep. **ClickUp today is TWO independent systems** — the Phase-0 graph mirror (read-only, shallow) *and* the per-user Tasks-app connector with a **live broker-gated write path** — so leaving ClickUp is coexistence-sync-then-invert, **not** WS-26's import-and-retire; the constraint-8 inversion is staged and recorded in spec §7. Spine: Paca's two-self-FK hierarchy (departments→projects→subprojects→tasks→subtasks as `pm_projects` + `pm_tasks`, types-as-data with the Epic-root rule), statuses-as-data with a semantic `category` (D-CRM-2 convergence), per-view fractional ordering (`pm_view_task_positions` — what lets People-Center and Center-slice boards order the same task differently), and a single activity spine. **First data-scoped app:** `pm_project_grants` on the shipped `email\|group:\|org` vocabulary (D12; sibling of C1's D13, which is unchanged), 404-not-403, and the full-portfolio view gives D14's zero-consumer `data:org:read` its **first consumer**. **Three owner answers recorded 2026-08-06 as D-PM-8/9/10, and two of them changed the build:** **D-PM-8** no portfolio/program layer — grants are the only grouping axis, a cross-department project simply carries several (a `pm_programs` table stays purely additive if wanted later); **D-PM-9** agent edits to ClickUp-linked tasks are treated **exactly like human edits** (*the agent proposed queueing agent-originated pushes for approval and was overruled*) — so during coexistence a mistaken agent edit reaches the live workspace with no human in between while `ACTION_BROKER_ENFORCE` is off; bounded by attribution (`agent:`), timeline-reversibility, and the fact that the enforce flip converts the whole class to queue-on-approval. Read D-PM-9's Cost paragraph before building WS-27f; **D-PM-10** ClickUp Spaces map to Centers **explicitly**, from agent-proposed suggestions (assignee-overlap → name match → EVAL-LOCKED content classification), owner-confirmed, applied as `group:` grants — and an **unmapped Space still imports in full with no group grant**, staying reachable in `/projects` for `data:org:read` holders and its assignees. This supersedes the "pilot vs all Spaces" framing: scope is now a per-Space decision the plan step surfaces, so a pilot and a full import are one code path. Tickets: **a** schema + `feature:projects` both sides + core API on the `gateway/db.py` seam — **BUILT 2026-08-06** (mig `146_projects.sql`, `routes/projects/` with zero `create_async_engine` calls, 115 hermetic cases + 5 mutants measured red; **not deployed — the migration has not been applied anywhere**) · **b** ClickUp org importer **+ the Space→Center mapping plan** — **BUILT 2026-08-06** (`routes/projects/mapping.py` + `import_clickup.py`; `POST /projects/import/clickup/plan` proposes and writes nothing, `POST /projects/import/clickup` applies the confirmed mapping; 25 hermetic cases, 4 mutants red incl. *applying the suggestion instead of the confirmed mapping*; **neither endpoint has been run — prod execution stays OWNER-GATE, §6**) · **c** two-way coexistence sync — three-way field merge, conflicts logged to the timeline (🟡 **blocked on WS-1's BO-1a + BO-1b, named prerequisites**; enabling push is OWNER-GATE) · **d** UI + Center (app + scope) projections, no forks — **BUILT 2026-08-06** (`src/app/projects/` tree + board + list + task panel + timeline, BFF proxy, nav/access registration, all six Centers linking at the SAME `/projects` path and differing only by `?center=`; 34 vitest cases incl. a registration fence, 6 mutants red) · **authoring landed 2026-08-06** — d shipped a UI that could read and drag but never **create**, so a member could only work with rows a ClickUp import had put there: new department / subproject (from the node, where the parent already is), new task (status not sent — the API picks the project's default), subtask from the panel, and **assignees as chips where an agent and a person share one field** (`lib/assignees.ts`, 17 vitest cases, 7 mutants red). That last one is where D-PM-4 stops being a schema note and is the precondition for WS-27f's dispatch being reachable at all · **e** the personal lens — **BUILT 2026-08-06** and **its shape changed**: `D-PM-6` was revised (owner-directed — *"the personal task manager should be a proper extension … a cohesive whole"*) from a mirror into **one store**. `pm_tasks` is THE task table; private work is a personal project (`pm_projects.personal_owner`); the GTD overlay is **per-member** (`pm_task_personal`, mig `147_projects_personal.sql`) so two assignees can hold different dispositions. Assignment is no longer a sync — the inbox row IS the project row, and completing it there moves the shared status. 31 hermetic cases, 6 mutants red. **Its surface landed the same day** — "My work" above the project tree in the SAME app (`components/MyWork.tsx` + `lib/mywork.ts`, 17 vitest cases, 7 mutants red): capture-first, four work lanes that render even when empty, untriaged counted in the header (the Weekly Review's question, answerable only because dispositions are derived not stored), and a completion checkbox that moves the **shared** status. e had shipped API-only, so the cohesion the revision bought was true in the schema and invisible to a member. One repair it forced: `TaskPanel` read the *selected* project's statuses, wrong for a task opened from My work — now resolved from the task's own root project. **Cost accepted: `gtd_items` becomes legacy and WS-27h retires it** · **f** automation + agent dispatch — **BUILT 2026-08-06** (`routes/projects/automation.py` + `agent_dispatch.py`, the `pm_task` node type in `workflows/engine/`, `PM_EVENT_TOPICS` served by the catalog; 34 hermetic cases, 10 mutants red). Both halves of `workflows_app.md` §13 — **U1** the task-mutation node and **U7** dispatch. The engine imports a transport-free SERVICE, not a route, so an automation's edit is indistinguishable in validation from a human PATCH and lands the same timeline row; **status is named, never keyed** (a graph pinned to one project's status UUID could only automate that project); a `pm_task` node is deliberately **not** write-class (the approval gate is for outward writes — now pinned by a test rather than true by accident); and "already in target state" is asserted to issue **no UPDATE at all**, because `update_row` stamps `updated_at` and a redundant write is invisible in a diff while making a task look freshly touched. Assignment dispatches from an event SINK beside the workflows dispatcher, so a broken agent cannot fail the act of assigning somebody a task, and the handoff activity is committed BEFORE the run starts. **Engine defect found and fixed:** `resolve_value` keeps an unresolvable `{{ref}}` as-is at run time by design and `{{trigger.missing}}` passes the publish gate because its *root* is legal — the literal would have reached Postgres as a would-be uuid and returned "Task not found", pointing the maker at the wrong thing · **i** attachments — **BUILT 2026-08-06** (mig `150_projects_attachments.sql`; 25 cases, 10 mutants red): one file store (`gtd_attachments` reused, upload rules imported) with a thin `pm_task_attachments` join that carries the ACCESS decision, so a file is readable by whoever can see the task rather than only its uploader; no attach-by-id endpoint exists, because naming an arbitrary attachment id would let a caller attach somebody else's private capture to their own task and read it back. **Caught before shipping:** the projects BFF proxy re-serialised every POST as JSON, so a multipart upload would have reached the gateway with NO FILE while still answering 201 · **j–r** the rest of the ClickUp-parity gap, measured against the built tree and sequenced in spec §11 (attachments, notifications/@mentions, filters+saved views, custom fields, tags, bulk edit, recurring, dependency UI, calendar, search) — all 🟢, and **n (bulk edit) gates g — and n is now BUILT (2026-08-07), so that dependency is SATISFIED**: an import that cannot be re-triaged in bulk is one somebody abandons halfway, leaving two live systems, which is the state the retirement exists to end. g itself stays 🔴 OWNER-GATE for its own reasons (§6); this changes the prerequisite, not the gate · **g** cutover + retirement inventory (both ClickUp systems) + token revoke + the root-`AGENTS.md` constraint-8 amendment (🔴 OWNER-GATE end-to-end) · **h** `gtd_items` retirement — the cost D-PM-6's revision accepted: union read, row migration into `pm_tasks` + `pm_task_personal`, then `items.py`'s 27 owner-scoped predicates retire with the table they scope (🟡 after e; the data move is 🔴 OWNER-GATE — it rewrites the owner's live task store).. **j BUILT 2026-08-07** (mig `152_projects_notifications.sql`, `routes/projects/notifications.py`, the header bell + mention picker; 39 hermetic + 27 vitest cases, 10 mutants red): closes §11.2's second gap — *"assignment is silent"*. **Three rules decide who hears**, each the whole reason for a rule: never the actor (a bell that pings you about your own click gets muted, and a muted bell notifies nobody about anything); never an agent (they are handed work by the WS-27f dispatch sink, so a row addressed to one sits unread forever inflating a badge nobody can clear — enforced in Python AND by a CHECK); and **never somebody who cannot open it**, which is the security property: the notification carries the task's TITLE, so delivering one outside the grant closure leaks it and lands them on a 404. That third rule needed `resolve_visibility_for`, which answers for a THIRD PARTY — `resolve_visibility` reads a `UserContext` and the recipient of a mention has no request in flight — by reading the tables `/auth/me` reads and handing them to the **real** `build_access`, so wildcards and allow/deny overrides resolve identically on both paths. Notifications are written **inside the transaction**, not emitted on the bus: `emit` swallows failures by construction so a broken workflow can never fail a task edit, which is right for agent dispatch and wrong here. A mention is an **address, not a name**, because 148 dropped `UNIQUE(name)` — `@Priya` has no answer. **Two bugs found on the way in, both shipping at the time:** every project-task file upload was answering **422** (`ACTIVITY_TYPES` never learned 150's `attachment`; all 25 attachment tests passed because they monkeypatch `record_activity` — the seam under test was mocked out), fixed with two tests that READ the migrations; and `/projects?task=` did nothing, though the People Center has linked there since WS-28b. **WS-27b's UI BUILT 2026-08-07** (`components/ImportClickUp.tsx`, `lib/importPlan.ts`; 18 vitest cases, 3 mutants red): the importer shipped with WS-27b and was **unreachable from the product** — the empty state named "import a ClickUp workspace" and no control anywhere did it, so a new install stayed empty and the only route to real data was curl. Three steps, only the last of which writes: Preview (`/plan`, reads the tenant) → Dry run (`dry_run:true`, exercises the flattening) → Import. **The mapping stays the owner's act (D-PM-10)** and the UI is built so it stays one: the suggestion is pre-filled and shown beside its confidence IN WORDS ("a guess — check it", not `0.45`, because a bare number invites acceptance without looking), and a CONFIRMED mapping always beats a fresh suggestion so a re-run never silently re-maps a Space somebody already ruled on. Unmapped Spaces are a notice rather than a blocker, matching the importer. ⚠️ The gate is unchanged and is now exactly one click: building this was agent-safe, pressing Import is the owner's act, and no agent has run either endpoint against production. **The Tasks-app mirror path BUILT 2026-08-07** (`routes/projects/import_tasks.py`, `POST /import/from-tasks`; 43 hermetic cases, 7 mutants red) — owner-directed: *"just show up all the data that is there in the Tasks app inside the Projects app"*. A SECOND importer rather than a flag: the ClickUp one needs a live token, spends LLM budget and demands a Center mapping BEFORE anything is written, which is backwards for "show me my work today". This reads `gtd_projects`/`gtd_items` — the mirror already on the box — so no API call, no token, no model spend, and it works when the connector is stale. One named department, with the real ClickUp shape beneath it — **Space → Folder → List** rebuilt as projects, each carrying its own `clickup_id`/`clickup_kind`, so promoting a Space node is how the department split happens later. **Verified against a real Postgres** (full migration set + seeded mirror), which found THREE defects the hermetic suite could not: `gtd_projects.space_id` is LOCAL-only so the Space was always NULL; `pm_projects` has no `clickup_snapshot` column, so the first real click would have 500'd (this shipped in #393 and was fixed before anybody pressed it); and the preview under-counted 4 vs 7 because container nodes were only tallied on the write path. The root IS org-granted (same act as `create_node`, narrower than bulk-granting a tenant). Pinned: nothing outside `pm_*` is written; only `source <> 'LOCAL'` rows are read (a personal capture must not be published to a shared board); the provider's own status names are kept; an orphaned task is COUNTED, not dropped. **Mutation found a real gap**: deleting `dry_run` from the write guards left every test green, because on a FIRST dry run `root_id is None` blocks the write anyway — on a SECOND one the department exists and `dry_run` is the only protection. That is the realistic case (preview → import → preview again) and it now has its own test. **k BUILT 2026-08-07** (`routes/projects/filters.py`, `lib/grouping.ts`, `components/FilterBar.tsx`; 34 hermetic + 24 vitest cases, 13 mutants red, and 23 checks against a REAL Postgres) — closes §11.2's third gap, the one whose name was the sentence *"my open bugs in Ops, grouped by assignee"*. **ONE filter builder serves both the list endpoint and saved views**, because a saved view is nothing but a stored set of these filters and two implementations would drift — a *saved* view showing a different set than the same filters typed by hand is the one thing it may not do, so a test compares the two outputs directly. **Every filter is a WHERE clause**: paging happens in SQL, so a filter applied in Python after `LIMIT` returns short pages, and *"page 2 is empty but there are 40 more"* is a bug people work around for months instead of reporting. **`overdue` means past due AND still open** — a finished task with a past due date is done, and permanent red is how a board teaches people to ignore red. **An unknown category is a 422 naming the five real ones**, not an empty board a client reads as "this project is empty". Unknown config **keys** are DROPPED while a bad **value** falls back, because those are different failures: a view is a preference written by an older client, so refusing one over an unrecognised key would make every deploy a migration of everybody's saved views, whereas rendering still has to produce something. On the board, **a task with two assignees appears in BOTH columns** (it is both people's work; picking one hides it from the other, so the header counts tasks not group sizes), **empty status lanes are kept** while every other grouping drops empties (a missing "In progress" column reads as "no such state", not "nothing in progress"), and **dragging is offered only when the columns are statuses** — a drop writes the field the columns represent, and status is the one that is a plain PATCH, so a card that can be dragged into a column which cannot accept it and snaps back is worse than an honestly static column. `toConfig` is deliberately NOT `toQuery`: a query string carries only text so `toQuery` writes `"true"`, and `fromConfig` refuses a string where a toggle belongs, so a view built from query shape would come back with every toggle silently cleared. The project's **order-bearing board is withheld from the chips** — `tree.py` seeds one `board` view per project and it owns every `pm_view_task_positions` row, so offering its ✕ would offer to delete every hand-arranged position; saved views sit at position 300 above the seeded pair and `orderBearingView` is one function used by both the drag handler and the delete guard. **A FIFTH live bug, found the same way as the previous four:** `due_before` was `CAST(:due_before AS timestamptz)` with the raw query-string value — asyncpg infers the parameter's type FROM that cast and then refuses to encode a `str`, so the query never reached Postgres and **`?due_before=…` answered 500** while the hermetic fake, which agrees with whatever SQL it is handed, stayed green. `parse_when` parses on this side and binds a real `datetime`; garbage is a 422 that says what was expected; a naive value is read as UTC rather than inheriting the connection's TimeZone. Two tests — one on the bound value's TYPE, one refusing any `CAST(:param AS timestamp…)` anywhere in the builder — so the next `after=` filter written the obvious way fails in CI instead of in front of a member. **l BUILT 2026-08-07** (mig `155_projects_custom_fields.sql`, `routes/projects/custom_fields.py`, `lib/customFields.ts` + the panel block and the Fields dialog; 47 hermetic + 36 vitest cases, 23 mutants red, 35 checks against a REAL Postgres) — ClickUp's signature feature, and the shape §5's non-goals already recorded as the additive path: **definitions in a table, values denormalised onto the task as JSONB keyed by `field_key`**. NOT a row per (task, field): that is the textbook EAV answer and costs a join per field on every board paint — five fields across two hundred imported tasks is a thousand rows to gather and re-pivot, per render — whereas the JSONB column arrives with the task for free. **The cost is stated rather than discovered**: a value is not referentially tied to its definition, so the DATABASE cannot stop a key no definition owns from being written, and that guarantee moves into Python. Hence the validation IS the feature: an unknown key is a **422 not a silent drop** (a typo that no-ops looks exactly like a save); a patch **MERGES** (a client that knows three of five fields must not wipe the other two — and an older client, or an automation written before a field existed, is precisely that client); an explicit **null CLEARS the key** rather than storing a null, because it is the only way to express "unset this" and a stored null makes "never filled in" and "deliberately emptied" one value in every filter; and **`true` is not the number 1** — `isinstance(True, int)` is True in Python, so the coercers are one-per-type in a dispatch dict specifically so the boolean check can never drift below the number one. **The deliberate departure from Paca:** its research notes record "deleting a definition does not clean task data" as an accepted cost; it is NOT accepted here, because a key left in the JSONB is invisible — no definition means no column, no form row, no filter — until somebody recreates the name and every old value resurfaces carrying the new meaning. The cleared count is reported (R7/R8). Two things a definition may not change once values exist, both because the stored values would stop meaning what they say: **`field_key` is never editable** (it is the identity every value is filed under) and **`field_type` is a 409 naming the count** (text→select cannot re-interpret what is already written); dropping a select option some task holds is refused the same way, adding one is free, and the UI shows the derived key while the name is still being typed since that is the last moment anybody can change it. **Custom fields are REVERTIBLE, which is what makes them first-class:** `patch_task` folds a custom edit into the SAME `field_change` activity under `custom.` rather than inventing an activity type — `record_activity` refuses a type the CHECK does not list, the trap that made every attachment upload answer 422 — and revert restores by **merging onto what the task holds NOW**, never writing back the whole object, since another field may have been edited since and replacing the blob would silently undo that too. **A bug the ticket's own tests caught before it shipped:** `changedValues` compared a form's boolean against a `null` baseline, so a task with an unanswered checkbox sent `open:false` on EVERY save and posted a timeline entry for an edit nobody made — a checkbox has no "unset" state to render, so `false` is its baseline. **And a fence that was quietly a subset check:** `test_projects_routes` asserted a LIST of mounted paths, which catches the module somebody remembered to add a path for; it now also reads the package directory and asserts every module declaring a `@router` route is imported by `__init__.py` — the C1 trap where a missing import mounts nothing while every direct-call test still passes. Verified by deleting the import and watching it fail. **m BUILT 2026-08-07** (mig `156_projects_tags.sql`, `routes/projects/tags.py`, `lib/tags.ts` + the panel picker, the filter row, a `tag` board axis and the Tags dialog; 31 hermetic + 37 vitest cases, 16 mutants red, 30 checks against a REAL Postgres) — the row the research notes left open ON PURPOSE: `paca_pm_research_2026-08.md` row 13 REFUSED Paca's model (*"a bare jsonb string array on tasks. No registry, no colors, no rename/merge — the weakest part of Paca's model"*) and §5 shipped `pm_tasks.tags TEXT[]` in its place with a registry named as additive later. **The array STAYS** — a join table would add a row per tag per task and a join to every board paint, to buy referential integrity this app enforces in one place, whereas the array arrives with the task and its GIN index (146) already answers "tagged X". What the registry buys: **one spelling per tag** (identity is case-INSENSITIVE via a unique index over `lower(name)` so two racing requests cannot create both, display is case-PRESERVING, and the task's array stores the REGISTRY's form — which is what makes "filter by bug finds all of it" true rather than aspirational, and what lets a rename be one statement); **rename**; **merge**; and a **colour**. **Applying an unregistered tag REGISTERS it** — refusing would make tagging a two-step errand (leave the task, create the tag, come back), which is how tagging gets abandoned, and an abandoned tag set is worse than a messy one. **The cost is stated: every typo becomes a tag** — which is exactly why merge is here and is not optional, and why the picker SHOWS the moment of creation rather than minting silently. **A rename onto an existing name is a 409, not a silent merge**: different operations, different outcomes, and one of them destroys a tag — quietly doing the destructive one because the names collided is what stops people using a rename button. **A task carrying BOTH tags ends a merge with the target ONCE** — the case that is easy to get wrong, and getting it wrong leaves a duplicate that renders twice and survives the next merge too; `merged_tags` is pure so that case is asserted directly, and the rewrite runs over affected rows in Python rather than as an `array_replace`, which would leave the duplicate. **TWO tag filters** because both questions get asked and neither answers the other: `tags` is ANY (`&&`) and `tags_all` is ALL (`@>`); collapsing them would silently pick a meaning, and with three tags the answers differ by almost everything. On the board a task with three tags appears in three columns, the same honesty as two assignees appearing in both theirs. **The migration backfills AND rewrites data, deliberately and narrowly:** `tags` has been on `pm_tasks` since 146 and the import path writes them, so an empty registry beside a tagged corpus means the first rename finds nothing; the winning display form is **the spelling people actually use** (most frequent, ties broken deterministically — `min()` alone would canonicalise 400 "Bug" to a single stray "BUG"), and task arrays are then made to agree, only ever swapping one CASING of a tag for another, with the count reported in a NOTICE. **A bug the live run caught in that block:** the canonicalisation used the implicit-comma `FROM pm_tasks t, unnest(t.tags) ... LEFT JOIN` form, where the LEFT JOIN binds only to `unnest(...)` and `t` is not in scope for its ON clause — the migration aborted with "invalid reference to FROM-clause entry for table t". Rewritten as `CROSS JOIN LATERAL … WITH ORDINALITY`, which also fixed a second problem the first version would have shipped: `array_agg(DISTINCT ...)` sorts by its own expression, so every task's tag list would have come back alphabetised. **n BUILT 2026-08-07** (`routes/projects/bulk.py` → `POST /projects/tasks/bulk`, `lib/selection.ts`, `components/BulkBar.tsx` + checkboxes on board and list; 35 hermetic + 32 vitest cases, 16 mutants red, 34 checks against a REAL Postgres; **no migration**) — **the ticket §11.3 names as gating g, so that prerequisite is now satisfied.** It **reuses `automation.apply_task_patch` rather than growing a second writer**: that service exists because WS-27f needed an edit indistinguishable in validation from a human PATCH, and a bulk endpoint with its own field handling would be a third opinion about what a task edit is. Tags go through the same registry the panel uses, because the registry cannot be true if bulk is a second door into the array. **Status is named, never keyed — load-bearing here rather than stylistic**: a selection spans projects and a status id belongs to one root, so `status_id` for fifty tasks across three projects puts two thirds of them in a lane that is not theirs; it is a 422 with its OWN message, because it is the mistake somebody makes by copying a single-task PATCH body and "unknown field" would not explain why the thing that works on one task is refused on fifty. **Assignees and tags are ADD/REMOVE, never SET** — "assign these to Priya" means ALSO Priya, and a replace wipes every individual assignment the fifty tasks already carried; the destructive spelling is absent rather than discouraged. **Shape validated once, outcomes per task**, because those are different failures: an unsettable field is the same mistake for all fifty (422 before any write), while a status name present in one project and absent from another is a fact about THAT task — failing the batch for it makes a mixed selection unusable, which is precisely the selection somebody makes after an import. **An invisible task is SKIPPED, not an error** (R5: per-id reporting says exactly what a per-id 404 says, and aborting would let a caller probe for existence). **One transaction** — a re-triage that half happened is harder to recover from than one that did not, because nobody can tell which half. **Re-asserting a value is not a change** (`moved_people` is pure so the claim is asserted directly): fifty tasks already Priya's would each gain a timeline entry saying nothing, and a count nobody can trust is worse than no count. **ONE notification per person per batch, not one per task** — being handed fifty tasks should ring once and say fifty; fifty bells is a bell people turn off, which is WS-27j's own argument applied to the case that would have broken it. In the browser the selection is **pruned whenever the filter changes** (select forty, narrow to three, press Done must not act on thirty-seven nobody can see), a shift-click ranges over the board's ON-SCREEN order, a two-assignee card drawn in two columns counts once, and the outcome line names every category including the boring ones. **AND IT UNCOVERED A SHIPPED WS-27j BUG** (spec §11.12): `notifications.deliverable` probed only `project_clause('t.root_project_id')` while `core.task_visibility_clause` — whose docstring warns a second implementation "would drift the moment one is edited alone" — carries TWO ways in. So (1) anybody assigned work in a project they hold no grant on was judged undeliverable: they could OPEN the task, the assignment notified nobody, and the response told the assigner they could not see it — the silent assignment WS-27j exists to end, still open for the most common case in a grant-scoped app; and (2) scoping to `root_project_id` ALSO missed a grant made on a SUBPROJECT — the old test asserted that scoping on the argument that "probing `project_id` would miss a grant made on an ancestor", which is BACKWARDS, as a real-Postgres run showed: the closure is recursive and expands DOWNWARD. Fixed to use the shared clause; the test that encoded the wrong reasoning now records why it was wrong. **A fake collision the fix exposed:** `FakeDB` matched the rule-3 probe by substring, and the new clause embeds both `pm_task_assignees` and the closure's `UNION` — exactly the AUDIENCE branch's fingerprint — so `deliverable` got a list of assignees where it expected a visibility answer and four tests failed for a reason unrelated to the code under test. A fake that dispatches on substrings needs fingerprints that are SPECIFIC, not merely present | | **WS-28** | **People Center — directory, org chart, and the assignment seam** *(minted 2026-08-06)* | `specs/people_center_app.md` | ✅ **a + b BUILT 2026-08-06 · b-write BUILT 2026-08-07** · 🟢 c–e dispatchable · 🔴 f owner-gate | Scope owner-set 2026-08-06: **directory, skills, org chart, capacity, seats/roles — exactly what assignment and planning need**; leave/onboarding/hiring are named as later phases so their absence is a decision. **The fact this spec exists to settle:** there are TWO people stores and that is deliberate — `app_user` answers *can they sign in and what may they see*, `gtd_people` answers *who are they and what can they do*, and the directory must include people with **no login** (contractors), which is why the Projects app's assignee is a plain string. They join on lowercased email, and **P-1 fixes that join before it is relied on**: migration 49 made `name` UNIQUE and left `email` unconstrained, so today two rows may share an address and an email→person join is ambiguous. Surfaces: directory (honouring WS-24 N4's HR projection, with a *restricted* empty state distinct from *none*) · person page · org chart from `manager_id` with a Center overlay that **shows** department/group mismatches rather than smoothing them · capability search over stated skills → résumé evidence → the existing `capability_embedding`, which **suggests and never assigns** · seats & roles matrix (read + propose; applying a membership change stays owner-gated per §6 (d)). Closes WS-13's outstanding *People directory read view* item. Tickets **a** key-shape fix (🟢) · **b** directory + person page (🟢) · **c** org chart (🟢) · **d** capability search (🟢, ranking EVAL-LOCKED) · **e** the Projects seams — directory-backed assignee picker listing agents and directory-only people, capacity derived from open assigned tasks (🟢) · **f** seats & roles writes (🔴 OWNER-GATE).. **a BUILT 2026-08-06** (mig `148_people_key_shape.sql` + `scripts/import_hr_people.py`; 22 cases, 11 mutants red, 1 equivalent): `UNIQUE(name)` dropped, partial unique on `lower(email)`, status CHECK. **P-1 did not name its own consequence** — the HR importer upserts `ON CONFLICT (name)` and would have failed outright, so a `source_key` (`:`) carries the upsert instead, backfilled BEFORE the constraint is dropped while `name` is still distinct. **Neither new constraint may block a deploy** (main was bitten twice this month): a duplicate address is quarantined into a new `email_conflict` column with a deterministic winner rather than failing `CREATE UNIQUE INDEX`, and the status CHECK is added `NOT VALID` then validated in a guarded block, so an unanticipated legacy value leaves a NOTICE instead of stopping the deploy. ⚠️ `schema.generated.sql` NOT refreshed — needs a live DB; regenerate on the first deploy that applies 148. **b BUILT 2026-08-06** (mig `149_people.sql`, `routes/people/`, `src/app/people/`; 32 hermetic + 28 vitest cases, 11 mutants red): `/people` directory, person page with all four panels, and the People Center's "Directory & org chart" sub-app flipped live — closing WS-13's outstanding read view. **Its own feature slug**, not `feature:tasks`: a manager who needs the org chart should not be handed the personal GTD task manager to get it. The gate is new but the HR **projection is imported** from `tasks.core` and a test asserts the function's *identity*, since two answers to "may this caller see skills" are two answers waiting to drift. Three filters (the `q` skills clause, `skill`, `has_capacity`) are dropped without `admin:members:read` so search cannot become an oracle for the hidden field — and the response carries `hr_visible` so the UI says "restricted" rather than leaving a blank strip to read as "nobody filled it in". Load is **computed from open assigned tasks** and carries `unestimated`, because a bar built from the estimate sum alone shows somebody holding thirty un-estimated tasks as completely free. The work panel is scoped by the **viewer's** grants and answers `available:false` without `feature:projects`. **Registration is FIVE places, not four** — the fifth is `test_org_access_enforcement.GATED_ROUTERS`, hand-maintained, where an absent router is unchecked rather than passing; also added the named `test_projects_is_registered_on_both_sides` that WS-27a never wrote. **b-write BUILT 2026-08-07** (`people/components/PersonEditor.tsx`, `people/lib/form.ts`, `people/lib/write.ts`; 26 hermetic + 23 vitest cases, 7 mutants red): closes the regression the 2026-08-06 scope narrowing opened — deleting the tasks app's People view took `PersonEditor` with it, and with it the only UI for creating a person, editing skills and uploading a résumé. The GET-only `/api/people` proxy is **unchanged**: the writes go to `/api/tasks/people`, where they have always lived. Controls are absent rather than disabled, driven by a new **`can_manage`** flag on the reads — hide-rather-than-disable is impossible unless the read tells the UI, and the alternative is drawing the button and letting the click find a 403. **Restoring it turned up three ways migration 148 had already broken the write routes**, each a 500 in front of an admin rather than a test failure: the status vocabulary moved (49's `inactive`/`on_leave` vs 148's CHECK) and is now ONE tuple shared by filter, facets, select and validation; `create_person` still refused a duplicate NAME, preserving precisely the behaviour 148 dropped `UNIQUE(name)` to remove; and nothing checked the address 148 made unique, so a duplicate was an `IntegrityError` instead of a 409 naming the other row. **The lesson recorded in spec §10:** a migration that changes a table's shape has to be walked against every route that WRITES it — the read routes were built after 148 and were correct by construction, the write routes predated it and were never revisited | @@ -652,7 +652,14 @@ service is down, for more than an hour: the cursor RE-ANCHORS, the gap's backlog mints nothing, and the cycle says so at WARNING on `sync.auto_lead_reanchored`. That guard was added by diff review after an OFF→ON round trip was measured minting **27 leads for a 27-day OFF window**, each pushing unattended into the live tenant — -so turning the flag off is genuinely a stop, not a pause that accumulates; **(3)** +so turning the flag off is genuinely a stop, not a pause that accumulates. ⚠️ **With +one bounded exception the owner should expect: mail received in the FINAL HOUR before +the flag goes back on IS minted** (up to one capped batch of it). That is deliberate — +the re-anchor clamps the epoch one hour back rather than resetting it to now, because +this step only runs when a sync persisted mail, so the cycle that detects the gap is +always the cycle carrying the message that woke it; resetting would drop that message +every single night. If the OFF window's tail matters, flip the flag on at a quiet +moment; **(3)** the accepted residual is that two concurrent syncs of one account can double-mint one visible, hand-deletable duplicate (a UNIQUE index on `crm_leads.email` is refused: 1,516 imported rows may already carry duplicates, the migration-148 shape). One diff --git a/apps/services/gateway/AGENTS.md b/apps/services/gateway/AGENTS.md index 895a7f78c..7a3a3e822 100644 --- a/apps/services/gateway/AGENTS.md +++ b/apps/services/gateway/AGENTS.md @@ -54,7 +54,7 @@ webhook receivers, OAuth callbacks, and the Control Plane API. - ⚠️ **`reports.py` (WS-26g) — read-only, and the funnel is defined against what the log RECORDS rather than what its name suggests.** `GET /crm/reports/{pipeline,funnel,win-loss,owners}`; no write, no Zoho call, no flag, no migration. `WEIGHTED_SQL` moved into `core.py` beside `WEIGHTED_TYPES` when this became its second consumer (`pipeline.py` re-exports it, so no caller moved) — a second copy would defeat `_crm_fakes._WEIGHTED_SUM_RE`, which reads the expression OUT of the statement text precisely so a drifted formula changes the tests' answer; `core.status_wire` absorbed `admin`'s and `pipeline`'s duplicate status projections for the same reason. ⚠️ The owner leaderboard's bucket key (`.strip().lower()` in Python) and its aggregate predicate (`lower(trim(owner_email))` in SQL) are one normalisation written twice and must stay byte-consistent: while the SQL lacked `trim()`, a padded address split a bucket the tally had already merged, so the leaderboard under-reported an owner, dropped a deal into no bucket at all, and still said `omitted: 0`. Four properties are load-bearing. **(1)** `crm_status_changes` logs TRANSITIONS only — `create_record` writes no row and the importer writes none — so all 551 imported deals have zero rows and a deal's first stage is never a `to_status`; "entered" is therefore a VISITED-SET union (`from_status`, `to_status`, and the deal's CURRENT stage), and dropping that last term reports an empty funnel for the whole live board. **(2)** Dwell is grouped by **`from_status`**, the stage being LEFT — `to_status` would label every measurement one lane too far on, plausibly. **(3)** The log stores NAMES, not ids, so a lane rename orphans its history; orphans are tallied into `unmatched` and never dropped. `entity_type = 'deal'` filters every such read — and note it is defence-in-depth, not the sole guard, since the funnel also keys through deal ids: the test that makes it load-bearing seeds a row stamped `lead` against a DEAL's id, which is realistic because `entity_id` has **no foreign key** (the log outlives the row on purpose). **(4)** The trailing window is bounded at BOTH ends. NULL `closed_at` — every imported closed deal until WS-26f f4's owner-gated backfill runs — falls outside it (zeros, never "closed today"), with the count reported so a 0% win rate is explicable; and so does a FUTURE `closed_at`, because f4's proxy is Zoho's `Closing_Date`, a forecast date that imported deals routinely carry ahead of today — with a lower bound only, running the repair would have started counting next quarter's deals as closed this quarter and inflating the cycle average by their forward span. The lost-reason breakdown carries a NAMED unattributed bucket (the importer bypasses both gates; `lost_reason_id` is `ON DELETE SET NULL`). **No `GROUP BY` is emitted, deliberately**: per-key aggregates in `get_pipeline`'s shape, because the weighted expression binds the lane's own default as `:stage_probability` and a grouped statement would have to reach it through a join and would stop being the expression the fixture and the fake both read. - **A hand-edited `lead_name` survives a PATCH that moves its inputs** (`core.lead_name_is_derived`): the name is re-derived only while the stored value still equals what the fallback chain would produce. Answered by recomputing rather than by a `lead_name_is_custom` column — a flag has to be maintained by every writer (importer, sync engine, agent tools) and the one that forgets it silently reverts a typed name. - ⚠️ **The timeline's THIRD source is email, and it is the ONE place in this package scoped to the CALLER rather than to the org** (WS-26d-email, spec §9). Everything else here follows D-CRM-3 — org-visible to every `feature:crm` holder, no owner predicate. Email cannot: the CRM is org-visible while a mailbox belongs to one person, so an unscoped join publishes one member's inbox to the whole company. `activities._timeline(entity, record_id, limit, user)` therefore **requires the caller** and all four routes pass it; a route that drops `user` again would compile, return a timeline, and have no identity left to scope by. The predicate is `activities._email_account_scope`, a **verbatim copy** of `routes/email/core.py::_account_scope` (D-CRM-4, the same call `broker_handlers.broker_gate` made — importing another route package's private helper is the coupling this package declined once already). ⚠️ **Two copies is a coincidence; a THIRD copy anywhere means promote it to a shared module instead.** The fragment hardcodes the alias `em`, so the query aliases `email_messages` as `em` and it drops in unchanged (`email/automation/analytics.py` had to `.replace()` it). Other invariants: the unit is the **thread** (`DISTINCT ON (account_id, COALESCE(thread_id, id::text))` — a row-per-message timeline double-counts every conversation, and grouping on a raw nullable `thread_id` folds every un-threaded message in an account into one entry); addresses resolve **once per record**, not per source, because a deal's set already contains its originating lead's and a per-source pass would return every inherited thread twice; `crm_deals` has no `email` column so a deal joins through `lead_id → crm_leads.email` **and** `crm_deal_contacts → crm_contacts.email`, unioned, with the lead's threads labelled `origin="lead"`; **no addresses means no query at all** (an empty `IN ()` is a syntax error, and the failure a fallback would produce is the whole mailbox on a record that names nobody); inbound `from_address` only in v1, and organizations deliberately do **not** join by domain (an `@fracktal.in` match would attach the entire company mailbox to our own org record). Index: `(account_id, LOWER(from_address->>'email'))` on `email_messages` — the two FTS GINs bury the address inside a `to_tsvector` and are usable only via `@@`. ⚠️ **`tests/unit/test_crm_email_timeline.py` carries a MUTATION FENCE**: deleting the `_email_account_scope(…)` call must turn `test_a_holder_with_no_mailbox_sees_no_email` and `test_two_holders_each_see_only_their_own_account` RED. `_crm_fakes.py` grew four readers so it can see that (a scope subquery, a lowercased JSONB address comparison, a composite LEFT JOIN, and `DISTINCT ON` grouping) — before them the fake did not merely ignore the scope, `_PLAIN_EQ` MISREAD the subquery's own `user_id = :uid`. Do not simplify the SQL to suit the fake; extend the fake. - - ⚠️ **`auto_lead.py` (WS-26d-autolead) — the package's second UNATTENDED writer, and the only one reached from another app's hook.** It registers **no routes** and is therefore NOT imported from `__init__.py` (same reason as `broker_handlers.py`); its one entry point `create_leads_from_new_mail(account_id)` is called from `routes/email/scheduler_hooks.py::process_new_mail`. **It lives here rather than in the email package because what it does is write a CRM record** — it owns `crm_auto_lead_cursors`, it goes through `records.create_record`, and its flag is a CRM owner gate. Unlike the timeline join above it **imports** the automation package's PUBLIC identity primitives (`sender_scope` / `resolve_org_domains` / `normalize_domain`) instead of copying them: D-CRM-4 declined to import another package's *private* helper, and a third copy of "is this person a colleague?" is exactly the drift that rule prevents. Six properties are load-bearing. **(1) The flag is read at the CALL SITE, before the step is entered** — `if auto_lead_enabled(): await create_leads_from_new_mail(...)` — so with `CRM_AUTO_LEAD` off no CRM code runs and no CRM query is issued on the mail path; `auto_lead_enabled` is the flag's ONE definition and a gate moved *inside* the step is pinned red by an AST assertion, not only by a runtime sentinel. **(2) THREE cursor facts, three questions.** `process_new_mail` is also reached by ~1-year deep resyncs and by a newly connected mailbox's first sync, and neither stamps `rules_held_back_at`, so "everything classified" would mint a lead per unknown sender across a year of mail — each born `zoho_dirty` and queued for the LIVE tenant within one 600s cycle (D-CRM-9), with no confirmation card on a scheduler hook and no delete tool. `received_at > activated_at` is the backfill discriminator, `rules_processed_at > processed_watermark` is the incremental cursor, and `last_run_at` is the DORMANCY clock. ⚠️ **The third column is not redundant and `activated_at` is not "set once":** the anchor means *the current ON epoch*, because `activated_at` alone did nothing about a flag turned off for four weeks and back on — the first ON cycle minted the whole OFF window (27 leads, measured). A gap in `last_run_at` beyond `REANCHOR_GAP_SECONDS` (3600) re-stamps all three and mints nothing from the gap, at WARNING on `sync.auto_lead_reanchored`. Dormancy reads `last_run_at` rather than the watermark on purpose: the watermark tracks MAIL, so a mailbox merely quiet over a weekend would be re-anchored and would drop the first message to arrive on Monday — the one the feature exists to catch — and a deliberately-held cursor (below) would be silently re-anchored past. **(2b) A failure never advances the cursor past lost work.** The watermark moves over the contiguous PREFIX that wrote its leads and stops at the first that raised; this step opens a second session per lead through `create_record` while holding the batch's own, so pool exhaustion fails many at once and an unconditional advance stepped over all of them (3 leads lost, measured). A held cursor logs `sync.auto_lead_stalled` at WARNING EVERY cycle, because a held cursor and a quiet mailbox both create nothing and only the level tells them apart. A failed first ACTIVITY is the counter-case — counted separately, never holding the cursor, since the lead is already committed and would be skipped on retry. **(3) Dedup is a SELECT guard plus in-batch de-duplication, never `ON CONFLICT`** — `crm_leads` has no unique constraint on email (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so the ticket's original upsert arm could not have fired. The cross-invocation race is ACCEPTED and recorded; **do not "fix" it with a unique index** (1,516 imported rows, the migration-148 shape). **(4) "External" is necessary, not sufficient** — `sender_scope` fails SAFE to `"external"`, which is the wrong direction when the consequence is a lead row for your own CFO in a live Zoho tenant, so the normalised internal-domain list is a second, independent gate that matches SUBDOMAINS too (`cfo@mail.fracktal.in` is the CFO), anchored on a leading dot so `notfracktal.in` is still a prospect. **(5) The lead goes through `records.create_record`, never raw SQL** (`_resolve_status`, the `owner_email` default, `validate_source` and `mark_dirty_on_insert` all live only there, and only the last is visible in the row afterwards), and `lead_name` is left to `compute_lead_name` over a display name STRIPPED before it is split. **(6) The first activity is `type='system'` — outside `sync_zoho.push_activities`' `type IN ('note','task')` predicate — carrying the subject and the sender in `meta` and an EMPTY body.** The step never selects `body_text` or `snippet`: the projection is the privacy boundary (D-CRM-12 applied to what a machine writes). **(7) The Sent probe folds case, unlike `_maybe_block_cold`** — `@>` is case-EXACT, so a reply from `asha@` after we wrote to `Asha@` minted a lead for somebody mid-conversation; this module uses `EXISTS (… jsonb_array_elements … lower(…) = :addr)` and leaves the email package's predicate alone. **(8) Both attacker-controlled strings are clipped** (`MAX_NAME_CHARS`, `MAX_SUBJECT_CHARS`): the display name becomes `lead_name`, which every list, board card and Zoho push then carries. `tests/unit/test_crm_auto_lead.py` (73 cases) carries a THIRTEEN-mutant fence over exactly those properties, and `_crm_fakes.py` gained two readers plus `fail_on(..., after=N)` for it — without the readers the Sent probe was invisible and the fake answered "yes" for every Sent message, and without the offset a prefix-only cursor and an unconditional one are indistinguishable. + - ⚠️ **`auto_lead.py` (WS-26d-autolead) — the package's second UNATTENDED writer, and the only one reached from another app's hook.** It registers **no routes** and is therefore NOT imported from `__init__.py` (same reason as `broker_handlers.py`); its one entry point `create_leads_from_new_mail(account_id)` is called from `routes/email/scheduler_hooks.py::process_new_mail`. **It lives here rather than in the email package because what it does is write a CRM record** — it owns `crm_auto_lead_cursors`, it goes through `records.create_record`, and its flag is a CRM owner gate. Unlike the timeline join above it **imports** the automation package's PUBLIC identity primitives (`sender_scope` / `resolve_org_domains` / `normalize_domain`) instead of copying them: D-CRM-4 declined to import another package's *private* helper, and a third copy of "is this person a colleague?" is exactly the drift that rule prevents. Six properties are load-bearing. **(1) The flag is read at the CALL SITE, before the step is entered** — `if auto_lead_enabled(): await create_leads_from_new_mail(...)` — so with `CRM_AUTO_LEAD` off no CRM code runs and no CRM query is issued on the mail path; `auto_lead_enabled` is the flag's ONE definition and a gate moved *inside* the step is pinned red by an AST assertion, not only by a runtime sentinel. **(2) THREE cursor facts, three questions.** `process_new_mail` is also reached by ~1-year deep resyncs and by a newly connected mailbox's first sync, and neither stamps `rules_held_back_at`, so "everything classified" would mint a lead per unknown sender across a year of mail — each born `zoho_dirty` and queued for the LIVE tenant within one 600s cycle (D-CRM-9), with no confirmation card on a scheduler hook and no delete tool. `received_at > activated_at` is the backfill discriminator, `rules_processed_at > processed_watermark` is the incremental cursor, and `last_run_at` is the DORMANCY clock. ⚠️ **The third column is not redundant and `activated_at` is not "set once":** the anchor means *the current ON epoch*, because `activated_at` alone did nothing about a flag turned off for four weeks and back on — the first ON cycle minted the whole OFF window (27 leads, measured). A gap in `last_run_at` beyond `REANCHOR_GAP_SECONDS` (3600) **CLAMPS `activated_at` to `now - 1h` and runs the batch anyway** — clamp, never reset, and never an early return. ⚠️ **This step is not invoked once per scheduler period**: `email_ingestion/scheduler.py:463-472` fires the hook only when a sync PERSISTED mail, so the cycle that trips dormancy always carries the message that woke it, and a reset-to-`now` anchor excluded that message permanently — every night, every weekend (measured). Clamping keeps the OFF-window backlog excluded while admitting the last hour, with that hour recorded as the accepted residual. The watermark is untouched on re-anchor. Dormancy reads `last_run_at` rather than the watermark because a cycle that ran and found no candidates advances one and not the other — and that is the state a deliberately-held cursor (below) sits in, which the watermark version would re-anchor past. **(2b) A failure never advances the cursor past lost work.** The watermark moves over the contiguous PREFIX that wrote its leads and stops at the first that raised; this step opens a second session per lead through `create_record` while holding the batch's own, so pool exhaustion fails many at once and an unconditional advance stepped over all of them (3 leads lost, measured). A held cursor logs `sync.auto_lead_stalled` at WARNING EVERY cycle, because a held cursor and a quiet mailbox both create nothing and only the level tells them apart. A failed first ACTIVITY is the counter-case — counted separately, never holding the cursor, since the lead is already committed and would be skipped on retry. **(3) Dedup is a SELECT guard plus in-batch de-duplication, never `ON CONFLICT`** — `crm_leads` has no unique constraint on email (`idx_crm_leads_email` is a plain index; only `zoho_id` is UNIQUE), so the ticket's original upsert arm could not have fired. The cross-invocation race is ACCEPTED and recorded; **do not "fix" it with a unique index** (1,516 imported rows, the migration-148 shape). **(4) "External" is necessary, not sufficient** — `sender_scope` fails SAFE to `"external"`, which is the wrong direction when the consequence is a lead row for your own CFO in a live Zoho tenant, so the normalised internal-domain list is a second, independent gate that matches SUBDOMAINS too (`cfo@mail.fracktal.in` is the CFO), anchored on a leading dot so `notfracktal.in` is still a prospect. **(5) The lead goes through `records.create_record`, never raw SQL** (`_resolve_status`, the `owner_email` default, `validate_source` and `mark_dirty_on_insert` all live only there, and only the last is visible in the row afterwards), and `lead_name` is left to `compute_lead_name` over a display name STRIPPED before it is split. **(6) The first activity is `type='system'` — outside `sync_zoho.push_activities`' `type IN ('note','task')` predicate — carrying the subject and the sender in `meta` and an EMPTY body.** The step never selects `body_text` or `snippet`: the projection is the privacy boundary (D-CRM-12 applied to what a machine writes). **(7) The Sent probe folds case, unlike `_maybe_block_cold`** — `@>` is case-EXACT, so a reply from `asha@` after we wrote to `Asha@` minted a lead for somebody mid-conversation; this module uses `EXISTS (… jsonb_array_elements … lower(…) = :addr)` and leaves the email package's predicate alone. **(8) Both attacker-controlled strings are clipped** (`MAX_NAME_CHARS`, `MAX_SUBJECT_CHARS`): the display name becomes `lead_name`, which every list, board card and Zoho push then carries. `tests/unit/test_crm_auto_lead.py` (75 cases) carries a FIFTEEN-mutant fence over exactly those properties, and `_crm_fakes.py` gained two readers plus `fail_on(..., after=N)` for it — without the readers the Sent probe was invisible and the fake answered "yes" for every Sent message, and without the offset a prefix-only cursor and an unconditional one are indistinguishable. 14. routes/admin/ -- Org access control `/admin` API + `/auth/me` (spec: ai-company-brain/specs/org_access_control.md, Phase 1): member roster and lifecycle (invite/suspend/remove — soft, because ~every user-scoped table keys people by email — **plus a separate hard delete**, below), role assignment, custom role CRUD, per-user allow/deny overrides, and the feature catalog the admin UI renders from. `GET /auth/me` is deliberately NOT admin-gated — every signed-in member calls it to resolve their own feature/agent access, and it returns resolved OUTCOMES (allowed feature slugs, runnable agent names) rather than raw permission patterns, so the matching rule has exactly one implementation. `GET /admin/members/{email}/access` returns each decision WITH its provenance (which role granted it, which override took it away) — the admin UI shows that verbatim rather than re-deriving it. Invariants enforced in `_common.py`: the org always keeps an owner, nobody assigns a role above their own rank, system roles are immutable, and **nobody locks themselves out**. That fourth one (`assert_not_self_lockout`, `colleague_onboarding.md` §2 Step 5 / N7+N8) is called by `update_member` (PATCH), `remove_member` (DELETE) **and `purge_member` (DELETE …/purge)** — three doors reach the same `is_active = False`, and while the check lived inside DELETE alone the PATCH had none: `PATCH {"status": "suspended"}` on your own row was refused only by `assert_owner_survives` firing coincidentally in a one-owner org, so a second owner opened it. ⚠️ The rule is **"any status that is not `active`"**, never a list of destructive ones — `EffectiveAccess.is_active` is `status == "active"` exactly, so `invited` is a lockout too, and an enumeration would have to remember it. Comparison is case-insensitive and empty-safe on both sides (an IdP that re-cases a UPN must not switch the guard off; a caller with no identity is not everybody). ⚠️ **It and `assert_owner_survives` both answer 409** — a test that asserts the bare status code cannot tell which fired, and for self-suspension the one that fires today on `main` is the wrong one; discriminate on the detail text and on what was written (`tests/unit/test_admin_member_offboarding.py`). **`purge_member` — `DELETE /admin/members/{email}/purge` (N8)** is the hard delete: a SEPARATE route on the same `admin:members:manage`, never a flag on Remove (which would put the irreversible path one typo from the reversible one). Its decision is **purge the person, keep their work** — the `app_user` row, every access grant (`user_role`, `user_permission_override`, `org_group_member`, `chat_session_participant`, `app_grants`, `app_tool_grants`), every credential (`email_accounts`, `wa_accounts`, `task_accounts`), their PRIVATE `chat_session` rows and their `access_request` row go; what they authored and **the audit trail stay** (an audit trail that disappears with the person is not one — `app_audit` already carries a FK-less `app_id` commented "audit survives hard delete"). ⚠️ **Nothing is anonymised, on purpose**: the address is the join key across ~50 tables, so scrubbing `owner_email` would orphan the apps rather than hide the person. ⚠️ **The three credential rows cascade, and the map is `members._CREDENTIAL_CASCADES`** — `email_accounts` takes the whole mirrored mailbox (**17 direct children, 20 with transitives**), `wa_accounts` the whole WhatsApp mirror (**14 / 16**; `wa_media` hangs off `wa_messages`, NOT off the account), `task_accounts` the SYNCED half of `gtd_items` **and `gtd_projects`**; the credential is `NOT NULL` on the row, so it cannot go without it. That map is hand-maintained and says so, and is pinned against `infra/postgres/` by a test that re-derives it — the first version named 15 of the 20 email tables, which on a route whose safety argument is "the admin is told the blast radius before clicking" is the wrong direction of error. ⚠️ **THREE tables are split across both lists, and each predicate is load-bearing:** `chat_session` by `visibility` (private deleted, shared kept — a room cascades `chat_message` and one person's off-boarding must not take a shared transcript), and `gtd_items` + `gtd_projects` by `account_id` (`IS NOT NULL` = the SYNCED mirror, counted and deleted explicitly; `IS NULL` = the LOCAL rows they authored here, kept). ⚠️ **A KEEP clause must exclude everything the delete side CASCADES away, not merely everything it names.** The `tasks` keep clause originally had no `account_id` predicate, so a member with 847 synced tasks was answered `kept: {"tasks": 847}` while all 847 went with `task_accounts` — the response reported a destruction as a survival. `_PURGE_DELETES`/`_PURGE_KEEPS` derive `count_sql` and `delete_sql` from ONE `where` clause so the count and the delete cannot differ, but that is a within-row-spec guarantee and says nothing about a third statement three entries up; one transaction, one commit, and `record_admin_change` fires BEFORE it (`acb_audit` has its own session, so the record of a destruction survives a rollback of it — though `acb_audit/log.py:49` swallows every exception, so a *completed* purge is NOT guaranteed to leave an audit row). Pinned by `tests/unit/test_admin_member_purge.py` — including the structural assertion that no audit table appears on the delete side at all, the exact permission slug on the route (deleting it leaves the `admin:members:read` floor, which `manager` holds), and the cross-table cascade fences built on `tests/unit/_schema_cascade.py`, which derives the FK graph from the numbered migrations. ⚠️ **`_admin_fakes._FakeDB` models no foreign keys and therefore no cascades**, so every cross-table claim here has to be structural; no behavioural case over a seeded fake can make one. Every write calls `invalidate_access` so a change lands immediately instead of after the resolver's 60s TTL. Tables: infra/postgres/130_org_access_control.sql. Same `_common.py`-is-the-leaf layout as routes/apps and routes/tasks — and here the leaf rule is strict: feature modules import from `_common`, **never from each other**. ⚠️ **The `/admin` auth floor is PER-ROUTE, not a package property.** `_common.py` creates the router with **no** `dependencies=`; every route declares `Depends(require_admin_user)` in its own signature. A route added without it inherits no floor at all and is reachable by any authenticated member — the easiest hole to ship in this package. `access_requests.py` — **sign-in requests** (`colleague_onboarding.md` §6 / N6a, migration 143): `/admin` was push-only, so somebody arriving at the front door produced a journald warning nobody read back (53 of them for one address over 18 hours on 2026-08-03/04, and the owner learned out of band). `acb_auth.access.resolve_access` now upserts an `access_request` row when — and ONLY when — `record_request=True`, which exactly one caller passes; `GET /admin/members/requests` + `POST .../{email}/approve|deny` let the owner answer it, both writes on the EXISTING `admin:members:invite` (no new slug — a new slug is nobody's grant until an admin creates it). **Approve provisions AND activates in one action** (`status='active'`, not `'invited'`) because an approval IS the decision to let somebody in and they are already at the door; leaving them `invited` would re-create the two-click trap §2 Step 1b documents. Both provisioning callers go through `_common.provision_member` — ONE path, so invariants 1 and 2 apply to approvals too (it calls `assert_owner_survives` itself, because `set_roles` REPLACES assignments and provisioning the last owner with the default `member` role would otherwise delete the org's only owner grant). ⚠️ **Both writes hold `admin:members:invite`, which is WEAKER than the `admin:members:manage` that suspends or off-boards, so every path by which the weaker one could reverse the stronger is a cross-gate escalation.** Two independent locks, and each is load-bearing for a different sequence: (1) `_load_request(db, email, *, allowed_statuses=…)` — keyword-only, no default — refuses an already-DECIDED row, because decided rows are kept on purpose (dw9) and the tab renders only `pending`, so a decided row is invisible *and* still addressable; approve takes `("pending",)`, deny takes `("pending", "denied")` since re-denying grants nothing, and denying an *approved* request is refused because it could only make the queue contradict the roster. (2) `_common._PROVISION_MEMBER_SQL`'s `ON CONFLICT` arms **name the statuses they rewrite and never negate**: `invited` → the caller's status (the one door to `active`), `removed` → the caller's status **only when it is not `active`** (so invite still returns an off-boarded person as `invited`, byte-for-byte as before, while approve cannot reinstate them — `removed → active` stays `PATCH /admin/members/{email}`), and `active`/`suspended` are never touched. ⚠️ A `<>`/`NOT IN` test against `app_user.status` is the mutation to watch for: it reads as tidier and silently rewrites rows set under a stronger permission. `tests/unit/test_signin_requests.py` pins the SQL **structurally** (`test_provisioning_only_ever_rewrites_a_status_it_names`) — its fake DB re-implements the `ON CONFLICT` arms in Python and a mirror can only agree with itself, so the behavioural cases there cannot see the statement being widened and must not be trusted to. ⚠️ **Lock (2) declines SILENTLY — it just does not rewrite the row — so it is only half an answer, and the other half is `APPROVE_MATRIX`.** Approve used to run its `_decide(…, "approved")` after that quiet decline: HTTP 200, request marked `approved`, `set_roles` re-granting `['member']` to an off-boarded member, and the person gone for good from a tab that renders only `pending` (the resolver's upsert never rewrites `status`). **`access_requests.APPROVE_MATRIX`, read by `_disposition_for` BEFORE anything is written, is the contract:** absent → provision; `invited` → activate + assign the roles; `active` → do nothing, leave their roles alone, resolve the request as `approved` and say so in `ApproveResult.detail`; `suspended`/`removed` → **409, request stays `pending`** so the person stays visible; anything else → refuse (fail closed). The invariant: **approve never rewrites the roles of a member who already exists in a state other than `invited`** — `provision_member` ends in `set_roles`, which REPLACES assignments, and roles are otherwise `admin:members:manage` territory. The matrix is pinned against `members.VALID_STATUSES`, so a fifth member status cannot ship without somebody deciding what approving one means. `_DECIDE_SQL` also binds the read's own status filter into the UPDATE (`AND status = ANY(:allowed) … RETURNING id`) and 409s on zero rows **before `db.commit()`**, so a lost race discards its own provisioning instead of half-applying it; each route must pass `_decide` the same tuple it passed `_load_request` (a test asserts that from the source). 15. routes/workflows/ -- Workflows app `/workflows` API (spec: ai-company-brain/specs/workflows_app.md; RFC: docs/workflow-editor/README.md): workflow CRUD over the React-Flow-native edit-model (`workflows.graph` jsonb, persisted verbatim), publish → compile to an immutable `workflow_versions.serialized` run-model (edit-model ≠ run-model; runs pin versions), run start/history/detail + a per-run SSE event stream (in-process hub in service.py; runs are supervised asyncio tasks — durable queueing is BO‑20), the served node catalog (agents from the live registry, integrations from acb_skills with availability probe, workflow tool registry, ready modules — the palette is never hard-coded, spec D7), Module Studio (workflow_modules CRUD + conversational generate on acb_llm tier routing + AST validate + subprocess test/run), the inbound webhook trigger `POST /workflows/hooks/{hook_token}` (public by token — in PUBLIC_ROUTES + the router's exempt list; optional HMAC `X-CC-Signature`; rate-limited; fires only published workflows with an enabled webhook trigger), and the cron schedule scanner (scheduler.py — apscheduler CronTrigger parsing inside a supervised asyncio loop with CAS claims on `last_fired_at`; started/stopped from main.py lifespan). The engine subpackage (engine/: templating, graph compile/validate, node handlers over injected NodeServices, MAF WorkflowBuilder runner, module AST validator + restricted subprocess runner) is transport-free — no FastAPI/DB imports — so it is unit-testable alone and movable into the orchestrator if isolation later demands. Agent nodes call `orchestrator.executor.run_agent` (source="workflow", MAF batch path — constraint #9); write-class tool nodes dispatch through `action_broker.propose/submit` (fail closed, constraint #4); module code is import-free/pure-transform only (real sandbox is BO‑7). Capability search (search.py): **keyword-only by explicit owner decision** — deterministic token/substring ranking over the live registries (no index table, no embeddings; an embedding-backed variant was built and deliberately removed in favour of BO‑22, the platform-wide semantic-search service, whose ranking backend will swap in behind the same API shape) — `GET /workflows/catalog/search` serves the palette's search box AND the copilot's shortlist from the same ranking. Workflow Copilot (copilot.py): `POST /workflows/{id}/copilot` — chat-to-build; the LLM emits `{reply, graph, new_modules}`; **missing modules are auto-created** (Module Studio AST validation, saved `ready` with `auto_created` provenance, name→id rewired), the graph is validated with one named-issue repair round against the same validators as publish, and the result is returned for CLIENT-side apply — the copilot never writes the workflow row. `_call_copilot` is the stubbing seam for tests. Tables: infra/postgres/132_workflows.sql. Slice 2: **approval node** — an `approval` node pauses the run (engine returns status `paused`; downstream marked `pending`), `service._hold_for_approval` files a `workflow.resume_run` proposal into the EXISTING Action Broker inbox (`pending_actions` → /approvals UI) with everything a resume needs in the `workflow_run_pauses.snapshot`; approving fires `broker_handlers._resume_run_handler` which replays the run with completed nodes' stored outputs (`precomputed` — no repeated side effects) and the gate resolved; a rejected proposal is reconciled lazily on run read (run → `cancelled`). **Event triggers** — `triggers.dispatch_event` starts runs for published workflows whose `kind='event'` binding matches `(source, event_type)` (empty type = all); fed by BOTH `/agent/webhook/{source}` (routes/agent.py calls it after agent routing; response carries `workflow_runs`) and the native ClickUp receiver via `ingestion.event_hooks` (a `post_sync.py`-style sink registry — ingestion never imports upward; main.py registers the dispatcher at startup). Same core-is-the-leaf layout as routes/tasks; ⚠️ `__init__.py` import order is load-bearing (static paths before crud's `/{workflow_id}`; a regression test pins it). Startup: main.py lifespan calls `service.reconcile_orphaned_runs()` BEFORE starting the scheduler — rows still `running` belong to a dead process and are swept to `failed` ("interrupted by a platform restart"); `paused` rows are deliberately untouched (resume rebuilds everything from the pause snapshot), and `runs.py` keeps the per-read lazy patch for reads that race the sweep. Run-history drill-in (spec F9): clicking a history row in the editor's RunConsole fetches the run detail and paints its recorded `node_results` onto the canvas (cleared when a live test run starts). Engine semantics are locked by a CI-blocking golden trajectory eval — `evals/trajectories/test_workflow_engine_trajectory.py`; `skill-eval.yml` triggers on `routes/workflows/**` so engine edits re-run the gate. **Publish authority** (spec Q3, migration 133): `POST /{id}/publish`, `/versions/{v}/rollback`, and `/disable` require the `workflows:publish` capability on top of the router's `feature:workflows` gate — they are the acts that ARM triggers to run unattended. Drafting, validate, Test runs, duplicate, and the copilot stay open to the feature (a draft fires no triggers and its writes are still broker-held). `/auth/me` returns a resolved `capabilities` list so the editor can grey out Publish with a reason instead of a bare 403 — the browser must never re-derive wildcard matching (`permissions` holds raw patterns; an owner has `*`). **Wait node** (F3 logic vocabulary): `{"seconds": N}`, ≤`WAIT_INLINE_MAX_SECONDS` (60) sleeps inline inside the run; longer pauses the run exactly like an approval but with `reason='wait'` + a `resume_at` deadline in the pause snapshot and NO broker proposal (nobody decides anything) — `scheduler.scan_due_waits()` runs in the same loop as cron triggers and hands matured pauses to the SAME `service.resume_run`, which routes by pause reason (`elapsed_waits` vs `resolved_approvals`, so an elapsed wait can never clear an approval downstream). A resumed wait must never sleep again: the handler only sleeps when the duration is inline-short. Lifecycle extras: `POST /{id}/duplicate` (crud.py — copies graph/variables/triggers into a fresh DRAFT; the hook token is ALWAYS regenerated, it is a credential) and `POST /{id}/versions/{v}/rollback` (publish.py — republishes version v's immutable snapshot as a NEW version; deliberately does not re-validate as a gate since rollback is incident response — catalog drift comes back as non-blocking `warnings`, and the draft edit-model is never clobbered). **Automation health** (spec R2, migration 134): every terminal run calls `service.evaluate_automation_health()`, which disables a published workflow after `AUTO_DISABLE_AFTER` (5) consecutive failures **from `UNATTENDED_TRIGGERS` only** (`schedule`/`webhook`/`event` — a maker's Test runs and agent `api` calls must never disable production). The streak is derived from `workflow_runs`, never a counter column, and is scoped to runs after `workflows.health_since`, which publish/rollback/enable each re-stamp — without that window a re-enabled workflow would re-disable on its next failure, since the failures that tripped the policy are still the newest rows. The disable is a CAS on `status='published'` so concurrent failing runs produce exactly one disable; `disabled_reason`/`disabled_at` are written the same way for the human Disable path, so the gallery answers "why is this off?" identically. Notification is in-product (persisted reason → gallery badge + editor banner, `workflows.auto_disabled` log, activity-feed `disabled` event); outward notification would be an outward write and belongs on the broker path. `POST /{id}/enable` (publish.py, same `workflows:publish` gate) is the way back: it re-arms the EXISTING live version rather than minting one, 409s if the workflow was never published, and is idempotent when already live. `_execute_run`'s `trigger_kind` is a REQUIRED keyword — a dropped kwarg would make the whole policy silently inert. **Trigger durability** (spec §3.3a): schedules are DB rows, not OS cron and not an APScheduler process — `CronTrigger` is a parser only. ⚠️ `compute_due_fire` only looks FORWARD, so a trigger with `last_fired_at IS NULL` yields no tick; `_claim_baseline` arms it on first sight instead of firing (a cron says *when*, not *how far back*). Without that step a new schedule never fires **at all** — it produced no tick, so it never got a baseline, so it produced no tick. `config.timezone` is an IANA wall clock (default UTC) validated at save with the cron, so a 9am job stays 9am across DST; the zone is passed to `CronTrigger.from_crontab`, and instants stay UTC-aware throughout. `update_workflow` rewrites trigger rows wholesale but CARRIES `last_fired_at` across for unchanged schedules (`_trigger_identity` = kind + cron@timezone) — otherwise every canvas save re-armed the cron and lost the already-fired-this-tick guarantee. Because the CAS claim commits BEFORE `start_run`, a claimed tick can never be re-offered: every path out of that block calls `service.record_skipped_run()`, which writes a terminal `cancelled` run row (cancelled, not failed — being busy must not feed the R2 auto-disable policy). **Hook URL**: `core.hook_url()` builds it from `settings.public_api_base_url` and `get_workflow` returns `hook_url`/`hook_path`; the browser must NEVER assemble one from `window.location`, because the control-plane `/api` proxy re-serializes JSON (breaking sender HMAC) and drops non-JSON bodies. The Next route `api/workflows/hooks/[token]/route.ts` is a raw-bytes passthrough that attaches no internal bearer. **Typed tool arguments** (`engine/tool_args.py` — n8n's typed-node-parameters pattern, Sim's `subBlocks`): a tool's `args_schema` value is a mini-language `type[?][|description]` over the closed set `{string,number,boolean,object,array}`; an unknown type degrades to `string` rather than raising (one bad declaration must not take the whole catalog down). It is parsed in ONE place and consumed in three — the catalog serves `args[]` (parsed) so the browser never re-implements the grammar, `validate_graph(tool_schemas=…)` blocks publish on a missing/unknown/mistyped argument (`tool_args` issue code), and `execute_tool` re-checks at run time because a draft Test, a copilot graph, or an older published version can all reach a handler that publish never saw. `{{refs}}` satisfy required checks and are exempt from type checks on both sides — they resolve at run time. Type checking is deliberately lenient (only container-vs-scalar category errors) so the messages that fire are worth reading. `tests/unit/test_workflows_tool_contract.py` holds each declaration to its handler by AST-scanning for `args.get("x")`/`args["x"]` — the drift it hunts is a handler growing an input the schema never declares (`_broker_write`'s `target_field` is dynamic, so it has its own explicit test). **Golden workflow fixtures** (`evals/trajectories/workflows/*.json` + `test_workflow_fixtures.py`): whole workflows paired with an expected outcome, one generic runner; `expect.publishable: false` fixtures pin the publish gates. Tool schemas and destructive actions come from the REAL registry so fixtures break when the shipped catalog changes; fixtures assert which seams were crossed (`agent_calls`/`tool_calls`/`tool_args`), because "succeeded" while silently never calling the integration is the failure mode they exist to catch. 16. agents.json -- Dynamic agent registry (persisted alongside pyproject.toml) diff --git a/apps/services/gateway/gateway/routes/crm/auto_lead.py b/apps/services/gateway/gateway/routes/crm/auto_lead.py index cc445ca99..8e6ab88f2 100644 --- a/apps/services/gateway/gateway/routes/crm/auto_lead.py +++ b/apps/services/gateway/gateway/routes/crm/auto_lead.py @@ -44,9 +44,13 @@ * ``processed_watermark`` — the incremental cursor over ``rules_processed_at``. * ``last_run_at`` — when this step last RAN. It is what detects dormancy, - and it is a separate column because the watermark cannot answer that - question: a quiet mailbox has a watermark hours old while the step has - been running faithfully every cycle. See :func:`_reanchor_if_dormant`. + and on a gap it CLAMPS ``activated_at`` forward to + ``now - REANCHOR_GAP_SECONDS`` rather than resetting it, so the OFF/outage + backlog stays excluded while the message that woke the step survives. + ⚠️ The step is invoked only when a sync PERSISTED mail + (``email_ingestion/scheduler.py:463-472``), never once per period, so this + column trips overnight and over weekends routinely — which is why + re-anchoring has to be cheap. See :func:`_reanchor_if_dormant`. Without the first, connecting a second mailbox mints a lead per unknown sender in a year of mail — each born ``zoho_dirty`` and queued for the live @@ -91,7 +95,7 @@ from __future__ import annotations import json -from datetime import UTC, datetime +from datetime import UTC, datetime, timedelta from typing import Any from acb_auth import UserContext, UserRole @@ -123,16 +127,27 @@ #: reads as "covered everything". MAX_CANDIDATES_PER_CYCLE = 200 -#: How long a gap in this step's own RUNS means the ON epoch ended. -#: Six scheduler periods at the 600s sync interval, so an ordinary slow cycle, -#: a restart or a single missed poll can never trip it. +#: How long a gap in this step's own RUNS means the ON epoch ended — and, +#: because the anchor is CLAMPED to it rather than reset, also how much recent +#: mail survives that gap. #: -#: ⚠️ This is the OFF→ON guard and it is not optional. ``activated_at`` alone -#: stops a deep resync, but it does nothing about a flag that was on, turned -#: off for four weeks, and turned back on: the cursor still carries day-1's -#: anchor, so the first ON cycle would mint the entire OFF window in one batch -#: — measured at 27 leads for a 27-day window, each pushing unattended into the -#: live tenant. Re-anchoring makes the anchor mean "the current ON epoch". +#: ⚠️ **This is not "six scheduler periods".** The step does not run once per +#: period: ``email_ingestion/scheduler.py:463-472`` fires the hook only when a +#: sync actually persisted mail, so a quiet mailbox does not run it at all. +#: What this measures is therefore "how long since mail last arrived AND was +#: processed", which on a real mailbox means it trips overnight and over +#: weekends as a matter of course. That is precisely why the re-anchor clamps +#: instead of resetting: tripping has to be cheap, because it is routine. +#: +#: One hour is the trade. Larger admits more of a genuine OFF window's backlog; +#: smaller starts excluding mail that arrived during an ordinary evening lull +#: just before the message that woke the step. +#: +#: The guard itself is not optional: ``activated_at`` alone stops a deep +#: resync, but it does nothing about a flag that was on, turned off for four +#: weeks, and turned back on — the cursor still carries day-1's anchor, so the +#: first ON cycle would mint the entire OFF window in one batch, measured at 27 +#: leads for a 27-day window, each pushing unattended into the live tenant. REANCHOR_GAP_SECONDS = 3600 #: The activity ``type`` the originating message is logged as. **Not 'note'.** @@ -262,10 +277,12 @@ async def create_leads_from_new_mail(account_id: str) -> dict[str, int]: cursor = await _load_or_activate_cursor(db, account_id) if cursor is None: # pragma: no cover — the row was just written return stats - if await _reanchor_if_dormant(db, account_id, cursor): - stats["reanchored"] = 1 - _emit(account_id, stats) - return stats + cursor, reanchored = await _reanchor_if_dormant(db, account_id, cursor) + stats["reanchored"] = int(reanchored) + # ⚠️ NO early return here, and that is the whole correction. This step + # is only ever invoked because a sync PERSISTED mail, so the batch that + # tripped the dormancy test is the batch containing the message that + # woke it up. Returning early discarded exactly that message, forever. await _run_batch(db, account, cursor, stats) _emit(account_id, stats) return stats @@ -393,50 +410,79 @@ def _aware(value: Any) -> datetime: return value if value.tzinfo is not None else value.replace(tzinfo=UTC) -async def _reanchor_if_dormant(db: Any, account_id: str, cursor: Any) -> bool: - """Did this step stop running? Then the ON epoch ended; start a new one. +async def _reanchor_if_dormant( + db: Any, account_id: str, cursor: Any, +) -> tuple[Any, bool]: + """Did this step stop running? Then CLAMP the epoch — never reset it. ``activated_at`` guards a deep RESYNC. It does nothing about a flag that was on, turned off for four weeks and turned back on — the cursor still - carries the old anchor, so the first ON cycle mints the whole OFF window - at once, unattended, into a live tenant. Re-anchoring is what makes - ``activated_at`` mean *the current ON epoch* rather than *the first time - anyone ever enabled this*. - - ⚠️ **Dormancy is measured on ``last_run_at``, not on - ``processed_watermark``** — a deliberate departure from the shape first - prescribed, for two reasons that both bite in production: - - * the watermark tracks MAIL, not runs. A mailbox that is simply quiet over - a weekend has a watermark 60 hours old while this step has run faithfully - every 600s, so a watermark-based test re-anchors it and the first message - to arrive on Monday — the one this whole feature exists to catch — falls - before the new anchor and mints nothing. Every Monday. - * a genuinely poison head message holds the watermark still ON PURPOSE - (property 5). A watermark-based test would then re-anchor after an hour - and skip the very backlog the stall was protecting, quietly undoing the - stall it was supposed to make visible. - - Both directions fail closed: an OFF window and a real outage each skip - their backlog. A missed lead is hand-creatable and visible in the mailbox; - 27 unattended pushes into the live Zoho tenant are neither. + carries the old anchor, so the first ON cycle would mint the whole OFF + window at once, unattended, into a live tenant (27 leads, measured). + + **The anchor is clamped to ``now - REANCHOR_GAP_SECONDS``, not set to + ``now``, and the batch still runs.** Both halves of that are load-bearing, + and the first version of this function got both wrong for the same reason: + it assumed the step is invoked once per scheduler period. **It is not.** + ``email_ingestion/scheduler.py:463-472`` reads ``synced`` off the sync + result and calls the hook **only when mail was actually persisted**, so a + mailbox with no new mail does not run this step at all — which means the + cycle that trips the dormancy test is *always* the cycle carrying the + message that woke it. Resetting the anchor to ``now`` and returning early + therefore excluded that message permanently: no mail 22:00→07:30, a cold + prospect writes at 07:30:50, the step runs at 07:31:05, and the one lead it + existed to catch fell a quarter-minute the wrong side of its own anchor. + Every night, every weekend. + + Clamping keeps the fail-closed property that matters — anything received + longer ago than the gap width stays excluded, so a real OFF window or a + multi-day outage still mints nothing from its backlog — while admitting + everything received inside the last ``REANCHOR_GAP_SECONDS``, which is + where the triggering message always is. + + ⚠️ **Documented residual:** mail received in the final hour of an OFF + window IS minted when the flag comes back on. It is bounded (one gap width, + and at most one capped batch of it) and it is the deliberate price of never + dropping the message that woke the step. `crm_app.md` §9 records it and + ``work_plan.md`` §6 (b) tells the owner about it before they flip. + + ``last_run_at`` rather than ``processed_watermark`` is still the clock + here, but the honest reason is narrower than first claimed: both freeze on + a mailbox that receives nothing, since neither advances when the hook never + fires. What separates them is a cycle that ran and found no *candidates* — + mail that landed outside the inbox, or was held back, or predates the + anchor. That cycle advances ``last_run_at`` and not the watermark, and it + is also exactly the state a deliberate stall (property 5) holds the + watermark in; keying dormancy on the watermark would re-anchor past a stall + after an hour and quietly undo it. + + Returns the cursor to run the batch against — re-read, because the anchor + it carries has just changed. """ gap = (now() - _aware(cursor.last_run_at)).total_seconds() if gap <= REANCHOR_GAP_SECONDS: - return False - at = now() + return cursor, False + # `max` is a floor, not arithmetic: inside this branch `last_run_at` is + # always older than the window, so it resolves to the window's start. It + # exists so a clock that jumped backwards cannot move the anchor EARLIER + # than the last known-good run and re-admit history. + anchor = max( + _aware(cursor.last_run_at), + now() - timedelta(seconds=REANCHOR_GAP_SECONDS), + ) await db.execute(text( "UPDATE crm_auto_lead_cursors " - "SET activated_at = :activated_at, " - "processed_watermark = :processed_watermark, " - "last_run_at = :last_run_at, updated_at = now() " + "SET activated_at = :activated_at, updated_at = now() " "WHERE account_id = :account_id" - ), {"account_id": account_id, "activated_at": at, - "processed_watermark": at, "last_run_at": at}) + ), {"account_id": account_id, "activated_at": anchor}) await db.commit() + # The watermark is deliberately NOT touched: OFF-window mail is excluded by + # the anchor predicate whatever its `rules_processed_at` says, and moving + # the watermark forward here would skip the triggering batch a second way. _log.warning("sync.auto_lead_reanchored", account_id=account_id, - gap_seconds=int(gap), threshold_seconds=REANCHOR_GAP_SECONDS) - return True + gap_seconds=int(gap), threshold_seconds=REANCHOR_GAP_SECONDS, + anchored_at=anchor.isoformat()) + return await _read_cursor(db, account_id), True async def _stamp_cursor( diff --git a/infra/AGENTS.md b/infra/AGENTS.md index 37231ccdb..e5c5b6d8f 100644 --- a/infra/AGENTS.md +++ b/infra/AGENTS.md @@ -5,7 +5,7 @@ Docker Compose, Postgres schema, LiteLLM tier config. LLM routing is via the gat ## Key Files - docker-compose.yml -- core services (Postgres 16 + pgvector, Redis 7) -- postgres/ -- schema files (00-10) + 09_app_user.sql (NextAuth users) + 11_integration_credentials.sql (unified credential store) + 130_org_access_control.sql (organization, membership lifecycle on app_user, org_role/org_role_permission/user_role, user_permission_override, feature_catalog — spec: ai-company-brain/specs/org_access_control.md) + 131_integration_memory_permissions.sql (additive: grants `integrations:use:*` + org-memory permissions to the seeded roles; `member` reads org memory but does not write it) + 143_access_request.sql (the sign-in queue — one row per address that authenticated with no `app_user` row, unique on `lower(email)`; `status` carries a `CHECK (pending|approved|denied)` because the vocabulary is load-bearing for ACCESS — approve acts only on `pending`, so a typo'd status would fall out of both the queue and the decided record; deliberately a SEPARATE table and not a fifth `app_user.status`, because an `app_user` row IS the org's member record and a stranger who merely knocked must not acquire one that a future join can surface — spec: ai-company-brain/specs/colleague_onboarding.md §6) + 144_crm.sql (the native CRM spine — organizations/contacts/leads/deals, **statuses as data** rather than enums, one `crm_activities` timeline whose four target FKs are all nullable under a CHECK requiring at least one, and a `crm_status_changes` dwell log; also seeds the `crm` feature_catalog row. Contains the schema's **one FK cycle** — `crm_leads.converted_deal_id` ⟷ `crm_deals.lead_id` — closed by a guarded `DO $$` on `pg_constraint`, because `ALTER TABLE … ADD CONSTRAINT` has no `IF NOT EXISTS`. Idempotency is pinned STATICALLY by `tests/unit/test_crm_migration.py`, which reads the file as text: the unit suite runs no database, so an idempotency claim that holds only by inspection is unenforceable. Spec: ai-company-brain/specs/crm_app.md §3) + 145_crm_zoho_sync.sql (what the two-way Zoho sync needs: `zoho_dirty`/`zoho_synced_at` on the four CRM record tables — and **only** those four, since pipeline vocabulary flows down-only and an activity's push signal is its NULL `zoho_id` — plus `crm_zoho_tombstones` (FK-less on purpose: the row it describes is gone by the time anyone reads it) and `crm_sync_cursors` (`module` PK, so the pull cannot silently rewind). Idempotent via `ADD COLUMN IF NOT EXISTS`, which — unlike `ADD CONSTRAINT` — Postgres supports directly, so no guarded `DO $$` is needed. Pinned by the same static `tests/unit/test_crm_migration.py`, which finds BOTH CRM migrations by CONTENT rather than by number. Spec: crm_app.md §7.1) + 158_crm_auto_lead_cursor.sql (WS-26d-autolead: `crm_auto_lead_cursors`, one row per `email_accounts` id, carrying THREE timestamps because they answer three different questions: `activated_at` (the start of the current ON epoch — `received_at > activated_at` is what tells a deep resync's year-old backlog apart from new mail on the shared `process_new_mail` hook), `processed_watermark` (the incremental cursor, advanced over a batch's successful PREFIX only), and `last_run_at` (the dormancy clock, stamped every cycle — a gap beyond an hour re-anchors the epoch so a flag turned off for weeks and back on mints nothing from the gap). ⚠️ `last_run_at` is deliberately NOT derived from the watermark: the watermark tracks MAIL, so a merely quiet mailbox would read as dormant and lose the first message to arrive afterwards. All three NOT NULL: a NULL cursor is a predicate that matches nothing, which reads exactly like a working feature. ⚠️ **It deliberately adds NO unique index on `crm_leads.email`**: the cross-invocation double-mint race is accepted (one visible, hand-deletable duplicate) because a UNIQUE constraint on a column where 1,516 imported rows may already carry duplicates is a deploy-blocking migration of exactly the shape 148 had to defuse. Inert until `CRM_AUTO_LEAD` is flipped, which is OWNER-GATE. Spec: crm_app.md §9 WS-26d-autolead). ⚠️ `CREATE TABLE IF NOT EXISTS` means a column or constraint added to an ALREADY-APPLIED migration file is silently skipped on that deployment; 143, 144 and 145 have never been applied anywhere (merging 143 is the OWNER-GATE), which is the only reason they are still editable in place. +- postgres/ -- schema files (00-10) + 09_app_user.sql (NextAuth users) + 11_integration_credentials.sql (unified credential store) + 130_org_access_control.sql (organization, membership lifecycle on app_user, org_role/org_role_permission/user_role, user_permission_override, feature_catalog — spec: ai-company-brain/specs/org_access_control.md) + 131_integration_memory_permissions.sql (additive: grants `integrations:use:*` + org-memory permissions to the seeded roles; `member` reads org memory but does not write it) + 143_access_request.sql (the sign-in queue — one row per address that authenticated with no `app_user` row, unique on `lower(email)`; `status` carries a `CHECK (pending|approved|denied)` because the vocabulary is load-bearing for ACCESS — approve acts only on `pending`, so a typo'd status would fall out of both the queue and the decided record; deliberately a SEPARATE table and not a fifth `app_user.status`, because an `app_user` row IS the org's member record and a stranger who merely knocked must not acquire one that a future join can surface — spec: ai-company-brain/specs/colleague_onboarding.md §6) + 144_crm.sql (the native CRM spine — organizations/contacts/leads/deals, **statuses as data** rather than enums, one `crm_activities` timeline whose four target FKs are all nullable under a CHECK requiring at least one, and a `crm_status_changes` dwell log; also seeds the `crm` feature_catalog row. Contains the schema's **one FK cycle** — `crm_leads.converted_deal_id` ⟷ `crm_deals.lead_id` — closed by a guarded `DO $$` on `pg_constraint`, because `ALTER TABLE … ADD CONSTRAINT` has no `IF NOT EXISTS`. Idempotency is pinned STATICALLY by `tests/unit/test_crm_migration.py`, which reads the file as text: the unit suite runs no database, so an idempotency claim that holds only by inspection is unenforceable. Spec: ai-company-brain/specs/crm_app.md §3) + 145_crm_zoho_sync.sql (what the two-way Zoho sync needs: `zoho_dirty`/`zoho_synced_at` on the four CRM record tables — and **only** those four, since pipeline vocabulary flows down-only and an activity's push signal is its NULL `zoho_id` — plus `crm_zoho_tombstones` (FK-less on purpose: the row it describes is gone by the time anyone reads it) and `crm_sync_cursors` (`module` PK, so the pull cannot silently rewind). Idempotent via `ADD COLUMN IF NOT EXISTS`, which — unlike `ADD CONSTRAINT` — Postgres supports directly, so no guarded `DO $$` is needed. Pinned by the same static `tests/unit/test_crm_migration.py`, which finds BOTH CRM migrations by CONTENT rather than by number. Spec: crm_app.md §7.1) + 158_crm_auto_lead_cursor.sql (WS-26d-autolead: `crm_auto_lead_cursors`, one row per `email_accounts` id, carrying THREE timestamps because they answer three different questions: `activated_at` (the start of the current ON epoch — `received_at > activated_at` is what tells a deep resync's year-old backlog apart from new mail on the shared `process_new_mail` hook), `processed_watermark` (the incremental cursor, advanced over a batch's successful PREFIX only), and `last_run_at` (the dormancy clock, stamped every cycle — a gap beyond an hour CLAMPS the epoch to `now - 1h`, so a flag turned off for weeks and back on mints nothing from the gap except its final hour, which is the recorded residual). ⚠️ The clamp is not a reset: the step runs only when a sync persisted mail, so the cycle that detects the gap always carries the message that woke it. All three NOT NULL: a NULL cursor is a predicate that matches nothing, which reads exactly like a working feature. ⚠️ **It deliberately adds NO unique index on `crm_leads.email`**: the cross-invocation double-mint race is accepted (one visible, hand-deletable duplicate) because a UNIQUE constraint on a column where 1,516 imported rows may already carry duplicates is a deploy-blocking migration of exactly the shape 148 had to defuse. Inert until `CRM_AUTO_LEAD` is flipped, which is OWNER-GATE. Spec: crm_app.md §9 WS-26d-autolead). ⚠️ `CREATE TABLE IF NOT EXISTS` means a column or constraint added to an ALREADY-APPLIED migration file is silently skipped on that deployment; 143, 144 and 145 have never been applied anywhere (merging 143 is the OWNER-GATE), which is the only reason they are still editable in place. ## Conventions - Postgres migrations are numbered SQL files diff --git a/infra/postgres/158_crm_auto_lead_cursor.sql b/infra/postgres/158_crm_auto_lead_cursor.sql index d6a752664..e5d89f0f8 100644 --- a/infra/postgres/158_crm_auto_lead_cursor.sql +++ b/infra/postgres/158_crm_auto_lead_cursor.sql @@ -95,6 +95,28 @@ CREATE TABLE IF NOT EXISTS crm_auto_lead_cursors ( updated_at TIMESTAMPTZ NOT NULL DEFAULT now() ); +-- ⚠️ `CREATE TABLE IF NOT EXISTS` above is a NO-OP against a database that +-- already has the table, so it cannot add a column to one. A scratch or dev +-- database that applied this file while it was still the two-column version +-- (it was numbered 157 then, and briefly had no `last_run_at`) would keep the +-- old shape and every cycle would fail on the missing column. These three +-- statements are the repair, and they are all no-ops on a fresh database: +-- * ADD COLUMN IF NOT EXISTS — nullable, because an existing row has no +-- value to give it and NOT NULL would refuse the ALTER outright; +-- * the backfill takes the best answer available, in order: the row already +-- tells us how far the step got, and failing that when it was activated; +-- * SET NOT NULL then restores the invariant, and is itself a no-op when +-- the column was created NOT NULL by the CREATE TABLE above. +ALTER TABLE crm_auto_lead_cursors + ADD COLUMN IF NOT EXISTS last_run_at TIMESTAMPTZ; + +UPDATE crm_auto_lead_cursors + SET last_run_at = COALESCE(processed_watermark, activated_at, now()) + WHERE last_run_at IS NULL; + +ALTER TABLE crm_auto_lead_cursors + ALTER COLUMN last_run_at SET NOT NULL; + -- The candidate query reads this row by primary key, so no second index is -- needed here. This one supports the operator question the log line raises — -- "which mailboxes has auto-lead run on, and when?" — without a seq scan diff --git a/tests/unit/test_crm_auto_lead.py b/tests/unit/test_crm_auto_lead.py index ab2c834ce..cbd07bf0b 100644 --- a/tests/unit/test_crm_auto_lead.py +++ b/tests/unit/test_crm_auto_lead.py @@ -136,6 +136,36 @@ def log(monkeypatch: pytest.MonkeyPatch) -> _Log: return recorder +class _Clock: + """A movable clock for the step's ``now()``. + + The dormancy tests are about ELAPSED TIME BETWEEN INVOCATIONS, and the + honest way to express "nine hours passed and the step was never called" is + to move the clock and not call it — not to hand-seed a ``last_run_at`` the + scheduler could never have produced. The first version of these tests did + exactly that and was green on a state that cannot occur. + """ + + def __init__(self, start: datetime) -> None: + self.at = start + + def __call__(self) -> datetime: + return self.at + + def advance(self, **delta: float) -> datetime: + self.at = self.at + timedelta(**delta) + return self.at + + +@pytest.fixture +def clock(monkeypatch: pytest.MonkeyPatch) -> _Clock: + """Starts at 22:00 — the beginning of the overnight lull these tests are + about.""" + movable = _Clock(datetime(2026, 8, 7, 22, 0, tzinfo=UTC)) + monkeypatch.setattr(auto_lead, "now", movable) + return movable + + @pytest.fixture def quiet_pipeline(monkeypatch: pytest.MonkeyPatch) -> None: """Silence the five mail steps ``process_new_mail`` runs before ours. @@ -1110,89 +1140,145 @@ def test_the_module_registers_no_routes() -> None: # ── done-when 8: an OFF→ON round trip mints nothing from the OFF window ───── -async def test_dw8_an_off_then_on_round_trip_mints_nothing_from_the_gap( - db: FakeCrmDB, on: None, log: _Log, +async def test_dw8_the_first_message_after_an_overnight_lull_is_minted( + db: FakeCrmDB, on: None, log: _Log, clock: _Clock, ) -> None: - """The hole ``activated_at`` alone does NOT cover. - - Flag on day 1 (cursor anchored), off days 2-29, on again day 30. Nothing - about the cursor changed while the flag was off, so the anchor still says - day 1 and every message in the OFF window passes both predicates — the - first ON cycle mints the entire four-week backlog in one batch, each lead - pushing unattended into the live tenant. Re-anchoring on dormancy is what - makes ``activated_at`` mean *the current ON epoch*. + """The regression the CLAMP exists to prevent, in its real shape. + + ⚠️ This step is NOT invoked once per scheduler period. + ``email_ingestion/scheduler.py:463-472`` reads ``synced`` off the sync + result and fires the hook **only when mail was actually persisted**, so a + mailbox with no new mail does not run it at all. Which means the cycle + that trips the dormancy test is ALWAYS the cycle carrying the message that + woke it — and a re-anchor that stamped ``now`` and returned early excluded + exactly that message, permanently, every night and every weekend. + + So the lull below is expressed the way the scheduler produces it: by NOT + CALLING the step. Nothing about ``last_run_at`` is hand-seeded. """ _seed_account(db) _seed_status(db) - _seed_cursor(db, activated_at=_at(1), watermark=_at(1), - last_run_at=_at(1, 10)) # the last cycle before the flag went off - for index in range(27): - _message(db, address=f"stranger{index}@elsewhere.com", - received_at=_at(2) + timedelta(days=index), - processed_at=_at(2) + timedelta(days=index)) - stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + # 22:00 — a sync persisted mail, the step runs, the cursor is activated. + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) - assert stats["reanchored"] == 1 - assert stats["candidates"] == 0 - assert stats["created"] == 0 - assert _leads(db) == [] - cursor = db.rows("crm_auto_lead_cursors")[0] - assert cursor["activated_at"] == cursor["processed_watermark"] - assert cursor["activated_at"] > _at(1) # a NEW epoch, not the old anchor + # 22:05 — an ordinary evening cycle, minting normally. + clock.advance(minutes=5) + _message(db, address="evening@elsewhere.com", + received_at=clock() - timedelta(minutes=2), + processed_at=clock() - timedelta(minutes=1)) + evening = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + assert evening["created"] == 1 + assert evening["reanchored"] == 0 + + # ── THE LULL ── nine and a half hours in which no mail arrives, so the + # scheduler never fires the hook and this step is never entered. + clock.advance(hours=9, minutes=26) + + # 07:31 — a cold prospect wrote at 07:30:45 and the sync persisted it. + _message(db, address="prospect@elsewhere.com", + received_at=clock() - timedelta(seconds=15), + processed_at=clock() - timedelta(seconds=5)) + morning = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + assert morning["reanchored"] == 1, "the lull must trip the dormancy test" + assert morning["created"] == 1, ( + "the message that WOKE the step was excluded by the anchor the same " + "cycle stamped — the overnight regression is back" + ) + assert sorted(row["email"] for row in _leads(db)) == [ + "evening@elsewhere.com", "prospect@elsewhere.com", + ] reanchors = log.at("warning", "sync.auto_lead_reanchored") assert len(reanchors) == 1 assert reanchors[0]["gap_seconds"] > auto_lead.REANCHOR_GAP_SECONDS -async def test_dw8_mail_arriving_after_the_reanchor_mints_normally( - db: FakeCrmDB, on: None, +async def test_dw8_the_clamp_lands_one_gap_width_back_not_at_now( + db: FakeCrmDB, on: None, clock: _Clock, ) -> None: - """The control: re-anchoring must start an epoch, not end the feature.""" + """The mechanism, asserted directly: the anchor moves FORWARD to the start + of the gap window, never to the clock. It is what admits the triggering + message while still excluding the window behind it.""" _seed_account(db) _seed_status(db) - _seed_cursor(db, activated_at=_at(1), watermark=_at(1), - last_run_at=_at(1, 10)) - _message(db, address="old@elsewhere.com", received_at=_at(2)) + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + clock.advance(days=27) + _message(db, received_at=clock() - timedelta(seconds=10), + processed_at=clock() - timedelta(seconds=5)) await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) - assert _leads(db) == [] - # Now a message that arrives after the new anchor. - later = datetime.now(UTC) + timedelta(minutes=5) - _message(db, address="new@elsewhere.com", received_at=later, - processed_at=later) + anchored = db.rows("crm_auto_lead_cursors")[0]["activated_at"] + assert anchored == clock() - timedelta( + seconds=auto_lead.REANCHOR_GAP_SECONDS) + assert anchored < clock() + + +async def test_dw8_an_off_then_on_round_trip_mints_nothing_from_the_gap( + db: FakeCrmDB, on: None, log: _Log, clock: _Clock, +) -> None: + """The hole ``activated_at`` alone does NOT cover, and the case the clamp + must not regress. + + Flag on day 1, off for 27 days, on again. Nothing about the cursor changed + while the flag was off, so a naive anchor still says day 1 and every + message in the OFF window passes both predicates — the first ON cycle + mints the entire four-week backlog in one batch, each lead pushing + unattended into the live tenant (27 of 27, measured). + """ + _seed_account(db) + _seed_status(db) + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + + for index in range(27): + clock.advance(days=1) + _message(db, address=f"stranger{index}@elsewhere.com", + received_at=clock(), processed_at=clock()) + # The flag comes back on a full day after the last of it arrived, so the + # whole window sits outside the clamp's one-hour tail. + clock.advance(days=1) + stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) - assert stats["created"] == 1 - assert [row["email"] for row in _leads(db)] == ["new@elsewhere.com"] + assert stats["reanchored"] == 1 + assert stats["candidates"] == 0 + assert stats["created"] == 0 + assert _leads(db) == [] + assert _activities(db) == [] + assert len(log.at("warning", "sync.auto_lead_reanchored")) == 1 -async def test_dw8_a_quiet_but_running_mailbox_is_never_reanchored( - db: FakeCrmDB, on: None, log: _Log, +async def test_dw8_mail_from_the_last_hour_of_the_off_window_is_minted( + db: FakeCrmDB, on: None, clock: _Clock, ) -> None: - """Why dormancy reads ``last_run_at`` and not ``processed_watermark``. - - A mailbox with no classified inbox mail over a weekend has a watermark 60 - hours old while this step has run faithfully every 600s. Anchoring the - dormancy test on the watermark would re-anchor it and drop the first - message to arrive on Monday morning — which is precisely the message this - feature exists to catch, dropped every Monday. ``last_run_at`` moves on - every cycle, so "quiet" and "not running" stay different facts. + """The DOCUMENTED RESIDUAL, asserted deliberately rather than accidentally. + + Clamping means the tail of the gap window is admitted: mail received + within ``REANCHOR_GAP_SECONDS`` of the flag coming back on IS minted. That + is bounded — one gap width, and at most one capped batch of it — and it is + the price of never dropping the message that woke the step, which is the + same message on every ordinary night. Recorded in `crm_app.md` §9 and in + the owner note on `work_plan.md` §6 (b). """ _seed_account(db) _seed_status(db) - _seed_cursor(db, activated_at=_at(1), watermark=_at(1), - last_run_at=datetime.now(UTC) - timedelta(seconds=300)) - monday = datetime.now(UTC) + timedelta(seconds=1) - _message(db, address="customer@elsewhere.com", received_at=monday, - processed_at=monday) + await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) + clock.advance(days=27) + + _message(db, address="mid_window@elsewhere.com", + received_at=clock() - timedelta(days=3), + processed_at=clock() - timedelta(days=3)) + _message(db, address="half_an_hour_ago@elsewhere.com", + received_at=clock() - timedelta(minutes=30), + processed_at=clock() - timedelta(minutes=30)) stats = await auto_lead.create_leads_from_new_mail(ACCOUNT_ID) - assert stats["reanchored"] == 0 - assert log.at("warning", "sync.auto_lead_reanchored") == [] - assert stats["created"] == 1 + assert stats["reanchored"] == 1 + assert [row["email"] for row in _leads(db)] == [ + "half_an_hour_ago@elsewhere.com" + ], "the accepted bound is ONE gap width — no more, and no less" async def test_dw8_every_cycle_stamps_last_run_at( @@ -1546,6 +1632,25 @@ def test_the_migration_is_idempotent(bare: str) -> None: ) +def test_the_column_added_after_the_first_cut_is_repaired_in_place( + bare: str, +) -> None: + """`CREATE TABLE IF NOT EXISTS` is a no-op against a database that already + has the table, so it cannot ADD a column to one. A scratch database that + applied this file while it was still the two-column version would keep the + old shape and fail on every cycle.""" + assert re.search( + r"ALTER TABLE crm_auto_lead_cursors\s+ADD COLUMN IF NOT EXISTS\s+last_run_at", + bare, + ), "no guarded ADD COLUMN for last_run_at — an already-created table keeps the old shape" + assert re.search( + r"ALTER COLUMN last_run_at SET NOT NULL", bare, + ), "the repaired column must end up NOT NULL like the created one" + assert re.search( + r"UPDATE crm_auto_lead_cursors\s+SET last_run_at = COALESCE", bare, + ), "SET NOT NULL will fail unless existing rows are backfilled first" + + def test_the_migration_drops_or_truncates_nothing(bare: str) -> None: assert not re.search(r"\b(DROP|TRUNCATE|DELETE\s+FROM)\b", bare, re.I) From b672c8b81a56e5c2053d4d2454cb7f5316322c0f Mon Sep 17 00:00:00 2001 From: Vijay Raghav Varada Date: Sat, 8 Aug 2026 04:41:04 +0530 Subject: [PATCH 4/4] docs(WS-26d-autolead): the migration header still described the design we reverted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Comments only — no executable line changed in this commit. The header of 158_crm_auto_lead_cursor.sql was written for the reset-to-now re-anchor and never updated when that became a clamp, so it contradicted the code twice. It repeated the false premise the code and the spec now explicitly disown — that the step "has run faithfully every 600s" on a quiet mailbox, which is why re-anchoring one would drop Monday's first message — and it claimed all three timestamps are re-stamped to now and the gap's backlog mints nothing. None of that is true: only activated_at moves, it moves to now - GAP, processed_watermark is deliberately left alone, and the final gap-width IS minted. That matters more here than in most comments. This block is the canonical description of crm_auto_lead_cursors — it is what the next person to touch dormancy logic reads before they touch it, and a stale comment beside correct code is worse than no comment, because it is evidence. The rewritten header now describes the clamp: which column moves and which deliberately does not, why clamping rather than resetting (the step runs only when a sync PERSISTED mail, so the cycle that detects the gap always carries the message that woke it), the measured failure the reset caused, and the accepted residual stated as a residual. Second fix, same understatement in four places: "up to one capped batch of it" understates the residual. The cap defers rather than drops, so the tail drains ACROSS cycles — 7 messages with a cap of 3 is 3 then 3 then 1, not 3 and the rest lost. The bound is on TIME, not volume: one gap-width of mail, drained across however many cycles the cap takes. Corrected in the migration header, the _reanchor_if_dormant docstring and work_plan §6(b) as asked, and in two places the review did not name but which carry the same sentence — crm_app.md §9's residual paragraph, which is the canonical spec statement of it, and the docstring of the test that asserts the bound. Leaving those two saying something the other three now contradict is the same failure this commit is fixing. ⚠️ Two of the five files are .py, and both diffs are entirely inside docstrings — `git diff` on them shows only comment text. No assertion, no logic, no test behaviour changed, which the re-run confirms: 592 / 24 / 421 across the three blocks, unchanged. Co-Authored-By: Claude Fable 5 --- ai-company-brain/specs/crm_app.md | 9 +-- ai-company-brain/work_plan.md | 3 +- .../gateway/gateway/routes/crm/auto_lead.py | 5 +- infra/postgres/158_crm_auto_lead_cursor.sql | 65 ++++++++++++++----- tests/unit/test_crm_auto_lead.py | 11 ++-- 5 files changed, 65 insertions(+), 28 deletions(-) diff --git a/ai-company-brain/specs/crm_app.md b/ai-company-brain/specs/crm_app.md index 8a59cdde8..927364caa 100644 --- a/ai-company-brain/specs/crm_app.md +++ b/ai-company-brain/specs/crm_app.md @@ -1619,10 +1619,11 @@ touched on re-anchor: OFF-window mail is excluded by the anchor predicate whatev `rules_processed_at` says, and moving it would skip the triggering batch a second way. **Documented residual, accepted:** mail received in the final `REANCHOR_GAP_SECONDS` of -an OFF window IS minted when the flag comes back on. It is bounded — one gap width, and -at most one capped batch of it — and it is the deliberate price of never dropping the -message that woke the step. A named test asserts it as the bound rather than leaving it -to be discovered. Fail-closed otherwise: a missed lead is hand-creatable and visible in +an OFF window IS minted when the flag comes back on. The bound is on TIME, not on volume +— one gap-width of mail (an hour), drained across however many cycles the per-cycle cap +takes, since the cap defers rather than drops. It is the deliberate price of never +dropping the message that woke the step. A named test asserts it as the bound rather than +leaving it to be discovered. Fail-closed otherwise: a missed lead is hand-creatable and visible in the mailbox; 27 unattended pushes into the live tenant are neither. **A failure never advances the cursor past lost work (2026-08-08 diff review, P1-2).** diff --git a/ai-company-brain/work_plan.md b/ai-company-brain/work_plan.md index 786ee97d8..d909d8a84 100644 --- a/ai-company-brain/work_plan.md +++ b/ai-company-brain/work_plan.md @@ -654,7 +654,8 @@ That guard was added by diff review after an OFF→ON round trip was measured mi **27 leads for a 27-day OFF window**, each pushing unattended into the live tenant — so turning the flag off is genuinely a stop, not a pause that accumulates. ⚠️ **With one bounded exception the owner should expect: mail received in the FINAL HOUR before -the flag goes back on IS minted** (up to one capped batch of it). That is deliberate — +the flag goes back on IS minted** — one gap-width of mail (an hour), drained across +however many cycles the per-cycle cap takes, not a single batch. That is deliberate — the re-anchor clamps the epoch one hour back rather than resetting it to now, because this step only runs when a sync persisted mail, so the cycle that detects the gap is always the cycle carrying the message that woke it; resetting would drop that message diff --git a/apps/services/gateway/gateway/routes/crm/auto_lead.py b/apps/services/gateway/gateway/routes/crm/auto_lead.py index 8e6ab88f2..bdb41154b 100644 --- a/apps/services/gateway/gateway/routes/crm/auto_lead.py +++ b/apps/services/gateway/gateway/routes/crm/auto_lead.py @@ -441,8 +441,9 @@ async def _reanchor_if_dormant( where the triggering message always is. ⚠️ **Documented residual:** mail received in the final hour of an OFF - window IS minted when the flag comes back on. It is bounded (one gap width, - and at most one capped batch of it) and it is the deliberate price of never + window IS minted when the flag comes back on. It is bounded by TIME, not by + one batch — one gap-width of mail (an hour), drained across however many + cycles the per-cycle cap takes — and it is the deliberate price of never dropping the message that woke the step. `crm_app.md` §9 records it and ``work_plan.md`` §6 (b) tells the owner about it before they flip. diff --git a/infra/postgres/158_crm_auto_lead_cursor.sql b/infra/postgres/158_crm_auto_lead_cursor.sql index e5d89f0f8..e65189202 100644 --- a/infra/postgres/158_crm_auto_lead_cursor.sql +++ b/infra/postgres/158_crm_auto_lead_cursor.sql @@ -35,23 +35,56 @@ -- wrote its leads, so a failure is never mistaken -- for work done. -- --- last_run_at when the step last RAN. This is what detects --- dormancy — an OFF window, or an outage — and it --- has to be its own column: the watermark tracks --- MAIL, so a mailbox that is merely quiet over a --- weekend has a 60-hour-old watermark while the --- step has run faithfully every 600s. Re-anchoring --- such an account would drop Monday's first --- message, which is exactly the message this --- feature exists to catch. (It also keeps a --- deliberate stall on a poison message stalled, --- instead of quietly re-anchoring past it.) +-- last_run_at when the step last RAN — stamped at the end of +-- every cycle, including the ones that considered +-- nothing and the ones that stalled. This is what +-- detects dormancy: an OFF window, or an outage. -- --- When `now() - last_run_at` exceeds the step's REANCHOR_GAP_SECONDS, --- all three are re-stamped to now: the ON epoch restarts and the gap's --- backlog mints nothing. Fail-closed in both directions — a missed lead --- is hand-creatable and visible in the mailbox; 27 unattended pushes --- into a live tenant are neither. +-- ⚠️ It is its own column for a NARROW reason, and +-- not the one you might expect. Both it and the +-- watermark freeze on a mailbox that receives +-- nothing, because the step is NOT invoked once +-- per scheduler period — `email_ingestion/ +-- scheduler.py:463-472` reads `synced` off the +-- sync result and fires the hook only when mail +-- was actually PERSISTED. What separates the two +-- columns is a cycle that ran and found no +-- CANDIDATES (mail outside the inbox, held back, +-- or predating the anchor): that advances +-- `last_run_at` and not the watermark. It is also +-- the state a deliberate stall holds the watermark +-- in, so keying dormancy on the watermark would +-- re-anchor past a stall and quietly undo it. +-- +-- WHAT HAPPENS ON A GAP. When `now() - last_run_at` exceeds the step's +-- REANCHOR_GAP_SECONDS, the step CLAMPS `activated_at` forward to +-- `now() - REANCHOR_GAP_SECONDS` — it does NOT re-stamp all three, it +-- does NOT set anything to `now()`, and it does NOT skip the batch: +-- +-- * `activated_at` moves forward to the start of the gap window. +-- * `processed_watermark` is deliberately left ALONE. OFF-window mail +-- is already excluded by the anchor predicate +-- whatever its `rules_processed_at` says, and +-- moving it would skip the triggering batch a +-- second way. +-- * `last_run_at` is advanced by the ordinary end-of-cycle +-- stamp, like any other cycle. +-- +-- Clamp rather than reset, because the step runs only when a sync +-- persisted mail — so the cycle that DETECTS the gap is always the cycle +-- carrying the message that woke it. Resetting the anchor to `now()` and +-- returning early therefore excluded that message permanently: measured +-- as no mail 22:00→07:30, a prospect writes at 07:30:50, the step runs at +-- 07:31:05, and the one lead it existed to catch fell fifteen seconds the +-- wrong side of the anchor its own arrival created. Every night, every +-- weekend. +-- +-- ⚠️ ACCEPTED RESIDUAL, and it is not "nothing": mail received in the +-- FINAL REANCHOR_GAP_SECONDS of the window IS minted — one gap-width of +-- mail (an hour), drained across however many cycles the per-cycle cap +-- takes. Everything older stays excluded, which is the fail-closed +-- property that matters: a missed lead is hand-creatable and visible in +-- the mailbox; 27 unattended pushes into a live tenant are neither. -- -- ⚠️ There is deliberately NO unique index on `crm_leads.email` to go -- with this. Two concurrent `process_new_mail` invocations for one diff --git a/tests/unit/test_crm_auto_lead.py b/tests/unit/test_crm_auto_lead.py index cbd07bf0b..a3debf542 100644 --- a/tests/unit/test_crm_auto_lead.py +++ b/tests/unit/test_crm_auto_lead.py @@ -1255,11 +1255,12 @@ async def test_dw8_mail_from_the_last_hour_of_the_off_window_is_minted( """The DOCUMENTED RESIDUAL, asserted deliberately rather than accidentally. Clamping means the tail of the gap window is admitted: mail received - within ``REANCHOR_GAP_SECONDS`` of the flag coming back on IS minted. That - is bounded — one gap width, and at most one capped batch of it — and it is - the price of never dropping the message that woke the step, which is the - same message on every ordinary night. Recorded in `crm_app.md` §9 and in - the owner note on `work_plan.md` §6 (b). + within ``REANCHOR_GAP_SECONDS`` of the flag coming back on IS minted. The + bound is on TIME, not on volume — one gap-width of mail (an hour), drained + across however many cycles the per-cycle cap takes, since the cap defers + rather than drops. It is the price of never dropping the message that woke + the step, which is the same message on every ordinary night. Recorded in + `crm_app.md` §9 and in the owner note on `work_plan.md` §6 (b). """ _seed_account(db) _seed_status(db)