feat(WS-29): MT-0 blockers + MT-1 scaffolding; migrations 157-159 scratch-verified (H1) - #404
Merged
Merged
Conversation
… model said so
`tenancy_and_visibility.md` §1 set the tenant boundary at the deployment on
2026-08-03 and §6 put row-level tenancy, an org switcher and multi-org users out
of scope. That was the right call for one company on one box. It is the wrong
call for a product sold per module, per user, per month — §1.4 of the new spec
prices the difference: a full stack per customer is ~6-8 GB of RAM and 156
migrations replayed per box per deploy, against a 10-seat customer's revenue.
§6 of that document prescribes the procedure for changing its mind: re-take the
decision in writing, with a date and a reason. This is the re-take.
The cost objection it recorded — "a WHERE organization_id = ? on 111 tables and
every query in the gateway" — turns out not to hold, and the reason is a seam
that already exists. `acb_common/db.py` is ONE engine and ONE `get_db()`, pinned
by `test_db_engine_seam.py`. So tenancy installs as a generated migration
(org_id column with a `current_setting` default + FORCE RLS policy per table),
one `SET LOCAL app.tenant_id` in `get_db()`, and a non-owner role — and zero
existing queries change. Two more seams were already built for other reasons and
turn out to fit: `EffectiveAccess.intersect()` is exactly the entitlement mask
module gating needs, so `require_permission("feature:crm")` inherits it with no
route edits; and `_emit_usage()` already computes per-call tokens and USD cost at
`/v1`, streamed responses included, so reselling AI is four additions to a
metering path rather than a subsystem.
What the decision costs is recorded too. §1.1 of the old document concluded that
leak sites 1-10 "cannot fire" because there is one organization row; that premise
is gone, so §6.4/§6.5 hand them back as work, and WS-14a's three slug-only
org_group joins go from cleanup to prerequisite. Two items are hard blockers
before a second tenant exists at all: integration credentials reach agents
through process-global `os.environ` (`executor.py:4388` — the code already says
so at `:4364`), and native-MAF self-mutation opens PRs against this monorepo,
which root AGENTS.md non-negotiable 3 already forbids for multi-tenant.
Anchors are measured against this tree, not inherited: `_ORG_MEMBER_SQL` is at
access.py:400 and `_HAS_OWNER_SQL` at :522, where the old document publishes :338
and :460. That document has had two anchor-correction passes for the same reason.
Visibility is untouched. §2-§5 of `tenancy_and_visibility.md` — the private →
Center → org ladder, the `group:` project grant, the per-surface gap table —
stay binding, and §7.8 restates the standing rule against a second scoping
doctrine. Tenancy is which company; visibility is who inside it.
Nothing is built. §5 is the phased plan and §8 lists what still needs an owner
answer, price points first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…process Review question from the owner, and the review found a real error. §0 claimed "one engine, one session factory, one get_db()" without qualification and §1.3 built "zero existing queries change" on top of it. That holds for the gateway request path. It does not hold for the process: there are eight distinct connection paths, and the seam test only ever guarded one call name. Measured and enumerated in a new §0.1 — the shared async seam; three email_ingestion engines (allow-listed, separate process); acb_graph's SYNC create_engine, which the ratchet does not inspect because it only looks for create_async_engine; three raw psycopg.connect callers (key_store, model_config, org_settings) that were invisible to it entirely; and Mem0, where the connection is opened by a third-party library from a conninfo string we hand it. An implementer who took the unqualified claim at face value would have bound the tenant in get_db(), watched the request path work, and shipped six unbound paths. This makes the RLS recommendation stronger rather than weaker, and the correction is what shows why: a server-side policy covers paths nobody enumerated, whatever driver opens them, and it fails CLOSED — unset app.tenant_id yields NULL, so `organization_id = NULL` returns zero rows and an unconverted path breaks loudly in a test instead of quietly serving another tenant in production. Phase 1 gains the ratchet extensions and an explicit Mem0 decision, and 3-4 weeks becomes 4-5. Two owner questions were also unanswered by the first draft and are now sections rather than assumptions. §1.6 — physical layout at multi-GB per tenant. Pooling is a logical decision and says nothing about layout. Measured where the gigabytes actually are: pgvector dominates (1536-dim float32 is ~6 KB, roughly double with HNSW, so ~1-1.5 GB for one tenant's email index), then agent_blob.content BYTEA, then bodies and FTS. Meeting media is already on the filesystem. The reframe: move blobs to object storage and the relational working set is hundreds of MB, not multiple GB — worth doing in any topology. Total size is not the constraint; working set vs RAM, HNSW memory and restore time are. Partition the heavy tables by tenant (LIST for the big ones, HASH for the tail), build per-tenant logical backup as its own capability rather than as a side effect of picking silo, and keep the eviction path warm, because a tenancy model you cannot reverse is the actual risk. §1.7 — one container per org on the SAME VPS, rejected as its own option since it is not the same proposal as one VPS per customer. It is the worst of the three: N Postgres containers cannot lend each other memory, so it fragments the exact resource pooling exists to share; on shared hardware it partitions no kernel, no page cache and no disk queue; it keeps every operational cost of database-per-tenant; and one box's exhaustion takes down every tenant, which is silo's ops cost with pool's blast radius. §1.8 — schema-per-tenant, which the first draft dismissed too quickly and which is the strongest rejected alternative. It keeps one instance, so the memory argument survives, and pg_dump -n gives per-tenant backup for free. It loses on one property: RLS fails closed, search_path fails open. A wrong tenant id returns zero rows; a wrong search_path returns a complete, valid-looking result set belonging to someone else, silently. With eight connection paths that difference is the whole argument. Recorded with the condition under which to re-take it — a few dozen large accounts rather than many small ones, which is the same condition that makes the silo tier make sense. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…answers Q5 Owner input: every customer is a company of 10-50 users, not an individual. That retires §1.4's lead argument, so §1.4 is rewritten rather than left standing. A 25-user company on three modules at ~₹500/user/module is ~₹37,500/month against a ~$35 VPS — about 8% of revenue. "The SMB tier does not exist under this model" was priced for a customer a quarter that size and is withdrawn. Dedicated infrastructure is not what breaks at this ACV. What survives is the cost that scales in people rather than servers, and it survives harder: 156 migrations times N boxes every deploy is linear in customers and paid weekly by a small team; at 20 customers deploy babysitting, per-box backup verification and N incident surfaces are realistically half an engineer, permanently. And N boxes means N versions of the access-control code in production — a migration that fails on customer 14 leaves customer 14 running the old permission check, which given what 130_org_access_control.sql defines is a security defect, not an ops annoyance. Pooled cannot have that failure at all. Crossover with Phase 1's one-time 4-5 weeks is ~8-12 customers. Which answers §8's open question 5, now struck and moved to a new §5.1: silo customers 1-5, build Phase 1 alongside, cut over at 8-12. Below the crossover silo is genuinely cheaper AND reaches revenue sooner, and five hand-run deployments teach you the module split and the price points — the two inputs §8 still lists as open — which is worth more than a month of architecture built against guesses. Four conditions keep it a bridge rather than drift, the load-bearing one being that every silo runs the pooled schema with organization_id populated and RLS on from day one, so a silo is a pooled deployment with N=1 and cutover is a data move. Skipping that is what turns the bridge into a rewrite. Plus a written cutover trigger, because a bridge with no trigger is a destination. New §1.4a records the WordPress question, because the intuition is common and reasonable and points the other way once followed through. Hostinger silos because the CUSTOMER owns the install — it cannot know or migrate a schema its customers install plugins into, and customer A may run WP 5.8 while B runs 6.4. That is the determining question, who controls the schema and the upgrade cadence, and CommandCenter is on the other side of it. WordPress's own answer when WordPress is the SaaS is Multisite: one database, per-site table prefixes over shared network tables, sharded 16/256/4096 ways at WordPress.com scale — not a database per site. And Multisite's per-site table prefix is schema-per-tenant in a different costume, hitting exactly the catalog pressure §1.8 predicts, which is why large networks shard. The example that looked like it argued for silo turns out to be independent confirmation of §1.8. §1.8's flip condition is tested against the new input and NOT met: 10-50 seats is mid-market, the core segment of Slack, Notion, HubSpot, Freshworks and Zoho, all pooled. The flip needs FEW customers; 10-50 seats implies the opposite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
… are retrofit Owner asked whether this recommendation survives without the anchor of what CommandCenter already is. New §1.8a is the audit, recorded because a reader two years out has to be able to tell "we chose this" from "we inherited this". Most of it survives. Pooled over silo at 10-50 seats rests on a comparison set — Slack, Notion, HubSpot, Freshworks, Zoho — that inherited nothing from us. RLS over application filtering rests on fails-closed vs fails-open, a property of the mechanisms. Container-per-org on shared hardware is resource arithmetic. Entitlements-not-permissions, credits-not-tokens and seats-not-actives are business principles with no code dependency. Three things do not survive, and are now labelled. Every "the seam already exists" argument is retrofit reasoning. Greenfield you write the tenant column into the first migration and the question evaporates. Relatedly, the 4-5 week Phase 1 estimate is entirely a retrofit number — the greenfield equivalent is about three days of schema discipline. That is the largest distortion in the document: this debate is expensive HERE because 143 tables were built without a tenant column, which is not evidence the decision is hard in general. §3.1's "don't add a proxy" is ~60% retrofit and its confidence was overstated. Greenfield, buying an AI gateway is close to a coin flip; what survives is that a separate proxy must re-resolve the tenant (a second boundary to get right) and lacks the app context per-module margin analysis needs. Conclusion unchanged, confidence lowered. The audit also produced two additions to Phase 1, both cheap now and expensive later. Treat organization_id as a distribution key rather than a filter column — in every PK and index prefix, related tables colocated — which costs nothing and is the precondition for sharding, where retrofitting it means rewriting every primary key. (The discipline, not Citus; Citus is unnecessary at this scale.) And make the external-IdP question an explicit call rather than a default, since SAML/SCIM is a tar pit to build, while noting honestly that the shipped RBAC is good enough that migration cost may already exceed the benefit. One argument was under-weighted and is independent of the codebase: agents here execute model-generated tool calls over content ingested from untrusted sources, which is a higher risk profile than ordinary SaaS and a real point in silo's favour. It does not flip the decision — an injected agent already holds its own tenant's data, and RLS blocks the incremental step at the server — but two things become non-negotiable rather than advisable, and Phase 1 now carries them: no agent ever gets a raw-SQL tool, and no agent-reachable path can set app.tenant_id. If either is violated, pooled is not defensible and §1 should be re-taken. Finally, the go-to-market risk no architecture answers: if two customers are competitors — plausible selling manufacturing software from a manufacturer — "is my data in the same database as theirs" is a procurement question, and "no, separate database" beats explaining a row-level policy. A sales argument for the dedicated-data tier, and the reason tenant_placement earns its keep on day one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…ge, not from it Owner asked whether it is dangerous that Google Workspace keeps every organization in one database. New §1.1a, because the premise carries a category error worth fixing permanently and the correction surfaced a layer this document was missing entirely. Workspace is not one database in any physical sense. It is one logical namespace sharded by customer across thousands of machines — Spanner and Colossus, customer as partition key — so one customer's data sits in its own key range on its own hardware, addressed through one logical system rather than N administratively separate ones. "Pooled" is a statement about the schema, not about the hardware. That is the same thing §1.8a's distribution-key discipline buys in miniature. The honest half is conceded rather than argued away: pooling concentrates consequence, and one authorization bug is potentially every customer where a silo would be one. No architecture argument makes that untrue. What the section adds is where the safety actually comes from, which is neither the topology nor luck. Two layers, both built precisely because the storage is pooled. Above it, Zanzibar — one central authorization service every product must ask, trillions of ACLs, millions of checks a second, sub-10ms p95, five nines, so no product re-implements access control and there is exactly one place to get it right that cannot be forgotten because there is no other way to answer the question. Below it, per-customer encryption keys: storage splits data into chunks encrypted with keys separate from other customers' and from other chunks of the same customer's data, so pooled storage is not pooled plaintext. The transferable rule: safety comes from a single un-forgettable enforcement point plus a layer beneath it that fails safe, not from how many database processes are running. Silo buys a weak version of that guarantee; a policy the database enforces is the stronger version, because it survives a developer forgetting. RLS is already this document's Zanzibar-analogue at our scale. The second layer was absent and is now a Phase 5 item: per-tenant envelope encryption for integration credentials, provider keys, message bodies and transcripts, DEK per tenant wrapped by a master KEK. It makes a raw storage or backup compromise tenant-scoped rather than global — the specific residual risk pooling introduces and the only one silo genuinely answered. Flagged to pull forward into Phase 1 if those columns are being touched anyway, since retrofitting encryption onto populated columns is materially harder than doing it at rest-write time. §1.4's comparison is unchanged: silo shrinks one category of bug, does nothing about the categories behind most real breaches, and adds wrong-database routing plus N versions of the access-control code in production. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…rements, not one Owner question, and the strongest argument for silo raised so far — strong because it attacks §1.4a's own test. If the customer controls the upgrade cadence then the test points at silo, and this document has to follow its reasoning rather than defend its conclusion. New §1.4b does that, and the honest answer is that three of the four things the phrase can mean are already fine and the fourth is real. The dividing line is not versions, it is schemas. Multiple CODE versions against one database is what every blue/green deploy and canary already does. Multiple SCHEMA versions in one database is impossible. So staged rollout and release channels are fine under expand/contract discipline — additive-only, never rename in place, drop the old column only after every running version has passed the read step, with a view per version where two versions need different shapes. That covers two or three adjacent versions over weeks. It does not cover eighteen months of drift. The case owners usually mean is per-customer features and configuration, and the finding that matters is that this platform already solved it without forking: Custom Apps are DB rows (114), Workflows are DB-persisted configuration orchestrating code-authored agents by explicit design (ADR-028), dynamic_agents are registered not compiled (15), custom fields are a table plus JSONB (155), and org/workflow/plugin/project/agent config is JSONB throughout. The design premise is that customers extend with data rather than forks, and per-tenant data is what a pooled database is good at. Reaching for per-customer code versions to deliver what the configuration layer already delivers trades a solved problem for an unsolved one. What customers asking for "our own version" almost always want is "don't change things under me without warning" — a release channel plus feature flags, not a fork, which is why no major SaaS offers version pinning and all of them offer rollout control. Google Workspace ships exactly this as Rapid vs Scheduled Release: same code, same storage, admin picks. Per-org feature flags are added to Phase 2, evaluated beside the entitlement mask since it is the same lookup shape. Genuine version pinning is the one case pooled cannot serve. Those customers go on the silo tier, which now has three unrelated demands pointing at one mechanism — compliance, the competitor objection, and version pinning — which is a good sign the tier is drawn correctly. Priced at what it costs, and capped contractually at current-plus-one, because a cap written after the first pinned customer is a negotiation and written before is a policy. Recorded with the condition that overturns §1: if pinning stops being the exception, the §1.4a test has flipped and we are on Hostinger's side of the line. Nothing in §5 is wasted if that happens — the silo customers are already the mechanism, and every silo running the pooled schema is what keeps both doors open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
Owner asked what the architecture should be given a clean slate. New §0.9 answers it ahead of §1, so the destination is readable without the retrofit argument attached — and answering it properly moves the emphasis off the layer this document has spent five revisions arguing about. Almost every multi-tenancy discussion is a database discussion. For CommandCenter that is the wrong emphasis for a product-specific reason: agents execute model-generated tool calls over input arriving from email and WhatsApp, where prompt injection is routine rather than exotic, and they write and run code. The database can be defended by a mechanism nobody can forget — a server-enforced RLS policy that fails closed. The agent runtime has no equivalent; no policy engine constrains what a model decides to do with the tools it holds. So the instinct that produced "a container per customer" was sound and pointed at the wrong layer. Put the container around the agent RUN, not around the database. That also explains why pooled-vs-silo has felt unsatisfying throughout: both options isolate the layer that was already easier to isolate. Three planes, three tenancy models. Control plane shared and cross-tenant by design, holding no tenant business data so a compromise exposes contracts rather than customers' mail. Data plane pooled behind RLS. Execution plane ephemeral per-run sandboxes, and that is where the budget goes. The execution contract is four clauses, each closing a named hole: one tenant binding; only the credentials that run needs, expiring with it; NO database connection at all, reaching data solely through a tenant-bound API; allowlisted egress; destroyed at end of run. The second clause is what makes a pooled data plane defensible given model-generated tool calls — an agent that cannot open a connection cannot escape RLS, cannot set app.tenant_id, and cannot be SQL-injected into another tenant. The third matters because isolation without egress control is theatre. This inverts a live owner decision, and the inversion is the point: WS-3's T2 tier is parked under D10 on the explicit ground that the ladder must hold against trusted colleagues rather than hostile users. Selling to external customers replaces that threat model, so un-parking T2 stops being optional hardening and becomes the architecture. P5-a already shipped, so the hard conceptual piece exists; what is parked is the tier above it. Phase 0 grows accordingly, 1-2 weeks to 2-4. Also recorded clean-slate: drop Neo4j, because Community offers one database and no real multi-tenancy, so the graph becomes a tenancy problem with no good answer — an edge table in Postgres inherits RLS for free, and removing a datastore removes a boundary, which is a security improvement and not just an ops one. Redis prefixing enforced by a wrapper client rather than convention, because a convention is a thing people forget. Per-tenant vector namespaces, HNSW being the one place physical separation earns its cost on merit. Placement indirection from day one, which is what lets the silo tier, the competitor objection and version pinning all be answered by one mechanism. Three invariants everything collapses to, the third being the one an ordinary SaaS architecture would not tell you to write down: agents hold no ambient authority. §0.9.8 maps the clean-slate target against §5 rather than leaving two plans in one document. The phased plan converges on the target for the data plane and diverges on the execution plane, which is the honest gap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
Consolidation pass. Five rounds of research and four revisions lived in a spec with no board row, which by this repo's own rules means it could not be dispatched: the loop selects from work_plan.md §2, and §1's seven-point contract is what makes a ticket handable to an agent. This closes both gaps. New §11 is the ticket list. MT-0 (four blockers) and MT-1 (nine tenancy tickets) carry the full contract — gate label, testable done-when, current anchors, verification commands. MT-2 through MT-5 are deliberately scoped-but-not- dispatchable, each naming the one input that would make it so, because writing acceptance for billing automation today would be inventing it and §1 point 3 forbids "owner call" dressed up as a criterion. The sequencing has one non-obvious edge worth stating: MT-0 does not block selling. The first five customers ship as silos while MT-1 is built. But MT-0a/b/d must be in before customer #2 regardless of topology, because process-global credentials and monorepo self-mutation are process-level defects, not database-level ones — silo does not fix either. MT-0c is the load-bearing ticket and it is OWNER-GATE. §0.9's thesis is that for this product the interesting boundary is the execution plane rather than the database: the database can be defended by a policy nobody can forget, and the agent runtime cannot. That makes the per-run sandbox architecture rather than hardening — and it inverts D10, which parked T2 on the ground that the ladder need only hold against trusted colleagues. Selling externally retires that threat model. Registered in §6 so an agent refuses it and says why. Board gains WS-29 and the decision register gains D15, which re-takes D11 with the reason and the consequences: row-level tenancy, org switcher, multi-org users and per-org credentials all move into scope, where D11 §6 listed all four as out; leak sites 1-10 stop being moot and become MT-1i, including _HAS_OWNER_SQL at access.py:522, a lockout RLS does not fix. D11's §2-§5 survive untouched — this changes tenancy only, never visibility. WS-14a is absorbed as MT-1i rather than left to collide with it. §11.2 is the week-one list, and its first item is a decision rather than a build: take MT-0c, because everything in §0.9.3 waits behind it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…ons tenancy D15 changed a rule nine documents restate, and a decision that only lives in the doc that made it is a decision the next agent will contradict. This is the DOX pass. New child spec, saas_multitenancy_implementation.md — SHAPES, not decisions, on the same contract user_management_contract.md uses: every decision cited to the parent, none re-argued, parent wins on disagreement. It holds what §11's tickets reference but do not contain, so MT-n is not re-derived per ticket: the generated per-table RLS template and the four clauses in it that each have an incident waiting behind them (the current_setting DEFAULT that is the entire "zero queries change" property, FORCE without which the table owner reads everything, WITH CHECK without which a tenant can WRITE another tenant's id, and the missing-ok flag whose absence turns "no rows" into a 500 on every unconverted path); composite-PK discipline and the FK-graph warning; the acb_app grants; the DB-backed coverage ratchet whose exemption list IS the security review; the tenant_session seam with SET LOCAL inside an explicit transaction, because SET LOCAL outside one is a silent no-op that presents as "the feature is broken" rather than "tenancy is broken"; the two new ratchets; Mem0's three options as a decision table; control-plane, entitlement and metering DDL; the sandbox contract and its no-DB-credentials assertion; three runbooks; and a ten-row trap table. Two traps in it are worth naming here because they are how tenant #2 actually fails, and neither is a leak: role seeding is still keyed slug='default', so org #2 gets no roles at all, and _HAS_OWNER_SQL is unfiltered, so org #2 is ownerless and nobody can grant access back. A lockout RLS does not fix. Paid a debt this workstream created: MT-1f promised user_management_contract.md would gain a rule and had not. R11 — never take the acting TENANT from input — is now written, with the reasoning that makes it R3's twin rather than a footnote: an identity you can spoof gets you one person's data, a tenant you can spoof gets you a company's. Two corollaries an implementer would otherwise shave: a background job carries its tenant on its job record and refuses to run without one, and a subdomain is a lookup to verify against the session, not an assertion — a trusted subdomain is a header with a friendlier name. The doc's fact-owner table now splits tenancy from visibility, since D11 was re-taken and D12 was not. Root AGENTS.md gains constraint 11 and its non-negotiable 3 now points at MT-0b as a ticketed hard blocker rather than a standing warning. Constraint 10's "ten binding rules" is corrected to eleven. Five docs that contradicted D15 are fixed rather than left to be discovered: department_centers.md's tenant rule (the "never for a department" half survives unchanged; the "separate deployment is for a separate organization" half was D11), agent_persistence_implementation.md and agent_file_and_memory_framework.md where agent_name is described as "the sole tenant key" — true under one deployment per tenant, ambiguous under pooled, and now pointing at MT-1b/MT-1g — colleague_onboarding.md's two D11 citations, and the single-owner registry, which now has separate rows for tenancy and visibility because they have separate owners. The gateway and workbench AGENTS.md files gain R11 where their builders will actually read it: the gateway because binding happens once in acb_common.db and a route reaching data any other way is the bug, the workbench because the subdomain lands there first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…he process The executor bridged resolved integration credentials to skills through the gateway's process-global os.environ. B6 Phase-5 Tier 0 scoped that with a restore token, which removed permanent accumulation but could not remove concurrent exposure — and the docstring said so itself: "os.environ is process-global, so under concurrent in-process runs the scoping is best-effort — two overlapping runs still share the env for the overlap window." Under one tenant that is a within-org concern, which is why it was deferred. Under two it is a credential leak, and the agents doing the reading execute model-generated tool calls over content ingested from email and WhatsApp. It is MT-0a in saas_multitenancy.md §6.1, a hard blocker before customer #2, and it blocks whether or not the tenancy work lands, because it is a process-level defect that separate databases do not fix. The bridge is now a ContextVar in acb_skills.integrations — per-task, copied into tasks created from the binding context, so the overlap window does not exist. bind/release keep the executor's existing call shape, so the three bind sites and three teardown sites are renames rather than restructuring. Verified red before the fix and quoted in the spec: replaying the test's exact interleaving against the old implementation printed "run B saw run A's : clk-secret-123". Only one direction reproduced — the other was masked because run B's teardown completed before run A resumed, which is itself a demonstration of how timing-dependent the old scoping was, and the reason the new test asserts both directions rather than one. Finding that shaped the change: only THREE in-process lines actually read these credentials from the environment (skill-clickup-sync/core.py twice, web_tools.py once); they now call integrations.credential(). The ~20 os.getenv calls in integrations.py are resolvers reading the OPERATOR's .env — a different thing, and they correctly still do. Subprocess scripts were already served entirely through code_tools._script_env, which now reads the binding. Two limits are deliberately left open and written down rather than papered over. An operator-provided env value still wins and is still process-global — unchanged precedence, and making that store per-tenant is MT-0d. And the declared-LIST is still a process-global dict despite a docstring calling itself coroutine-local, so a concurrent run can still widen which names are looked up; that is now much less dangerous, because a widened name yields nothing unless this run also holds the credential. 12 new tests pass, 72 across the affected fence. No lint regressions on any touched file; three files improved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…rovider keys Two blockers from saas_multitenancy.md §6, both live the moment a second company exists on the box, neither fixed by separate databases. MT-0b — self-mutation containment. Root AGENTS.md non-negotiable 3 has said since it was written that native MAF agents land self-mutations by opening a PR against THIS monorepo and that third parties must never push to it. Nothing enforced it, and work_plan.md WS-3 records why: no first_party field existed anywhere, "the phrase occurs only in comments and one test helper". Migration 157 creates organization.first_party, DEFAULT false, backfilling the operator's own org to true — so today's behaviour is unchanged and every org created afterwards is contained by construction rather than by someone remembering. The gate runs FIRST in attempt_self_mutation, before the attempt tally, the sandbox, git or the network, because a tenant's failure should cost us nothing. It fails closed on every path: unreachable DB, missing column, and the multi-tenant case, where the untenanted query requires count(*) = 1 rather than falling back to the default org. That last choice is the containment — a default-org fallback would keep answering true after tenant #2 arrived, which is precisely the leak. SELF_MUTATION_DISABLED=1 is an operator hard-off that short-circuits before any query. One test deliberately guards the other direction, that a first-party org still reaches the tally, because "refuse everything" would pass every other assertion while silently switching the feature off for Fracktal too. MT-0d — credentials stop being deployment-wide. provider_keys was `provider TEXT PRIMARY KEY`, one key per provider for the whole box, and mcp_servers, plugins and model_config had no owner column at all. tenancy_and_visibility.md §1.1 called that "exactly the right shape" and it WAS, under D11; under D15 it means tenant B's agent resolves tenant A's OpenAI key. Migration 158 re-keys all four, and refuses to re-key rather than proceed if any row would be left ownerless. The resolution rule is the part worth reviewing. _resolve_org(None) resolves to THE SOLE organization — literally count(*) = 1 — so the ~20 existing call sites keep working unchanged today and every one of them fails closed the moment a second org exists, which is exactly when MT-1 must supply a real tenant. Reads return empty; writes raise, because a credential written with no owner is how a key ends up readable by the wrong tenant, and a write is a better place to fail loudly than a read. The in-memory cache was the other half and correct SQL does not protect it: ProviderKeyStore._cache was keyed by provider alone, so the second tenant asking for "openai" would have been served the first tenant's DECRYPTED key straight from memory with no query issued. Now keyed (organization_id, provider), pinned by its own test. Both suites verified red first — 14 of 16 failed against the pre-fix source (the 2 that passed assert migration file contents, which do not depend on it). 113 tests pass across the affected fence. No lint regressions; model_config improved 4 → 0.⚠️ Migrations 157 and 158 are NOT verified against a live database — no Docker daemon in the build environment. Statically checked instead: all four auto-generated constraint names confirmed against schema.generated.sql, and no foreign key anywhere references the primary keys being re-pointed, so the drop-and-re-add is safe. Run both against a scratch Postgres before deploying; apply_migrations.sh replays from 02_ upward and a failure there fails the deploy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…both ways
Owner delegated the MT-0c call. Interrogating it rather than rubber-stamping it
found that the ticket was the wrong shape AND that it was already violated.
saas_multitenancy.md §0.9.3 says "no agent ever gets a raw-SQL tool" and makes it
a CONDITION on the pooled tenancy decision. query_history already was one:
a model-generated SQL string executed through acb_graph.get_session() —
connection path 4 in §0.1, the sync create_engine the seam ratchet never
inspected — registered in agent-orchestrator/config.json, injected at
_tool_injection.py:623, advertised to the model as "Run a SELECT-only SQL query".
Its keyword-substring guard was wrong in both directions, measured:
SELECT role, content, created_at FROM chat_message → REJECTED
because CREATED_AT contains the substring CREATE. That is the tool's own
documented example, so most real queries failed.
SELECT * FROM provider_keys → ALLOWED
as were email_messages, app_user and every other table. The guard policed
verbs; nothing policed tables.
So this was a live within-org read primitive today, independent of tenancy, and a
cross-tenant one the day MT-1 lands. query_history now takes search criteria —
values from the model, never syntax; a fixed statement with bound parameters over
the two tables it always documented; results narrowed to the acting member's own
sessions when the run context names one, which the old tool never did despite its
docstring example reading another member's conversations by email address. A
build-failing ratchet stops the shape returning.
D16 splits MT-0c, recorded agent-proposed and overrulable per the D13 convention.
MT-0c-1 ships now because it is cheap and closes a live defect. MT-0c-2 — the
container/microVM tier — stays parked and stays OWNER-GATE, because D10's
reasoning survives the phase we are actually in: with one tenant per box, an
escaped agent reaches only the data it already had. T2 becomes a precondition of
the §5.1 pooled cutover at customer 8-12, not of Phase 0. Building
Firecracker-grade isolation before customer #1 exists is speculative
infrastructure paid for out of the runway that should be buying customers.
The split does not weaken §0.9.3. Both stated conditions hold: the first is now
shipped, and the second cannot be violated before app.tenant_id exists (MT-1b).
The container tier is defence in depth on top of them.
9 new tests, verified red first (6 of 9 failed against the SQL version). 101 pass
across the skills surface. The pinned call contract in test_tool_schema_diet.py
was updated deliberately — that ratchet exists to stop a contract changing by
accident and it caught this correctly, which is the system working.
Phase 0 is now complete: MT-0a, MT-0b, MT-0c-1, MT-0d all built.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…not wired up MT-1a — the control plane. Migration 159 adds tenant_placement, user_identity and org_membership, plus acb_common/placement.py to resolve them. Placement is the piece that makes D15 reversible. Every row is (pool, primary) on day one, and that is the point rather than an argument against it: moving a customer to their own database becomes a data move plus a row update instead of an architecture change made under pressure while that customer waits. Three unrelated demands land on this one mechanism — the dedicated-data tier, the "is my data in the same database as my competitor's" procurement question, and version pinning. §1.6's line governs: a tenancy model you cannot reverse is the actual risk. Deliberately ADDITIVE AND INERT. app_user is untouched and still authoritative; 159 seeds a shadow copy so the tables are populated and inspectable. Cutting the auth path over is MT-1a-2 and is NOT bundled here — acb_auth/access.py carries two ON CONFLICT (email) upserts on the live sign-in path, and a half-migrated identity is worse than an unmigrated one. (The spec cited those upserts in members.py; measured, they are not there — another stale anchor, corrected by grep rather than trusted.) Caught one real bug in my own migration before it shipped: the seed read u.name, and the column is display_name. MT-1b — the RLS migration exists as a GENERATOR, and its output is not a numbered migration. That is the judgment call in this commit and it is not timidity. Read scripts/apply_migrations.sh: its lock-timeout design exists because of a 14h44m outage where a hung session held a lock, the runner queued an ACCESS EXCLUSIVE behind it, and Postgres's FIFO lock queue put every later reader behind the WAITING alter. Sending mail stopped. MT-1b is that exact shape, 135 tables over — ADD COLUMN NOT NULL and SET NOT NULL scan and take ACCESS EXCLUSIVE, the backfill rewrites every row, and there is no database in this environment to try any of it against. So the generator emits four separately-appliable phases into infra/postgres/generated/, outside the sequence the deploy replays: nullable add (no scan), batched backfill (re-runnable), constraints (the ACCESS EXCLUSIVE phase), policies (instant — and a cliff, because the moment it applies every connection that has not bound app.tenant_id reads zero rows). That ordering is not a preference: MT-1c must be deployed and verified before phase 4 or the product goes dark. Promoting these is a human act in a window, not a consequence of a file reaching main. Discovery: 146 tables, 135 tenant-scoped, 11 exempt. The exemption map is the security review — each entry carries its reason, and test_tenant_coverage.py trips if the list grows past 15. The generator also reports exempt names that match no table, which correctly flagged _prisma_migrations as LiteLLM's vendored schema rather than ours. The coverage ratchet has two layers because they fail at different times: source-level (always runs) catches a new table when its migration lands; DB-level (skips without Postgres, and says so loudly in the skip reason) is the only thing that proves the migration was APPLIED rather than merely written. A green run without it proves the SQL exists, nothing more. 13 tests pass, 2 skip for want of a database. Ruff clean on all new files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…entory missed
The seam ratchet inspected `create_async_engine` only. So acb_graph's SYNC
`create_engine` was invisible to it, and the three raw `psycopg.connect` callers
were invisible to everything. §0.1 exists because of that gap; this closes it
with two AST ratchets, allow-list-with-a-reason, in the shape
test_db_engine_seam.py already established.
Red-verified in both directions, which is the part that matters for a ratchet:
a scratch probe calling all three constructors tripped all three rules; a second
probe using `from psycopg import connect as _pg_connect` was caught by alias,
while the same identifiers written in a DOCSTRING tripped nothing. That is the
AST-over-grep property actually holding rather than being asserted.
And building it found something reading could not. §0.1 said "measured
repo-wide" and listed eight paths. It was measured across apps/ and packages/ —
which is precisely where the ratchets scan, and therefore exactly where a blind
spot cannot hide. Two more live in scripts/:
scripts/import_hr_people.py:177 create_async_engine — and it UPSERTs people
rows, which are TENANT DATA. Once phase-4
policies are on it will either fail or write
unowned rows.
scripts/check_infra.py:40 psycopg.connect — reads pg_extension only,
no tenant data, benign.
Both verified independently before accepting them. The inventory was wrong in
the same shape as the thing it was documenting: a scan whose roots decide its
answer. §0.1 now lists ten with each disposition, and records that rows 9 and 10
were found by BUILDING the ratchet rather than by reading — the second time this
workstream has had to correct that section, and worth leaving visible.
Deliberately not widened: the ratchet roots stay at apps/ and packages/. Pulling
scripts/ in is an application-of-policy call, not ratchet mechanics, and it
belongs with the decision about how operator scripts bind a tenant at all.
19 tests pass. Ruff clean on both files. Confirmed still true today: no
asyncpg.connect, no psycopg2, no psycopg_pool, no sqlite3 anywhere in apps/ or
packages/ — and the psycopg ratchet records that measurement date so a future
pool import gets the rule widened rather than allow-listed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
MT-1c binding half — `tenant_session()` in acb_common/db.py. RLS enforces isolation server-side but only against a session that has told the server which tenant it acts for, and this is the one place that happens. That singularity is what makes the pooled decision defensible across ten connection paths nobody can hold in their head — an inventory that has now been wrong twice. Two failure modes are pinned as tests rather than trusted to review. SET LOCAL, never SET: the engine pools connections, so a session-scoped SET survives the connection's return to the pool and the next borrower — different request, different customer — inherits it. One word, and the only symptom is a cross-tenant read under concurrency that no unit test reproduces by accident, so the source is asserted directly. And begin() before the SET LOCAL: outside a transaction it is a SILENT no-op, the policy then sees an unset GUC and every query returns nothing, which reads as "the feature is broken" rather than "tenancy is broken". get_db() is deliberately unchanged and now documents that it is NOT tenant-bound — ~200 call sites still use it, and a docstring is the only thing between a reader and the assumption that it is safe once policies apply. Converting them is the rest of MT-1c. MT-1i — the leak sites D15 un-mooted. Five predicates, all DERIVED from the row being authorised rather than from a constant, because tenancy_and_visibility.md §2 done-when 1 is explicit that a hardcoded slug='default' swaps one wrong constant for another. The two org_group slug-only joins in rooms.py now tie g.organization_id to the caller's own row; the inline my_groups query was lifted to MY_GROUPS_SQL so a hermetic string test can reach all three (done-when 2). _GROUP_MEMBER_SQL and _ORG_MEMBER_SQL join the actor and constrain both sides to the actor's org — so the `org` subject now means "active members of the actor's organization" instead of every row in app_user. _HAS_OWNER_SQL asks whether the org being provisioned into has an owner, rather than whether an owner exists anywhere: that one was never a leak, it was a LOCKOUT, and RLS does not fix it. Anchor (c) was stale by ~62 lines — the spec published access.py:330-336 and _GROUP_MEMBER_SQL is at :392-398. Exactly the failure this corpus keeps having, caught because the anchors were re-derived by grep instead of trusted. Verified red first, with the six failures quoted in the work log, including _GROUP_MEMBER_SQL joining org_group with no org tie and _ORG_MEMBER_SQL as a bare `SELECT email FROM app_user WHERE status = 'active'`. Fail-closed direction checked: an actor with no row, or a NULL organization_id, expands to nothing and the run keeps the actor's own access — narrowing, never widening, which is the only safe direction for an intersection fold. One judgement call, flagged rather than buried: _BOOTSTRAP_ORG_SLUG was introduced so the owner guard and the bootstrap insert cannot disagree about which org they mean. Two independent copies of that literal IS the bug. A rename, not a semantic change; site 8's hardcoding still stands and its real fix is org provisioning. 121 tests pass across the affected fence, zero skipped. Zero lint introduced — verified against a HEAD worktree rather than asserted.⚠️ NOT discharged: tenancy_and_visibility.md §2 done-when 3, the DB-backed two-org behavioural fixture. It belongs in test_session_authority.py / test_rooms.py, both of which skip entirely here (21 skipped, no reachable Postgres). That criterion needs a live database and is explicitly still open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
…what needs a database Four tickets change state, and three of them are PARTIAL rather than done. That distinction is the point of this commit: a board that reads green when the migration has never been applied is worse than one that reads red. MT-1a partial — schema built and inert; the auth-path cutover is MT-1a-2 and is not bundled, because acb_auth/access.py has two ON CONFLICT (email) upserts on the live sign-in path and a half-migrated identity is worse than none. MT-1b generated but NOT applied, deliberately outside the replayed sequence. MT-1c seam and ratchets built; ~200 call sites and the Mem0 decision remain. MT-1i built, with §2 done-when 3 explicitly still open for want of a database. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
RLS protects Postgres. It does not protect Redis, and today's keys carry no
tenant at all. §1.9's requirement is not "prefix the keys" — it is that the
wrapper be the only way in and that it be INCAPABLE of producing an unprefixed
key, because a convention is a thing people forget.
So `TenantKey.value` is composed from validated fields on every access rather
than stored, leaving no accessor that can return something shorter;
`__post_init__` re-runs the builder's checks so direct construction is not a
bypass and also requires the org to match the currently bound one. `TenantRedis`
enumerates its command surface instead of proxying — no `__getattr__`, private
underlying client — and every command funnels through one `_raw()` that refuses a
plain `str` and refuses a key whose org differs from the bound one, which catches
a key captured in a closure and used after a rebind. Tenant comes from a
ContextVar with no `default=`, mirroring MT-0a's credential binding, and
`current_organization()` raises rather than returning a fallback.
No existing call site was touched. Converting ~58 key sites across 10 clients is
a separate, riskier change and must not ride along; the migration path is in the
module docstring, including the instruction NOT to write a dual-read shim — every
key here is cache, presence or a bounded stream, so conversion is a cache-cold
event rather than a data migration.
42 tests, ruff clean, both verified independently before this commit rather than
taken on report. Two AST ratchets, both verified red with probe files: one on any
module importing `redis`, one on hand-written `cc:` literals — the second exists
because a module can build an untenanted key and hand it to someone ELSE's
client, which the import ratchet cannot see. Both have stale-entry tests, so the
allow-lists can only shrink.
Three findings the ratchets cannot cover, all spot-verified against the tree:
chat.py:707 SCAN match="cc:active:*" — a cross-tenant session enumeration
the moment a second tenant exists. Highest severity in the
inventory.
consumer.py:95 _GROUP = "cc-ingest" is ONE consumer group shared by every
tenant; §1.9 requires one per tenant.
non-cc keys ingestion:{clickup,zoho,gmail,dlq}, session_mem:,
email:att:cache: are untenanted and match no `cc:` ratchet.
Plus orchestrator/agents.py:436, which hands redis_url to
agent_framework's RedisHistoryProvider — chat history keyed
outside this wrapper entirely, needing its own decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
The wrapper is built; the ~58 call sites are not converted, and that split is deliberate. What matters more than the status is the three findings a ratchet structurally cannot see, now recorded where the follow-up ticket will read them: a SCAN that enumerates every tenant's sessions, a consumer group shared across all tenants, and untenanted namespaces that carry no cc: prefix at all — including chat history keyed by agent_framework outside this wrapper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
New saas_multitenancy_handover.md — the thing to hand an agent that has a database. Everything in WS-29 was built without one, which is why three migrations have never been applied and the RLS phases sit outside the deploy sequence. It owns order, gates and verification rather than architecture; §0 is a paste-ready brief, and H1→H8 each carry a gate that must pass before the next begins. The correction that changes the plan's size: the session-conversion surface is 561 sites across 138 files, not the "~200" this spec has been claiming. The dominant idiom is the ALIASED import — `from gateway.db import get_db as _get_db`, then `await _get_db()`, 441 of the 561 — so every grep for `get_db()` missed four fifths of it. Heaviest files are tasks/items.py (23), notes/meeting_bot.py (18), and three email/automation modules at 14-15 each. That reorders the work. H2 (conversion), not H3 (the RLS migration), is the long pole, and the runbook says so in its own section rather than leaving an implementer to discover it in week three. It also warns what a missed binding looks like in practice: an EMPTY SCREEN, not an error, because fail-closed means zero rows. Ordering is stated as one non-negotiable and everything else movable: H2 before H3, because phase 4 is a cliff — the instant policies apply, any connection that has not bound app.tenant_id reads nothing. Rollback for that phase is given inline (DISABLE ROW LEVEL SECURITY), since it is the only phase anyone will want to roll back. H1 says to restore a production dump rather than use an empty database, because an empty one hides every backfill and constraint problem these migrations can have; names the duplicate-key risk in 158's four PK swaps to check BEFORE applying; and points at the u.name/display_name bug caught by reading the schema dump rather than by testing, with the instruction to assume there is another one. Also carried forward so they are not lost: the three Redis sites no ratchet can catch (chat.py's SCAN across every tenant's sessions; one consumer group shared by all tenants; untenanted non-cc: namespaces including chat history keyed outside the wrapper by agent_framework), the three criteria that need a live database, the seven standing rules, and what is deliberately out of scope — MT-0c-2 under D16, and MT-2…MT-5 blocked on owner inputs rather than on engineering. Board row and planning index updated to point at the runbook first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QPR4UXjHeLReVSdntribcT
… the operator test_tenant_coverage.py's two database-backed tests gate H3 on DATABASE_URL — set means "the operator pointed me at a migrated Postgres", unset means skip loudly. Measured on a dev machine with a root .env: `import litellm` (reached through acb_llm by test_integration_env_scoping.py's executor import) calls load_dotenv() at import time, which plants the .env's local-dev DATABASE_URL into os.environ mid-collection. Collected after that file, the two gates un-skipped and ran against an unmigrated localhost database — 2 failed where the no-.env build environment showed 2 skipped, and the baseline's meaning depended on which machine ran it. tests/conftest.py now snapshots DATABASE_URL before any test module imports (conftest imports first, ahead of any litellm import), and _db_ready() answers to the snapshot. An operator-exported DATABASE_URL still un-skips both tests — verified against a fully migrated scratch: both run and fail on their real assertions (130 tables missing scoping; superuser connection), which is the correct red at the H1 stage. Baseline after the fix: 213 passed, 2 skipped, 0 failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-09 consolidation Rows carried up to 29.5k chars each (~77k tokens for §2) while next-ticket.md tells the supervisor to read §2 only. Rows now carry state + gates + pointers; narratives moved verbatim to owning specs as "Board record (2026-08-09)" sections (D18.2). Also: R5 minted (tenant-ready by construction — owner call D18.1, parallel + ratchet); D17 (Mem0 Option A) and D18 recorded; D11 bannered superseded; D10.1 premise re-scoped with expiry annotations; D9 phrase re-swept; D13/D14 annotated; 18 stale-vs-merged row claims corrected (branch protection, ledger, backups, deploy recovery, WS-13/26/27 states); §6 rotation unblock. Entry points rebuilt: planning AGENTS.md drops the lying 2026-06-20 built-table, gains the 16 missing index rows; apps/AGENTS.md and the glossary drop the "zero handlers" falsehood; root AGENTS.md constraint 11 carries R5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…corded §8 items 1-2 ANSWERED by the owner: Core ₹600/user/mo + ₹300/module add-ons; AI resells as a ₹10 "AI action" credit at ~50% gross margin (rate card = cost×2) — MT-2/MT-3 now lack only their seven-point ticket contracts, not inputs. Mem0 path-8 binding decided as Option A, conninfo options (D17, agent-proposed). §5.1 cutover trigger marked ADOPTED. MT-1a's stale members.py:173/access.py:447 anchor pair corrected to the measured access.py:205/:509 in both specs. Handover: status → in execution; H1 RESULT block added (scratch gate PASSED 2026-08-09 — full-ladder replica, 213 passed/2 skipped, prod apply = PR #404, verify by the three deploy-log ok lines); state table re-dated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ge re-swept for D15
Every compacted work_plan.md §2 row narrative now lives verbatim in its owning
spec under "Board record (2026-08-09)", with that day's corrections enumerated
(18 files). The D15/D16 conflict inventory is fixed across the corpus:
deployment-per-tenant claims, "internal Fracktal tool" planning premises,
"second tenant deployment" (D9) phrasing, and slug='default' teachings now sit
behind superseded fences or carry dated annotations — rewrite class:
agent_platform_hardening §1.5 (D16 banner), permissions_sandbox P5-c/d,
DESIGN_LIMITATION_native_maf_mutation ("not settled yet" was false — settled
2026-08-08, MT-0b built). multi_user_organization_research §17.3 gains the
REJECTED-MECHANISM banner (X-Organization-Id, forbidden by R11). Known-false
claims corrected where found: "zero handlers", "no branch protection",
"no Center reachable", deploy-path BROKEN (recovered 2026-08-06/07 UTC, tip
health-verify failure open). Status headers added to 5 headerless files and
re-dated where they contradicted fact; stale-warning banners on reference.md,
system_architecture.md, agent_repo_compatibility.md, FOUNDATION_CONTINUATION.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ows the seam it locks Two red checks on PR #404, both real: - attempt_self_mutation ran the first-party DB gate before the max-attempts short-circuit, so an at-the-limit run cost a DB round-trip and, where the DB is unreachable (CI), returned the gate's failure reason instead of the limit's. The limit check is now a pure in-memory peek that runs first; the first-party gate still precedes the attempt registration, so a tenant run refused by the gate consumes nothing - both invariants hold. - The trajectory eval still called executor._inject_integrations_to_env, which 722cf2a (MT-0a) replaced with the ContextVar binding in acb_skills.integrations. The eval now locks the same invariant through the new seam, and additionally asserts credentials never appear in os.environ - the specific regression MT-0a exists to prevent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vjvarada
pushed a commit
that referenced
this pull request
Aug 9, 2026
…d have applied `main` landed the SaaS multi-tenancy programme in #404 while this branch was open. Merged in rather than left to rot; four conflicts, three migration-number collisions, and one real defect on the far side. ## Conflicts - `work_plan.md` — took main's restructured Apps table (the D18 consolidation moved narratives into board records); WS-27's row updated for o–t, WS-29's for what this branch carries. - `project_management_app.md` — both sides append at the end; kept both, board record last. - `test_crm_agent.py` — main's parametrised structure with THIS branch's assertion. Main's asserted the error names `ACB_AGENT_USER_EMAIL`; S1-4 deleted that fallback, so the message is now about the missing actor. - `test_org_access_control.py` — both additive, kept both. ## Renumbered: 157/158/159 -> 160/161/162 Main's H1 applied ITS 157/158/159 on a full-ladder replica, so those numbers are spoken for. Ours become 160_projects_recurrence, 161_projects_tenancy, 162_app_user_email_case, with every reference rewritten. Phrase-anchored, not bare-number: `admin/roles.py:112,157` and `search.py:145-158` are line numbers. Verified on a live Postgres 16: 157→162 apply in order, and re-apply, clean. `app_user_email_lower_key` present and `app_user_email_key` gone; `pm_tasks .organization_id` NOT NULL -> `organization`; `provider_keys` PK is `(organization_id, provider)`. ## D-MT-2 is answered, and not by us D15 — pooled, RLS, `app.tenant_id` bound by `tenant_session()`. `saas_multitenancy .md` is canonical. `multi_tenancy.md` keeps its measured record and says so at the top; WS-29c is struck as superseded by MT-1b + MT-1c. Migration 161 stays, because the generated set is deliberately NOT replayed by `apply_migrations.sh` and `routes/projects/core.py` reads that column on every request — the two compose, and 161's header now documents the seam. ##⚠️ The defect: MT-1b would have failed in the maintenance window `gen_tenant_migration.py` scoped by column NAME. `crm_contacts`, `crm_deals` and `crm_activities` have an `organization_id` that REFERENCES `crm_organizations` — a customer company, not the tenant. For each it emitted a no-op phase 1, then: UPDATE crm_contacts SET organization_id = <tenant> WHERE organization_id IS NULL; Reproduced live: ERROR: insert or update on table "crm_contacts" violates foreign key constraint "crm_contacts_organization_id_fkey" On an EMPTY table those same statements return `UPDATE 0` / `ALTER TABLE` and look healthy. It needs one real row — the condition production has and a fixture does not — and it fails after phase 1 has already run. This is the same homonym that corrected this branch's own published count from 6 scoped tables to 3. `test_tenant_coverage.py` could not see it: both its layers match the column name, and its source-level test asserts that exempt entries have reasons, not that every table lands in a bucket. Fixed at generation time instead of apply time: `discover_homonyms()` derives the conflicting tables from the migrations, `HOMONYM_BLOCKED` is the human sign-off, and the generator refuses to emit and exits 1 when they disagree. Blocked tables are listed in every generated file's header, for whoever is holding the psql prompt. Regenerated: 133 tables scoped, 3 blocked. NOT filed in `EXEMPT`, deliberately — exempt means "needs no isolation", and those three hold customer CRM records. They carry none today, which is a hole, and `EXEMPT` is where a hole would stop being reviewed. Closing it is a rename across every CRM route: an owner call, written up in `multi_tenancy_leak_audit.md` §2.1. ## The ratchet survives, retargeted `test_tenancy_boundary.py` matches the FK's TARGET, which is the thing `test_tenant_coverage.py` structurally cannot, so it is kept rather than retired as a duplicate. It now reads `EXEMPT`/`HOMONYM_BLOCKED` from the generator rather than keeping a second copy of a decision, and adds three assertions: every table lands in exactly one bucket (the partition the other file's docstring promises and its code does not check), the blocked list is derived not asserted, and an exemption cannot launder a homonym. Baseline 123 -> 113 as main's MT-0d keyed four more tables and MT-1a's control plane became exempt. Mutation-tested: dropping one entry from `HOMONYM_BLOCKED` turns three tests red and makes the generator exit 1; reverted byte-identically. ## Verification 5586 backend passed / 49 skipped · 1106 frontend · 208 theme · `tsc`, `ruff`, `xenon` clean · migrations 157→162 applied and re-applied against live Postgres 16 · all four generated phases driven against `pm_tasks` to prove they compose with 161 (second FK accepted, RLS policy created). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VmFScimSbeyHcLdut7RT4W
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this shipsThe WS-29 multi-tenancy branch: MT-0a (per-run credentials), MT-0b (self-mutation containment, migration 157), MT-0c-1 (no raw-SQL agent tools), MT-0d (per-org provider keys, migration 158), MT-1a partial (control plane, migration 159 + placement.py), MT-1b generated RLS files (NOT applied — they live outside the deploy sequence by design), MT-1c binding seam + ratchets, MT-1e Redis wrapper. Specs and the execution runbook (
saas_multitenancy_handover.md) ride along.Merging this = H1's "apply to production via the normal deploy". The deploy ledger will apply 157/158/159 once, with the pre-migration backup taken first (fail-closed).## H1 verification (2026-08-09, local scratch — first time these migrations touched any database)- Full ladder 00→156 replayed onto a fresh pgvector:pg16: 154 files, 0 failures.- Seed exercised every backfill path: case-duplicate emails (app_user.emailis case-sensitively unique; 159 must collapse them), NULLdisplay_name, NULLorganization_id, empty email, rows in all four re-keyed credential tables.- 157/158/159 applied clean; idempotent re-run clean; every runbook §H1 verify query correct: -organization:default | first_party=t, sole row - composite PKs:provider_keys(organization_id, provider),model_config(organization_id, key),mcp_servers(organization_id, name);plugins_org_name_key UNIQUE (organization_id, name); zero NULL orgs -tenant_placement= orgs = 1 (pool/primary/ap-south-1);user_identity= distinctlower(email)= 4 (earliest row's casing + display_name won);org_membership= 3 (NULL-org and empty-email rows correctly excluded)- Baseline suite: 213 passed, 2 skipped, 0 failed (the 2 skips are the H3 DB gates, as documented).- Red-check: withDATABASE_URLpointed at the migrated scratch, both H3 gate tests run and fail on their real assertions (~130 tables unscoped; superuser connection) — the gates are not vacuous.## One defect found and fixed here (817596b5)import litellmcallsload_dotenv()at import time. On any machine with a root.env, pytest collection planted the localDATABASE_URLintoos.environmid-run, silently un-skipping the two DB gate tests and pointing them at an unmigrated local database (2 failed where the build env showed 2 skipped).tests/conftest.pynow snapshotsDATABASE_URLat launch and the gates answer to the snapshot. Operator-exported URLs still work — verified both directions.## Owner checklist before/at merge1. (Recommended, ~10 min) Repeat the scratch check against a production dump on the box — the one thing a local replay cannot cover is prod-only data states. Commands are in the session handoff.2. Merge. Watch the deploy log for the three- 15N_*.sql ... oklines — the log lines, not the job conclusion, are the evidence (deploy-verify has blessed dead services before).3. Post-deploy, run the runbook §H1 verify queries against prod.🤖 Generated with Claude CodeAdded 2026-08-09 — documentation consolidation rides this PR
Owner-directed consolidation of the work plan + corpus for multi-tenancy (52 files, 3 commits:
258ab6be,25534b65,a06be620):work_plan.md§2 rows compacted from up to 29.5k chars to ≤~2k (state + gates + pointers); narratives moved verbatim into owning specs as "Board record (2026-08-09)" sections with dated corrections. §2 drops from ~77k to ~7k tokens.saas_multitenancy.md§8).multi_user_organization_research.md§17.3 header-tenancy carries a REJECTED banner (R11).b09093a8health-verify failure open — box one docs-only commit behind), stale MT-1a anchors.Verified by an adversarial fresh-eyes pass (residual-language sweep, board integrity, record round-trip, index completeness, contradiction check, truncation guard); its 12 findings were repaired before commit.