Skip to content

[codex] Add scoped MCP toolkits#1145

Merged
RhysSullivan merged 12 commits into
mainfrom
codex/toolkits-scoped-mcp
Jun 26, 2026
Merged

[codex] Add scoped MCP toolkits#1145
RhysSullivan merged 12 commits into
mainfrom
codex/toolkits-scoped-mcp

Conversation

@RhysSullivan

@RhysSullivan RhysSullivan commented Jun 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • Serve toolkit-scoped MCP resources on local and keep session ids bound to the exact MCP resource.
  • Add black-box local toolkit MCP coverage plus shared target coverage for cloud, self-host, and Cloudflare.
  • Harden Cloudflare e2e/runtime compatibility for toolkit MCP: protected-resource metadata routing, streamable HTTP e2e client usage, local-network egress for e2e, legacy D1 connection schema repair, and malformed integration config cleanup.
  • Keep toolkit policy replacement semantics intact. Toolkit policies replace the active provider for that toolkit resource instead of composing with global policies.
  • Fix toolkit loading states so direct toolkit hard loads use the detail skeleton immediately instead of flashing the grid skeleton.
  • Refine toolkit grid cards to match their skeleton layout: no MCP path, no scope label, no timestamp, neutral add-card styling, and icons composed from connected toolkit connections.

Visual Evidence

First step of the self-host toolkit UI scenario, showing toolkit cards with the refined layout and no path/scope/date metadata:

Toolkits card layout

Validation

  • bun run lint
  • bun run --cwd packages/plugins/toolkits typecheck
  • bun run --cwd e2e test:selfhost selfhost/toolkits-ui.test.ts
  • bunx oxfmt --check apps/host-cloudflare/src/db/data-migrations.ts apps/host-cloudflare/src/db/data-migrations.test.ts apps/host-cloudflare/src/mcp/auth.ts apps/host-cloudflare/wrangler.jsonc e2e/scenarios/toolkits-mcp.test.ts e2e/src/surfaces/mcp.ts e2e/setup/cloudflare.boot.ts e2e/local/toolkits-mcp.test.ts
  • bun run --cwd apps/local typecheck (passes with existing typed-error warnings in apps/local/src/main.ts)
  • bun run --cwd apps/host-cloudflare typecheck
  • bun run --cwd apps/host-cloudflare test -- src/db/data-migrations.test.ts
  • bun run --cwd packages/plugins/google test -- src/sdk/openapi-ownership-migration.test.ts
  • cd e2e && bunx --bun vitest run --project local local/toolkits-mcp.test.ts
  • cd e2e && E2E_NODE_BIN=/opt/homebrew/bin/node bun run test:cloudflare scenarios/toolkits-mcp.test.ts
  • cd e2e && bun run test:selfhost scenarios/toolkits-mcp.test.ts
  • cd e2e && bun run test:cloud scenarios/toolkits-mcp.test.ts

Notes

  • cd e2e && bun run typecheck is currently blocked by the unrelated untracked e2e/scripts/google-oauth-matrix.ts, which has missing imports and type errors outside this PR scope.

Header Polish Evidence

Toolkit detail header after removing the duplicate tool count and tightening the top bar:

Toolkits detail header polish

Additional validation for the header polish:

  • bunx oxfmt --check packages/plugins/toolkits/src/page.tsx
  • git diff --check -- packages/plugins/toolkits/src/page.tsx
  • bun run typecheck in packages/plugins/toolkits
  • bun run test in packages/plugins/toolkits
  • bunx vitest run --project selfhost selfhost/toolkits-ui.test.ts in e2e

Hidden Personal Connections Note

The manage-connections dialog now explains when personal connections are hidden from a shared workspace toolkit:

Toolkits hidden personal connections note

Additional validation for the note:

  • bunx oxfmt --check packages/plugins/toolkits/src/page.tsx e2e/selfhost/toolkits-ui.test.ts
  • git diff --check -- packages/plugins/toolkits/src/page.tsx e2e/selfhost/toolkits-ui.test.ts
  • bun run typecheck in packages/plugins/toolkits
  • bun run test in packages/plugins/toolkits
  • bunx tsc --noEmit in e2e
  • E2E_SELFHOST_URL=http://100.81.219.45:44614 E2E_SELFHOST_ADMIN_EMAIL=admin@e2e.test E2E_SELFHOST_ADMIN_PASSWORD=e2e-admin-password-123 bunx vitest run --project selfhost selfhost/toolkits-ui.test.ts in e2e

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 26, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 3f68c6a Commit Preview URL

Branch Preview URL
Jun 26 2026, 10:03 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 26, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 3f68c6a Jun 26 2026, 10:04 PM

@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@pkg-pr-new

pkg-pr-new Bot commented Jun 26, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@1145

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@1145

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@1145

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@1145

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@1145

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@1145

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@1145

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@1145

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@1145

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@1145

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@1145

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@1145

executor

npm i https://pkg.pr.new/executor@1145

commit: 3f68c6a

@RhysSullivan RhysSullivan changed the title Add scoped MCP toolkits [codex] Add scoped MCP toolkits Jun 26, 2026
@RhysSullivan RhysSullivan marked this pull request as ready for review June 26, 2026 18:54
@greptile-apps

greptile-apps Bot commented Jun 26, 2026

Copy link
Copy Markdown

Greptile Summary

This PR introduces scoped MCP toolkits: each toolkit gets its own /mcp/toolkits/:slug endpoint, with session IDs bound to the exact resource that minted them. It ships the full toolkit plugin (storage, policy provider, HTTP API), hardens Cloudflare e2e/runtime compatibility (protected-resource metadata routing, legacy D1 schema repair, malformed config cleanup), and refines the toolkit grid/detail UI.

  • Core policy layer (packages/core/sdk/src/executor.ts): a new ToolPolicyProvider seam lets plugins replace the global tool_policy table with a custom rule source per executor instance; toolkit sessions wire this to per-toolkit allow-lists, blocking personal tools from org-owned toolkits and enforcing capability boundaries.
  • Transport layer (packages/hosts/mcp/src/envelope.ts, apps/local/src/mcp.ts): /mcp/toolkits/:toolkitSlug is added alongside /mcp; every session is now bound to the exact McpResource (default or toolkit) that created it, rejecting cross-resource reuse with a 403.
  • Cloudflare hardening (apps/host-cloudflare/src/db/data-migrations.ts, apps/host-cloudflare/src/mcp/auth.ts): ensureCloudflareD1SchemaCompatibility repairs the legacy item_id \u2192 item_ids column rename and nulls out non-JSON integration configs; toolkit-scoped protected-resource metadata routes are registered for all three hosts (cloud, selfhost, Cloudflare).

Confidence Score: 4/5

Safe to merge; the core resource-binding, auth, and policy-enforcement paths are well-tested. Two non-blocking concerns exist in the toolkit server's legacy-policy heuristic and connection-list query amplification.

The isLegacyConnectionPolicy heuristic can silently skip an intentional approve policy, and connectionsList with an active toolkit provider triggers O(3xN_tools) DB queries. Neither breaks existing flows, but the heuristic can produce wrong policy outcomes for certain toolkit configurations.

packages/plugins/toolkits/src/server.ts (isLegacyConnectionPolicy heuristic and its interaction with connectionsList in executor.ts)

Important Files Changed

Filename Overview
packages/plugins/toolkits/src/server.ts New file: full toolkit plugin server implementation. Manages toolkit CRUD, connection/policy storage, and ToolPolicyProvider wiring. The isLegacyConnectionPolicy heuristic can misclassify intentional approve policies as legacy proxies.
packages/plugins/toolkits/src/shared.ts New file: HTTP API schema definitions (ToolkitsApi group) and shared response/payload types for the toolkits plugin. Looks correct.
packages/core/sdk/src/executor.ts Adds ToolPolicyProvider support: replaces global policy resolution with a pluggable provider. connectionsList now calls toolsList when a provider is active, causing O(3N) DB queries per call for the toolkit provider. Functionally correct but has a performance concern.
packages/hosts/mcp/src/envelope.ts Adds /mcp/toolkits/:toolkitSlug route alongside /mcp; mcpDispatch is now resource-aware. Proactive session teardown on Forbidden was removed (noted in prior review thread).
apps/local/src/mcp.ts Adds per-resource executor creation, session-to-resource binding, and toolkit path parsing. Executor handle leak on pre-initialized-session errors was flagged in a prior thread and is unchanged here.
apps/host-cloudflare/src/db/data-migrations.ts Adds ensureCloudflareD1SchemaCompatibility: fixes malformed integration configs and migrates legacy item_id schema. The non-atomic rebuildLegacyConnectionTable was flagged in a prior thread.
apps/host-cloudflare/src/mcp/auth.ts Adds toolkit-scoped protected-resource metadata routes for Cloudflare Access. Dynamic slug extraction from URL, resource metadata response building, and metadata URL routing look correct.
apps/host-selfhost/src/mcp/auth.ts Adds toolkit-scoped protected-resource metadata handling for selfhost. Uses indexOf for slug extraction (vs. startsWith in Cloudflare), but since slugs are [a-z0-9-] the difference is harmless.
packages/hosts/cloudflare/src/mcp/session-store.ts Passes McpResource through to DO headers on both new-session creation and session forwarding. The resource binding check in the DO base class relies on sessionMeta.resource being present; pre-migration DOs without this field will crash (flagged in prior thread).
packages/hosts/mcp/src/in-memory-session-store.ts Extends session ownership to include resource; forward() and create() now check resource key equality. resource ?? defaultMcpResource in dispatch is a safe but redundant fallback since McpResource is non-nullable.
apps/cloud/src/mcp/mount.ts matchMcpSuffix now handles toolkit slug segments in both bare and org-prefixed forms. bareMcpPath updated to produce /mcp/toolkits/:slug paths. Logic looks complete for all 4 URL patterns.
packages/core/sdk/src/plugin.ts Adds ToolPolicyProvider and ToolPolicyProviderRule types plus the toolPolicyProvider hook on PluginSpec. Interface is clean; list() is required, resolve() is optional, only one plugin may provide the active source.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Client
    participant Envelope as MCP Envelope
    participant Auth as McpAuthProvider
    participant Store as McpSessionStore
    participant DO as Session DO
    participant Exec as Executor

    Client->>Envelope: POST /mcp/toolkits/:slug
    Envelope->>Auth: authenticate(request)
    Auth-->>Envelope: Authenticated(principal)
    Envelope->>Store: dispatch resource toolkit slug
    Store->>DO: createSession(token, resource)
    DO->>Exec: createExecutorHandle activeToolkitSlug
    Note over Exec: toolPolicyProvider replaces global tool_policy
    Exec-->>DO: ExecutorHandle
    DO-->>Client: 200 OK + mcp-session-id

    Client->>Envelope: POST /mcp/toolkits/:slug + mcp-session-id
    Envelope->>Store: dispatch existing session
    Store->>DO: forward + verify resource key matches
    DO-->>Client: MCP response
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Client
    participant Envelope as MCP Envelope
    participant Auth as McpAuthProvider
    participant Store as McpSessionStore
    participant DO as Session DO
    participant Exec as Executor

    Client->>Envelope: POST /mcp/toolkits/:slug
    Envelope->>Auth: authenticate(request)
    Auth-->>Envelope: Authenticated(principal)
    Envelope->>Store: dispatch resource toolkit slug
    Store->>DO: createSession(token, resource)
    DO->>Exec: createExecutorHandle activeToolkitSlug
    Note over Exec: toolPolicyProvider replaces global tool_policy
    Exec-->>DO: ExecutorHandle
    DO-->>Client: 200 OK + mcp-session-id

    Client->>Envelope: POST /mcp/toolkits/:slug + mcp-session-id
    Envelope->>Store: dispatch existing session
    Store->>DO: forward + verify resource key matches
    DO-->>Client: MCP response
Loading

Reviews (9): Last reviewed commit: "Fix console route contract for nested to..." | Re-trigger Greptile

Comment on lines 549 to +552
const matches =
accountId === sessionMeta.userId && organizationId === sessionMeta.organizationId;
accountId === sessionMeta.userId &&
organizationId === sessionMeta.organizationId &&
resourceKey === mcpResourceKey(sessionMeta.resource);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Missing resource on pre-deploy DO sessions causes TypeError

sessionMeta.resource will be undefined for any Durable Object session that was created before this PR is deployed, because the stored JSON won't contain the resource field. When such a session receives a request under the new code, mcpResourceKey(sessionMeta.resource) immediately throws TypeError: Cannot read properties of undefined (reading 'kind') — both for warm DOs (where this.sessionMeta was populated by old init() code) and cold DOs (where stored JSON is deserialized without resource).

A safe fix: mcpResourceKey(sessionMeta.resource ?? defaultMcpResource). The same default applies to the resolveAndStoreSessionMeta result for the cloud (apps/cloud/src/mcp/session-durable-object.ts) and the Cloudflare host (apps/host-cloudflare/src/mcp/session-durable-object.ts).

Comment thread packages/hosts/mcp/src/envelope.ts
Comment on lines +44 to +104
readonly name?: unknown;
}>();
return new Set(
result.results
.map((row) => row.name)
.filter((name): name is string => typeof name === "string"),
);
};

const rebuildLegacyConnectionTable = async (db: D1Database): Promise<void> => {
const statements = [
`DROP TABLE IF EXISTS connection_next`,
`DROP TABLE IF EXISTS connection_legacy_item_id`,
`CREATE TABLE connection_next (
integration text NOT NULL,
name text NOT NULL,
template text NOT NULL,
provider text NOT NULL,
item_ids json NOT NULL,
identity_label text,
description text,
tools_synced_at integer,
oauth_client text,
oauth_client_owner text,
refresh_item_id text,
expires_at integer,
oauth_scope text,
oauth_token_url text,
provider_state text,
created_at integer NOT NULL,
updated_at integer NOT NULL,
row_id text PRIMARY KEY NOT NULL,
tenant text NOT NULL,
owner text NOT NULL,
subject text NOT NULL
)`,
`INSERT INTO connection_next
(integration, name, template, provider, item_ids, identity_label,
description, tools_synced_at, oauth_client, oauth_client_owner,
refresh_item_id, expires_at, oauth_scope, oauth_token_url,
provider_state, created_at, updated_at, row_id, tenant, owner, subject)
SELECT integration, name, template, provider,
CASE
WHEN item_ids IS NOT NULL AND item_ids <> '{}' THEN item_ids
ELSE json_object('token', item_id)
END,
identity_label, description, tools_synced_at, oauth_client,
oauth_client_owner, refresh_item_id, expires_at, oauth_scope,
oauth_token_url, provider_state, created_at, updated_at, row_id,
tenant, owner, subject
FROM connection`,
`ALTER TABLE connection RENAME TO connection_legacy_item_id`,
`ALTER TABLE connection_next RENAME TO connection`,
`DROP TABLE connection_legacy_item_id`,
];
for (const statement of statements) {
await db.prepare(statement).run();
}
};

export const ensureCloudflareD1SchemaCompatibility = async (db: D1Database): Promise<void> => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Non-atomic table rebuild leaves the DB in a broken state on mid-migration failure

rebuildLegacyConnectionTable executes seven DDL statements with sequential await db.prepare(stmt).run() calls. D1 auto-commits each statement individually, so there is no rollback if a step fails. The critical window is between step 5 (ALTER TABLE connection RENAME TO connection_legacy_item_id) and step 6 (ALTER TABLE connection_next RENAME TO connection): if step 6 fails, the connection table no longer exists, breaking every subsequent startup. Worse, ensureCloudflareD1SchemaCompatibility's early-return guard (if (connectionColumns.size === 0) return) will silently skip the fix on the next boot because PRAGMA table_info('connection') returns zero rows for a non-existent table.

D1 db.batch() provides all-or-nothing semantics for multiple statements. Using it here would make the entire rebuild atomic:

await db.batch(statements.map((sql) => db.prepare(sql)));

@RhysSullivan RhysSullivan force-pushed the codex/toolkits-scoped-mcp branch 3 times, most recently from bed2d28 to be7fa25 Compare June 26, 2026 20:28
@RhysSullivan RhysSullivan force-pushed the codex/toolkits-scoped-mcp branch from be7fa25 to 2459f6c Compare June 26, 2026 21:14
@RhysSullivan RhysSullivan merged commit ca8a962 into main Jun 26, 2026
15 checks passed
@RhysSullivan RhysSullivan deleted the codex/toolkits-scoped-mcp branch June 26, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant