feat(sdk): harden tool manifest projection for indexing

[aryasaatvik]

Summary

Hardens the tool manifest projection used by indexing so manifest reads observe the same stale-catalog freshness behavior as tools.list, and lets plugins persist explicit source revisions for produced tool catalogs.

This keeps the existing separate tool_schema_manifest table and column shape. The change is a greenfield semantic cleanup of the manifest contract, not a table rewrite.

Changes

• Run stale connection catalog sync before executor.tools.manifest() reads manifest rows.
• Add optional ResolveToolsResult.sourceRevision so plugins can describe the source revision that produced a catalog.
• Persist plugin-provided source revisions into manifest rows, falling back to the existing generic integration/connection revision when absent.
• Set OpenAPI source revisions from specHash.
• Set MCP source revisions from a hash of the discovered MCP tool manifest.
• Add SDK regressions for stale manifest self-healing and plugin-provided source revision persistence.

Call Stack

executor.tools.manifest(filter)
  -> syncStaleConnectionTools()
     -> compare integration.config_revised_at with connection.tools_synced_at
     -> produceConnectionTools() for stale connections
  -> read tool_schema_manifest rows
  -> apply filters and policy visibility
  -> return compact indexing projection

plugin.resolveTools()
  -> returns tools, definitions, optional sourceRevision
produceConnectionTools()
  -> sourceRevision ?? generic integration/connection revision
  -> write tool rows, definition rows, manifest rows, catalog revision row

Tests

• bun run --cwd packages/core/sdk test -- src/executor.test.ts
• bun run --cwd packages/core/sdk typecheck
• bun run --cwd packages/plugins/openapi test -- src/sdk/spec-blob.test.ts src/sdk/plugin.test.ts
• bun run --cwd packages/plugins/openapi typecheck
• bun run --cwd packages/plugins/mcp test -- src/sdk/plugin.test.ts
• bun run --cwd packages/plugins/mcp typecheck
• bun run --cwd packages/plugins/semantic-search test -- src/sdk/tool-search-index.test.ts
• bun run --cwd packages/plugins/semantic-search typecheck
• bunx oxfmt --check packages/core/sdk/src/plugin.ts packages/core/sdk/src/executor.ts packages/core/sdk/src/executor.test.ts packages/plugins/openapi/src/sdk/backing.ts packages/plugins/mcp/src/sdk/plugin.ts
• bunx oxlint -c .oxlintrc.jsonc --deny-warnings packages/core/sdk/src/plugin.ts packages/core/sdk/src/executor.ts packages/core/sdk/src/executor.test.ts packages/plugins/openapi/src/sdk/backing.ts packages/plugins/mcp/src/sdk/plugin.ts
• git diff --check

Notes

No deploy was performed. This is one milestone PR rather than a stack because the existing semantic-search indexing code already consumes tools.manifest() and passed its targeted tests against this change.

Stack

1. feat(sdk): harden tool manifest projection for indexing #63 👈 current
2. perf(semantic-search): index from manifest projection #64
3. perf(semantic-search): snapshot manifest partitions #65

[greptile-apps]

Greptile Summary

This PR hardens the tools.manifest() contract by mirroring the stale-catalog freshness sync already applied in tools.list(), and extends ResolveToolsResult with an optional sourceRevision field that plugins can populate with content-specific hashes (spec hash for OpenAPI, manifest object hash for MCP).

• toolsManifest now calls syncStaleConnectionTools before reading tool_schema_manifest rows, ensuring indexing sees up-to-date catalogs after integration config changes without requiring a manual refresh.
• ResolveToolsResult.sourceRevision is new and optional; the executor prefers it over the generic integration/connection revision when present, and both OpenAPI (specHash) and MCP (sha256Hex(manifest)) implementations populate it.
• Two regression tests cover the stale self-healing path and the plugin-provided revision persistence path.

Confidence Score: 4/5

Safe to merge; the changes are additive and best-effort, and the existing stale-sync logic was already proven in the tools.list path.

The logic is a straightforward mirror of an already-working pattern applied to a second call site. The sourceRevision field is optional with a safe fallback, so no existing integrations break. The only rough edge is the pre-existing as cast in the MCP plugin that now hides sourceRevision from the TypeScript type — harmless at runtime but worth cleaning up.

packages/plugins/mcp/src/sdk/plugin.ts — the stale type assertion on line 973 should be removed or updated to reflect the new conditional sourceRevision field.

Important Files Changed

Filename	Overview
packages/core/sdk/src/executor.ts	Adds syncStaleConnectionTools to toolsManifest so manifest reads observe the same freshness behavior as tools.list; uses plugin-provided sourceRevision with fallback to the existing generic revision calculation.
packages/core/sdk/src/plugin.ts	Extends ResolveToolsResult with an optional sourceRevision?: string field and clear JSDoc explaining fallback behavior; clean, non-breaking interface addition.
packages/plugins/mcp/src/sdk/plugin.ts	Imports sha256Hex and hashes the full MCP manifest object to produce sourceRevision; the pre-existing as type assertion is now stale and hides sourceRevision from the TypeScript type.
packages/plugins/openapi/src/sdk/backing.ts	Propagates openApiConfig.specHash as sourceRevision on both the cached-defs fast path and the compile-from-spec fallback path; correctly undefined when the config has no hash.
packages/core/sdk/src/executor.test.ts	Adds two regression tests: stale manifest self-healing (integration revision bump triggers re-sync) and plugin-provided source revision persistence; both are well-structured and isolated.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant tools.manifest
    participant syncStaleConnectionTools
    participant DB
    participant plugin.resolveTools

    Caller->>tools.manifest: manifest(filter)
    tools.manifest->>syncStaleConnectionTools: "yield*"
    syncStaleConnectionTools->>DB: findMany(integration, config_revised_at IS NOT NULL)
    DB-->>syncStaleConnectionTools: revised integrations
    syncStaleConnectionTools->>DB: findMany(connection, integration IN revised)
    DB-->>syncStaleConnectionTools: connections
    loop for each stale connection
        syncStaleConnectionTools->>plugin.resolveTools: resolveTools(...)
        plugin.resolveTools-->>syncStaleConnectionTools: "{ tools, definitions, sourceRevision? }"
        syncStaleConnectionTools->>DB: upsert tool_schema_manifest rows
    end
    tools.manifest->>DB: findMany(tool_schema_manifest, filter)
    DB-->>tools.manifest: fresh manifest rows
    tools.manifest-->>Caller: ToolSchemaManifest[]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant tools.manifest
    participant syncStaleConnectionTools
    participant DB
    participant plugin.resolveTools

    Caller->>tools.manifest: manifest(filter)
    tools.manifest->>syncStaleConnectionTools: "yield*"
    syncStaleConnectionTools->>DB: findMany(integration, config_revised_at IS NOT NULL)
    DB-->>syncStaleConnectionTools: revised integrations
    syncStaleConnectionTools->>DB: findMany(connection, integration IN revised)
    DB-->>syncStaleConnectionTools: connections
    loop for each stale connection
        syncStaleConnectionTools->>plugin.resolveTools: resolveTools(...)
        plugin.resolveTools-->>syncStaleConnectionTools: "{ tools, definitions, sourceRevision? }"
        syncStaleConnectionTools->>DB: upsert tool_schema_manifest rows
    end
    tools.manifest->>DB: findMany(tool_schema_manifest, filter)
    DB-->>tools.manifest: fresh manifest rows
    tools.manifest-->>Caller: ToolSchemaManifest[]

Loading

Comments Outside Diff (1)

1. packages/plugins/mcp/src/sdk/plugin.ts, line 973 (link)

   Stale type assertion hides sourceRevision

   The as Effect.Effect<{ readonly tools: readonly ToolDef[] }, StorageFailure> cast was valid before this PR, but now that the generator can include sourceRevision, the assertion silently drops the field from the TypeScript type. At runtime the object is intact and sourceRevision flows correctly to the executor, but any code inside this file that tries to access .sourceRevision on a locally-typed reference to resolveTools's return would see a TS error. The cast should be removed (the inferred type is already compatible with the plugin interface's Effect.Effect<ResolveToolsResult, StorageFailure>) or updated to include sourceRevision?.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

   

_{Reviews (1): Last reviewed commit: "fix(sdk): harden tool manifest revisions" | Re-trigger Greptile}