perf(semantic-search): snapshot manifest partitions

[aryasaatvik]

Summary

Hardens the per-run manifest snapshot layer so scans are pinned to the manifest captured at index creation time. This protects the KV-only scan path from accidentally falling back to a later live catalog read.

Changes

• Add regression coverage for create-time manifest snapshot stability.
• Prove scan does not reread a changed live manifest after create.
• Assert the scanned job keeps the original snapshot fingerprint and source revision.

Call Stack

ToolSearchIndex.create()
  -> listToolManifests() once
  -> partition manifests
  -> write index-manifest/v1/{runId}/{partition} to executor.cache

ToolSearchIndex.scan()
  -> read partition snapshot from executor.cache
  -> never call executor.tools.manifest()
  -> materialize jobs from the snapshot

Tests

• bun run --cwd packages/plugins/semantic-search test -- src/sdk/tool-search-index.test.ts
• bun run --cwd packages/plugins/semantic-search typecheck
• bunx oxfmt --check packages/plugins/semantic-search/src/sdk/tool-search-index.test.ts
• bunx oxlint -c .oxlintrc.jsonc --deny-warnings packages/plugins/semantic-search/src/sdk/tool-search-index.test.ts

Stack

Base: #64

Stack

1. feat(sdk): harden tool manifest projection for indexing #63
2. perf(semantic-search): index from manifest projection #64
3. perf(semantic-search): snapshot manifest partitions #65 👈 current

[greptile-apps]

Greptile Summary

This PR adds a single regression test to tool-search-index.test.ts that proves scan() reads the manifest snapshot captured at create() time rather than re-querying the live manifest after it changes.

• Injects a mutable liveManifests closure, calls create(), then mutates the closure to simulate a live catalog update, and asserts that the subsequent scan() job still carries the original "fp-at-create" fingerprint and "source-at-create" source revision.
• Guards the destructured tool variable with both an explicit expect(tool).toBeDefined() and a TypeScript narrowing guard, ensuring the test fails loudly if makeTools(1) ever returns an empty array.

Confidence Score: 5/5

Test-only change that adds targeted regression coverage with no production code modifications; safe to merge.

The change is confined to a single test file and adds one well-structured Effect test. It correctly mutates the live manifest after create() and asserts that scan() reads from the KV snapshot, verifying the isolation guarantee end-to-end. The expect(tool).toBeDefined() guard ensures the test fails explicitly if the fixture changes, and all assertions target the right observable state.

No files require special attention.

Important Files Changed

Filename	Overview
packages/plugins/semantic-search/src/sdk/tool-search-index.test.ts	Adds one new Effect-based test that verifies the manifest snapshot isolation guarantee; test structure, assertions, and helper reuse all look correct.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Test
    participant ToolSearchIndex
    participant executor.tools.manifest
    participant executor.cache (KV)

    Test->>ToolSearchIndex: "create(runId, partitionCount=1)"
    ToolSearchIndex->>executor.tools.manifest: "manifest() [call #1]"
    executor.tools.manifest-->>ToolSearchIndex: "[{ fp-at-create, source-at-create }]"
    ToolSearchIndex->>executor.cache (KV): write index-manifest/v1/runId/0

    Note over Test: mutate liveManifests to fp-after-create

    Test->>ToolSearchIndex: "scan(runId, partition=0, limit=10)"
    ToolSearchIndex->>executor.cache (KV): read index-manifest/v1/runId/0
    executor.cache (KV)-->>ToolSearchIndex: [{ fp-at-create, source-at-create }]
    Note over ToolSearchIndex: manifest() never called again
    ToolSearchIndex-->>Test: "{ processed:1, changed:1, skipped:0 }"
    Note over Test: asserts job.fingerprint == fp-at-create
    Note over Test: asserts counters.manifest == 1

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Test
    participant ToolSearchIndex
    participant executor.tools.manifest
    participant executor.cache (KV)

    Test->>ToolSearchIndex: "create(runId, partitionCount=1)"
    ToolSearchIndex->>executor.tools.manifest: "manifest() [call #1]"
    executor.tools.manifest-->>ToolSearchIndex: "[{ fp-at-create, source-at-create }]"
    ToolSearchIndex->>executor.cache (KV): write index-manifest/v1/runId/0

    Note over Test: mutate liveManifests to fp-after-create

    Test->>ToolSearchIndex: "scan(runId, partition=0, limit=10)"
    ToolSearchIndex->>executor.cache (KV): read index-manifest/v1/runId/0
    executor.cache (KV)-->>ToolSearchIndex: [{ fp-at-create, source-at-create }]
    Note over ToolSearchIndex: manifest() never called again
    ToolSearchIndex-->>Test: "{ processed:1, changed:1, skipped:0 }"
    Note over Test: asserts job.fingerprint == fp-at-create
    Note over Test: asserts counters.manifest == 1

Loading

_{Reviews (4): Last reviewed commit: "test(semantic-search): assert snapshot t..." | Re-trigger Greptile}