feat(sdk): add cache primitive

[aryasaatvik]

Summary

• Add a generic SDK cache primitive backed by Effect KeyValueStore.
• Let hosts provide durable cache storage through ExecutorConfig.cache, while default executors get a bounded in-memory fallback.
• Add a reusable Cloudflare KV adapter export without wiring host app config that upstream main does not currently declare.

Changes

• packages/core/sdk/src/executor.ts
  • Imports effect/unstable/persistence/KeyValueStore.
  • Adds ExecutorConfig.cache?: KeyValueStore.KeyValueStore.
  • Adds Executor.cache: KeyValueStore.KeyValueStore.
  • Creates a default in-memory string-only KeyValueStore with TTL and capacity eviction.
  • Installs the resolved cache store on the returned executor object.
• packages/core/sdk/src/test-config.ts
  • Allows tests to pass a cache store through makeTestConfig and makeTestWorkspaceHarness.
• packages/core/api/src/server/scoped-executor.ts
  • Reads an optional KeyValueStore.KeyValueStore service from the Effect context.
  • Passes it to createExecutor only when a host layer provides one.
• packages/hosts/cloudflare/src/key-value-store.ts
  • Adds makeCloudflareKeyValueStore(kv) over Cloudflare KVNamespace.
  • Adds layerCloudflareKeyValueStore(kv) for host boot layer composition.
  • Maps KV failures into KeyValueStore.KeyValueStoreError.
  • Implements paginated size and bounded parallel clear.
• packages/hosts/cloudflare/package.json
  • Exports @executor-js/cloudflare/key-value-store.
• .changeset/polite-caches-pull.md
  • Records the public SDK surface change through the fixed release group.

AST-level outline

// SDK public config
interface ExecutorConfig<TPlugins> {
  tenant: Tenant
  subject?: Subject
  db?: ExecutorDbInput | ExecutorDbFactory
  blobs?: BlobStore
  cache?: KeyValueStore.KeyValueStore
  plugins?: TPlugins
}

// SDK public executor value
type Executor<TPlugins> = {
  integrations: ...
  connections: ...
  oauth: OAuthService
  tools: ...
  providers: ...
  policies: ...
  execute: ...
  close: ...
  cache: KeyValueStore.KeyValueStore
} & PluginExtensions<TPlugins>

// createExecutor resolution
const blobs = config.blobs ?? makeFumaBlobStore(fuma)
const cacheStore = config.cache ?? makeMemoryCacheStore()

return Object.assign({ ..., cache: cacheStore }, extensions)

// API host seam
const cache = yield* Effect.serviceOption(KeyValueStore.KeyValueStore)

yield* createExecutor({
  tenant,
  subject,
  db,
  blobs,
  ...Option.match(cache, {
    onNone: () => ({}),
    onSome: (store) => ({ cache: store }),
  }),
  plugins,
})

// Cloudflare host package adapter
makeCloudflareKeyValueStore(kv)
  -> KeyValueStore.makeStringOnly({ get, set, remove, clear, size })

layerCloudflareKeyValueStore(kv)
  -> Layer.succeed(KeyValueStore.KeyValueStore, makeCloudflareKeyValueStore(kv))

Call-stack trace

Host-provided cache path

host boot layer
  -> Layer.succeed(KeyValueStore.KeyValueStore, hostStore)
  -> makeScopedExecutor(accountId, organizationId, organizationName)
     -> Effect.serviceOption(KeyValueStore.KeyValueStore)
     -> createExecutor({ cache: hostStore, ... })
        -> const cacheStore = config.cache
        -> executor.cache = cacheStore

Default cache path

createExecutor({ cache: undefined, ... })
  -> makeMemoryCacheStore()
     -> KeyValueStore.makeStringOnly({ get, set, remove, clear, size })
  -> executor.cache = inMemoryStore

Cloudflare adapter path

makeCloudflareKeyValueStore(env.CACHE)
  -> store.get(key) calls kv.get(key)
  -> store.set(key, value) calls kv.put(key, value)
  -> store.remove(key) calls kv.delete(key)
  -> store.size lists all pages and counts names
  -> store.clear lists all pages and deletes keys in batches of 50

Usage / pseudocode

import { Effect } from "effect";
import { layerCloudflareKeyValueStore } from "@executor-js/cloudflare/key-value-store";
import { KeyValueStore } from "effect/unstable/persistence/KeyValueStore";

const cacheLayer = layerCloudflareKeyValueStore(env.CACHE);

const program = Effect.gen(function* () {
  const store = yield* KeyValueStore;
  yield* store.set("catalog:revision", "v1");

  const executor = yield* createExecutor({
    tenant,
    subject,
    db,
    cache: store,
    plugins,
    onElicitation: "accept-all",
  });

  yield* executor.cache.set("plugin:derived-view", "...");
});

yield* program.pipe(Effect.provide(cacheLayer));

Tests

• bunx oxfmt --check packages/core/sdk/src/executor.ts packages/core/sdk/src/test-config.ts packages/core/api/src/server/scoped-executor.ts packages/hosts/cloudflare/src/key-value-store.ts packages/hosts/cloudflare/src/key-value-store.test.ts packages/hosts/cloudflare/package.json .changeset/polite-caches-pull.md
• git diff --check
• bunx oxlint -c .oxlintrc.jsonc --deny-warnings packages/core/sdk/src/executor.ts packages/core/sdk/src/test-config.ts packages/core/api/src/server/scoped-executor.ts packages/hosts/cloudflare/src/key-value-store.ts packages/hosts/cloudflare/src/key-value-store.test.ts
• bun run --cwd packages/core/sdk typecheck
• bun run --cwd packages/core/api typecheck
• bun run --cwd packages/hosts/cloudflare test -- src/key-value-store.test.ts
• bun run --cwd packages/hosts/cloudflare typecheck
• bun run typecheck

Validation notes

• Root bun run format:check and root bun run lint currently scan .scratchpad and fail on existing scratchpad files outside this PR. Touched-file format and lint checks pass.
• Root bun run test runs the suite but currently prints two unrelated apps/cloud auth expectation failures where the received identity includes organizationSlug. The cache primitive diff does not touch apps/cloud.

Deferred scope

• No tool-schema cache consumers.
• No tools.list cache consumer.
• No tool manifest changes.
• No semantic-search-specific cache work.
• No Cloudflare app CACHE binding wiring because upstream main does not currently declare a CACHE binding in CloudflareEnv or wrangler.jsonc.

[greptile-apps]

Greptile Summary

This PR adds a generic cache primitive to the SDK executor surface, backed by Effect's KeyValueStore. Hosts can supply a durable store (e.g. Cloudflare KV) via ExecutorConfig.cache; executors that omit it receive a bounded in-memory fallback with TTL expiration and LRU eviction.

• SDK changes: ExecutorConfig.cache?: KeyValueStore.KeyValueStore is added alongside a makeMemoryCacheStore() fallback (2,048-entry capacity, 10-minute TTL) that uses delete+set for correct LRU ordering; the resolved store is exposed as executor.cache.
• API seam: makeScopedExecutor reads the store from the Effect context via Effect.serviceOption and forwards it only when a host provides it, leaving the default fallback path intact.
• Cloudflare adapter: makeCloudflareKeyValueStore(kv) and layerCloudflareKeyValueStore(kv) are added under @executor-js/cloudflare/key-value-store, wrapping KVNamespace with paginated size/clear and bounded-parallel deletes (50-at-a-time).

Confidence Score: 5/5

Safe to merge. Both previously flagged concerns — LRU ordering on set of an existing key and the missing in-memory cache test suite — are fully addressed in this revision.

The LRU fix (delete + set in the set handler) is in place and directly exercised by the new executor-cache.test.ts test that fills to capacity, overwrites a key, triggers overflow, and asserts the correct survivor. TTL expiry is covered with a patched Date.now. The Cloudflare adapter's paginated size and bounded-parallel clear are covered by the fake-KV test. The Effect-context seam in scoped-executor correctly uses serviceOption so a missing host layer falls back to the default store without any error. No new logic gaps were found across all changed files.

No files require special attention.

Important Files Changed

Filename	Overview
packages/core/sdk/src/executor.ts	Adds cache?: KeyValueStore.KeyValueStore to ExecutorConfig, exposes cache on Executor, and introduces makeMemoryCacheStore() with TTL, LRU eviction (correctly implemented via delete+set), and capacity management. The previous LRU ordering bug is fixed.
packages/hosts/cloudflare/src/key-value-store.ts	Adds makeCloudflareKeyValueStore and layerCloudflareKeyValueStore. Paginated listKeys correctly uses list_complete for loop termination; deleteKeys batches in groups of 50. Error wrapping via KeyValueStoreError is consistent across all methods.
packages/core/api/src/server/scoped-executor.ts	Reads KV store from Effect context with serviceOption; spreads it into createExecutor only when present. The optional spread pattern is clean and correctly leaves the default fallback path unchanged when no host provides a store.
packages/core/sdk/src/executor-cache.test.ts	New test file covering the in-memory fallback: basic get/set/remove, TTL expiration via Date.now patching, and LRU position refresh on set of an existing key. Addresses previously noted gap in unit-test coverage.
packages/hosts/cloudflare/src/key-value-store.test.ts	Tests round-trip get/set/remove, paginated key counting via size, and bounded-parallel deletion via clear. The fake KV correctly tracks concurrent delete count to validate the 50-at-a-time constraint.
packages/core/sdk/src/test-config.ts	Adds cache?: KeyValueStore.KeyValueStore to TestConfigOptions and threads it through makeTestConfig; makeTestWorkspaceHarness inherits it transitively via makeTestConfig. Minimal, correct change.
packages/hosts/cloudflare/package.json	Adds ./key-value-store export pointing at the new adapter source. Consistent with the existing ./blob-store export pattern.
.changeset/polite-caches-pull.md	Declares a patch bump for the executor package via the fixed release group. Description accurately reflects the public API addition.

_{Reviews (2): Last reviewed commit: "fix(sdk): refresh cache writes in LRU or..." | Re-trigger Greptile}

[aryasaatvik]

Additional downstream context from my fork: the cache primitive is the base layer I use to move hot-path derived data out of D1 and into host-provided KV.

High-level shape:

host provides KeyValueStore
  -> createExecutor({ cache })
  -> executor.cache
  -> typed schema stores
  -> schema preview cache / schema view cache / manifest snapshot cache

AST outline of downstream usage:

makeCloudflareApp(env)
  -> layerCloudflareKeyValueStore(env.CACHE)

createExecutor(...)
  -> cacheStore = config.cache ?? makeMemoryCacheStore()
  -> KeyValueStore.toSchemaStore(cacheStore, ToolTypeScriptPreviewCacheEntry)
  -> KeyValueStore.toSchemaStore(cacheStore, ToolSchemaViewCacheEntry)
  -> KeyValueStore.toSchemaStore(cacheStore, ToolListCacheEntry)

semanticSearch
  -> KeyValueStore.toSchemaStore(executor.cache, ManifestSnapshotEntry)

Fork permalinks:

• Cloudflare host wiring: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/apps/host-cloudflare/src/app.ts#L61-L68
• Executor typed cache stores: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/core/sdk/src/executor.ts#L1955-L1962
• Schema view cache keyed by tool manifest fingerprint: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/core/sdk/src/executor.ts#L3586-L3609
• Semantic-search manifest snapshot cache: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/semantic-search/src/sdk/tool-search-index.ts#L918-L1005
• Snapshot schema and key shape: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/semantic-search/src/sdk/manifest-snapshot.ts#L22-L44

Call stack:

tools schema view
  -> compute manifest fingerprint
  -> toolSchemaViewStore.get(cacheKey)
  -> cache hit returns precompiled view
  -> cache miss reads tool/definition rows and writes cache

semantic-search reindex
  -> tools.manifest() once
  -> write partitioned manifest snapshots to executor.cache
  -> queue scan messages read KV snapshots
  -> scan path avoids repeated D1 manifest reads

This PR is intentionally just the primitive. Once it lands, schema-view caching, TypeScript preview caching, and manifest snapshot work can be proposed as follow-up consumers without forcing this PR to carry those policies.