spike(durable): prove Foreman durables run offline on the filesystem adapter

.beads/export-state.json

@@ -1 +1 @@
-{"last_dolt_commit":"c7bb5qcv1ctkhf392dl3d1f974oogh1u","timestamp":"2026-07-28T00:44:28.7090854-07:00","issues":454,"memories":0}
+{"last_dolt_commit":"fb1t0tec2l8cs1b4jr63mqbir0gi8ttb","timestamp":"2026-07-28T02:16:02.5366637-07:00","issues":456,"memories":0}

docs/zapier-durable-questions-for-engineers.md

@@ -1,5 +1,56 @@
 # Durable Execution — Questions for Zapier Engineers
 
+> ## 🧪 UPDATE 2026-07-28 (`@zapier/zapier-durable@0.11.0`) — filesystem adapter runs our durables offline; five new questions
+>
+> Following the direct steer from your team (2026-07-27) that `@zapier/zapier-durable`
+> ships three adapters and that the **filesystem** adapter is the recommended path for
+> running durables outside Zapier, we ran it. **It works.**
+>
+> Repro (no credentials, no network, no early-access allowlist):
+> `packages/agents/scripts/durable-filesystem-spike.ts` — a Foreman-shaped durable
+> (`ctx.step` → `ctx.createCallback` approval gate → suspend → deliver → resume → done)
+> completes in-process against `FilesystemClient`.
+>
+> **What this changes about question 2 below.** Q2 asks you to expose the callback URL
+> because an external orchestrator can't obtain it. In-process, `ctx.createCallback`
+> **does** return `[promise, callbackUrl]` directly — so the limitation is in the *wire
+> protocol* (`getDurableRun`), not in the SDK's design. Our `__report_callback_url_*`
+> self-report step exists purely to work around the wire gap, and it is unnecessary on
+> the filesystem adapter.
+>
+> **New, evidence-backed questions:**
+>
+> 1. **Is the filesystem adapter supported for production self-hosting, or is it a
+>    dev/test convenience?** Your team described custom adapters as "supported but
+>    undocumented" and the filesystem adapter as "not officially supported yet." We are
+>    considering running end-user automations on it, so we need to know which it is
+>    before we depend on it.
+> 2. **Will `getDurableRun` expose the callback URL (or gain a resume-by-token
+>    endpoint)?** Restating Q2 with the new evidence above: the SDK already has the URL
+>    locally; only the remote read path withholds it.
+> 3. **Is `client.callback(token, payload)` the intended programmatic delivery path for
+>    non-Zapier adapters?** On the filesystem adapter, `callbackBaseUrl` is a `file://`
+>    URL — we measured `file://<stateDir>/callbacks/<token>` — so `callbackUrl` is **not**
+>    POST-able and the only delivery route we found is the adapter client (or the
+>    `durable-callback` CLI). If so, is `callbackUrl` better understood as an opaque
+>    **token carrier** whose last path segment is the real identifier, rather than a URL?
+>    A host that treats it as a URL works on the Zapier adapter and silently breaks on
+>    filesystem.
+> 4. **`CallbackRequest` is typed `unknown`, and the payload is the raw body.** Passing
+>    `{ payload: {...} }` fails edge validation with
+>    `validation_failed` / `additionalProperties`. The README documents only CLI
+>    delivery, so there is no worked example of the programmatic shape. Worth one line
+>    in the README.
+> 5. **Why is `zod` pinned to exactly `4.2.1` rather than a range?** An exact pin in
+>    `dependencies` forces every consumer that already pins zod (we are on `4.4.3`
+>    repo-wide) into an override. `4.4.3` works fine in our end-to-end run, so a
+>    `^4.2.1` range would remove the friction.
+>
+> **Housekeeping:** `@zapier/zapier-durable` publishes no `repository`, `bugs`, or
+> `homepage` field, and `gitlab.com/zapier/zapier-sdk` is behind Zapier SAML SSO — so
+> there is no public tracker we can file any of this against. **Where would you like
+> bug reports for these packages to go?**
+
 > ## ✅ UPDATE 2026-07-05 (SDK 0.81.0) — durables now WORK for accounts on Zapier's early-access allowlist (apply for access; not GA); questions narrowed
 >
 > The two original blockers below are **RESOLVED** on `@zapier/zapier-sdk@0.81.0`:

packages/agents/package.json

@@ -65,6 +65,7 @@
     "@mastra/voice-openai": "0.13.0",
     "@opentelemetry/api": "^1.9.1",
     "@supabase/supabase-js": "^2.105.4",
+    "@zapier/zapier-durable": "^0.11.0",
     "@zapier/zapier-sdk": "^0.91.0",
     "ai": "^7.0.37",
     "async-mutex": "^0.5.0",

packages/agents/scripts/README.md

@@ -87,6 +87,22 @@ Run with `npx tsx --env-file=.env.local scripts/<name>.ts`.
 > These probes hit a real Zapier account and may create/delete cloud resources.
 > They self-clean, but run them against a non-sensitive test account.
 
+### Offline durable spike
+
+`durable-filesystem-spike.ts` is the exception — it needs **no credentials, no
+network, and no Zapier early-access allowlist**. It runs a Foreman-shaped durable
+(step → human-approval gate → resume) entirely in-process on the
+`@zapier/zapier-durable` filesystem adapter, into a temp dir it cleans up.
+
+```bash
+npx tsx scripts/durable-filesystem-spike.ts
+```
+
+It prints the measured `callbackUrl`. Note that on this adapter it is a `file://`
+URL and therefore **not** HTTP-POSTable — delivery goes through
+`client.callback(token, payload)`, where `token` is the last path segment. See
+`foreman-02lu` for the full finding.
+
 ## Root scripts (`/scripts`)
 
 | Command | Purpose |

packages/agents/scripts/durable-filesystem-spike.ts

@@ -0,0 +1,131 @@
+/**
+ * Spike (foreman-02lu): run a Foreman-shaped durable with a human-approval gate
+ * on the FILESYSTEM adapter — no Zapier early-access allowlist, no network.
+ *
+ * What this answers:
+ *   1. Does `@zapier/zapier-durable` run in-process on the filesystem adapter?
+ *   2. Does it work with the repo-wide `zod` override (4.4.3) even though the
+ *      package pins zod 4.2.1 exactly?
+ *   3. What does `ctx.createCallback` hand back as `callbackUrl` locally — is it
+ *      something Foreman's /automations Approve/Deny could POST to, or does
+ *      delivery have to go through the adapter client?
+ *
+ * Run:  npx tsx scripts/durable-filesystem-spike.ts
+ * No credentials required.
+ *
+ * STABILITY: the package README states it is pre-1.0 and that MINOR versions are
+ * breaking until 1.0, and it ships one roughly every 1-2 weeks (0.5.2 -> 0.11.0
+ * between 2026-06-02 and 2026-07-27). The `^0.11.0` range is deliberate: npm
+ * caret on a 0.x version allows patches only (`^0.11.0` rejects 0.12.0), so it
+ * pins us below the next breaking minor. Re-run this spike on every minor bump.
+ */
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { configureDurable, createClient, defineDurable, getConfig } from "@zapier/zapier-durable";
+import { z } from "zod";
+
+const stateDir = mkdtempSync(join(tmpdir(), "foreman-durable-spike-"));
+
+function log(label: string, value?: unknown) {
+  if (value === undefined) console.log(`\n${label}`);
+  else console.log(`  ${label}`, typeof value === "string" ? value : JSON.stringify(value));
+}
+
+async function main() {
+  // 1. Select the adapter explicitly rather than relying on the documented
+  //    default, so this proves the env-selected path Foreman would ship.
+  process.env.ZAPIER_DURABLE_ADAPTER = "filesystem";
+  process.env.ZAPIER_DURABLE_FS_DIR = stateDir;
+  configureDurable({ adapter: "filesystem", filesystem: { baseDir: stateDir } });
+
+  log("[1] config");
+  const cfg = getConfig();
+  log("adapter:", cfg.adapter);
+  log("stateDir:", stateDir);
+
+  const client = createClient();
+  log("client:", client.constructor.name);
+  log("callbackBaseUrl:", client.callbackBaseUrl);
+
+  // 2. A durable shaped like Foreman's: one step, then a human-approval gate.
+  //    payloadSchema exercises zod — the version the override forced on it.
+  let observedCallbackUrl: string | undefined;
+
+  const approvalDurable = defineDurable({
+    name: "foreman-spike-approval",
+    description: "One step, then wait for a human decision.",
+    inputSchema: z.object({ subject: z.string() }),
+    run: async (ctx, input) => {
+      const prepared = await ctx.step("prepare", async () => ({
+        subject: input.subject,
+        preparedAt: "fixed-for-determinism",
+      }));
+
+      const [approval, callbackUrl] = await ctx.createCallback({
+        name: "human-approval",
+        payloadSchema: z.object({ approved: z.boolean(), note: z.string().optional() }),
+      });
+
+      // THE POINT OF THE SPIKE: in-process the URL is right here, no
+      // `__report_callback_url_*` step needed to smuggle it out.
+      observedCallbackUrl = callbackUrl;
+
+      const decision = await approval;
+      return { subject: prepared.subject, approved: decision.approved, note: decision.note };
+    },
+  });
+
+  // 3. First tick — should suspend at the gate.
+  log("[2] first tick (expect suspend at the approval gate)");
+  const first = await approvalDurable({ subject: "ship it" });
+  log("done:", first.done);
+  log("executionId:", first.executionId ?? "(none)");
+  log("callbackUrl:", observedCallbackUrl ?? "(never observed)");
+
+  if (first.done)
+    throw new Error("expected the durable to suspend at the callback, but it finished");
+  if (!observedCallbackUrl) throw new Error("ctx.createCallback did not surface a callbackUrl");
+  if (!first.executionId) throw new Error("no executionId returned; cannot resume");
+
+  // 4. Deliver the decision. The token is the last URL segment; the adapter
+  //    client is the delivery path (there is no HTTP server locally).
+  const token = observedCallbackUrl.split("/").pop() as string;
+  log("[3] delivering approval via the adapter client");
+  log("token:", token);
+  // `CallbackRequest` is `unknown` — the payload IS the body, not `{ payload }`.
+  // Wrapping it fails edge validation against the gate's payloadSchema.
+  const delivered = await client.callback(token, { approved: true, note: "spike" });
+  log("callback response:", delivered);
+  if ("error" in delivered) throw new Error(`callback delivery rejected: ${delivered.error}`);
+
+  // 5. Resume and confirm the journal replayed to completion.
+  log("[4] resume");
+  const second = await approvalDurable(first.executionId);
+  log("done:", second.done);
+  log("result:", second.result);
+  log("error:", second.error?.message ?? "(none)");
+
+  if (!second.done) throw new Error("durable did not complete after callback delivery");
+  const result = second.result as { approved?: boolean; subject?: string; note?: string };
+  if (result?.approved !== true) throw new Error(`expected approved=true, got ${result?.approved}`);
+
+  log("[5] VERDICT");
+  log("filesystem adapter ran end-to-end:", true);
+  log("zod override (4.4.3) accepted by payloadSchema:", true);
+  log("callbackUrl available in-process:", observedCallbackUrl);
+  log("callbackUrl is HTTP-POSTable:", /^https?:\/\//.test(observedCallbackUrl));
+}
+
+main()
+  .then(() => {
+    console.log("\nPASS");
+    process.exitCode = 0;
+  })
+  .catch((err) => {
+    console.error("\nFAIL:", err);
+    process.exitCode = 1;
+  })
+  .finally(() => {
+    rmSync(stateDir, { recursive: true, force: true });
+  });