From 612bd075c1936837574ac88bac07f4700ff97561 Mon Sep 17 00:00:00 2001 From: "mintlify[bot]" <109931778+mintlify[bot]@users.noreply.github.com> Date: Tue, 7 Jul 2026 01:11:58 +0000 Subject: [PATCH 1/2] docs: document QiForge oracle latency tuning knobs --- .../reference/bundled-plugins/memory.mdx | 6 ++++-- build-an-oracle/reference/environment-variables.mdx | 13 +++++++++++++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/build-an-oracle/reference/bundled-plugins/memory.mdx b/build-an-oracle/reference/bundled-plugins/memory.mdx index 4427975..73f8954 100644 --- a/build-an-oracle/reference/bundled-plugins/memory.mdx +++ b/build-an-oracle/reference/bundled-plugins/memory.mdx @@ -51,14 +51,16 @@ Memory context does not arrive in the prompt because the agent calls a tool to f 5. `relationships` — people and organizations mentioned in past conversations 6. `recent` — notable things from the last few sessions -**When it runs:** at agent-compile time, before turn 1. The fetcher runs once per session and caches the result for **5 minutes** (keyed by `sessionId`). Subsequent turns within the same session reuse the cached context — the Memory Engine is not called again unless the cache expires. +**When it runs:** at agent-compile time, before turn 1. The fetcher caches the result for **5 minutes**, keyed by `roomId`. Subsequent turns within the same room reuse the cached context — including a fresh session for the same room — and the Memory Engine is not called again unless the cache expires. **What appears in the prompt:** if at least one slot is non-empty, the runtime inserts a `## What you know about the user` block into the system prompt containing all populated slots. If every slot is empty (no prior memory for this user), the block is omitted entirely. +The block is deduped and budgeted before it reaches the model: entities are collapsed by name (keeping the richest summary), facts / episodes / communities are collapsed by normalized text so exact repeats across the six slots don't render 4-5 times, and per-entity and whole-block caps are applied. If either cap trims content, the block ends with a `_(More remembered — ask me to recall.)_` footer — trimmed detail is still reachable via the memory tool. No unique fact is dropped, and paraphrases survive. + **Implication for oracle authors:** you do not need to instruct the agent to "look up the user's context" or "recall memory before responding" in `config.prompt.opening` or anywhere else. The context is already in the system prompt when the agent sees the user's first message. Adding such instructions is redundant and wastes tokens. -The 5-minute session cache means very-recent memory writes (e.g. the agent just called `memory-engine__add_memory` in the same session) may not appear in the fetched context until the next session or cache expiry. This is intentional — the fetcher is optimised for read latency, not write-through consistency. +The 5-minute room-keyed cache means very-recent memory writes (e.g. the agent just called `memory-engine__add_memory` in the current turn) may not appear in the fetched context until the cache expires. This is intentional — the fetcher is optimised for read latency, not write-through consistency. ## Adding global oracle knowledge diff --git a/build-an-oracle/reference/environment-variables.mdx b/build-an-oracle/reference/environment-variables.mdx index b3ed7cf..f2e6702 100644 --- a/build-an-oracle/reference/environment-variables.mdx +++ b/build-an-oracle/reference/environment-variables.mdx @@ -73,6 +73,19 @@ Always present. Both are `z.coerce.number()` — set them as plain integer strin Cross-field check (`validateLlmProviderKey`): the API key for the selected `LLM_PROVIDER` must be present — `OPEN_ROUTER_API_KEY` when `LLM_PROVIDER=openrouter` (the default), `NEBIUS_API_KEY` when `LLM_PROVIDER=nebius`. A missing key fails boot with a named-field error (e.g. `OPEN_ROUTER_API_KEY` / `NEBIUS_API_KEY`) rather than a generic upstream 401 at request time. The per-role model ids are hardcoded per provider — there is no env var to swap the main model id (use the `resolveModel` hook for that). +### Latency tuning + +| Variable | Type | Default | Notes | +| --- | --- | --- | --- | +| `MAIN_REASONING_EFFORT` | `'low' \| 'medium' \| 'high'` | `'medium'` | Extended-thinking effort for the **main** agent role only. Sub-agent reasoning is unchanged. Lower values reduce time-to-first-token at some cost to hard multi-step reasoning; higher values do the reverse. `'medium'` preserves current behaviour, so leaving this unset is safe. | + + +Two related latency wins ship **always-on** — there is no env var to disable them: + +- **Request-path caches.** The per-user `SqliteSaver` is reused across the two build-time hook calls per turn instead of being rebuilt (and re-running `setup()`); the agent builder reads prior build-time checkpoints via a light path that skips the message-history join (O(1) instead of O(history)); and the Memory Engine `userContext` cache is keyed by `roomId` instead of `sessionId`, so a fresh session for the same room reuses the cached context. +- **Memory-context compaction.** The `## What you know about the user` prompt block is deduped across all six memory buckets (entities collapsed by name, keeping the richest summary; facts / episodes / communities collapsed by normalized text — lowercase, punctuation stripped, whitespace collapsed) and bounded by fixed internal caps (per-entity summary and whole-block budget). When either cap trims content, the block ends with a `_(More remembered — ask me to recall.)_` footer so trimmed detail stays reachable via the memory tool. No unique fact is dropped; paraphrases survive. + + ### Misc | Variable | Type | Default | Notes | From a6e4cf060ac8b60a8cef5892b421186b656b5428 Mon Sep 17 00:00:00 2001 From: "mintlify[bot]" <109931778+mintlify[bot]@users.noreply.github.com> Date: Tue, 7 Jul 2026 01:13:06 +0000 Subject: [PATCH 2/2] docs: tighten latency-tuning wording in env vars reference --- build-an-oracle/reference/environment-variables.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-an-oracle/reference/environment-variables.mdx b/build-an-oracle/reference/environment-variables.mdx index f2e6702..f2db1e6 100644 --- a/build-an-oracle/reference/environment-variables.mdx +++ b/build-an-oracle/reference/environment-variables.mdx @@ -82,7 +82,7 @@ Cross-field check (`validateLlmProviderKey`): the API key for the selected `LLM_ Two related latency wins ship **always-on** — there is no env var to disable them: -- **Request-path caches.** The per-user `SqliteSaver` is reused across the two build-time hook calls per turn instead of being rebuilt (and re-running `setup()`); the agent builder reads prior build-time checkpoints via a light path that skips the message-history join (O(1) instead of O(history)); and the Memory Engine `userContext` cache is keyed by `roomId` instead of `sessionId`, so a fresh session for the same room reuses the cached context. +- **Request-path caches.** Three wins ship together. The per-user `SqliteSaver` is reused across the two build-time hook calls per turn instead of being rebuilt on each call. The agent builder reads prior build-time checkpoints via a light path that skips the message-history join (O(1) instead of O(history)). The Memory Engine `userContext` cache is keyed by `roomId` instead of `sessionId`, so a fresh session for the same room reuses the cached context. - **Memory-context compaction.** The `## What you know about the user` prompt block is deduped across all six memory buckets (entities collapsed by name, keeping the richest summary; facts / episodes / communities collapsed by normalized text — lowercase, punctuation stripped, whitespace collapsed) and bounded by fixed internal caps (per-entity summary and whole-block budget). When either cap trims content, the block ends with a `_(More remembered — ask me to recall.)_` footer so trimmed detail stays reachable via the memory tool. No unique fact is dropped; paraphrases survive.