K3 — Search is a tool call, and retrieval belongs over the tools

[davidmckayv]

K3. Follows K1 (answer from a live system) and K2 (as me, not as the deployment). Fuller version
in Notion under Architecture, same title.

What OpenBot is, and what that forces

OpenBot is a template an enterprise deploys, not a hosted product we run. The deployment picks its
own model — OpenAI, Gemini, Anthropic, a gateway, something self-hosted. OPENAI_BASE_URL,
ANTHROPIC_BASE_URL and GOOGLE_GENERATIVE_AI_BASE_URL exist for exactly that reason.

So anything that only works on one model vendor's API is not an answer for us. A deployment on GPT
or a self-hosted model would get nothing, and the scaling story of a governance platform cannot be
conditional on which model somebody chose.

The enterprises deploying this can be Stripe-shaped: 1,000 tools is the design target, not a
hypothetical. Stripe hit exactly this and built their own layer. So do we.

The two rules

1. We never index a customer's documents. A Bot answers from a live system by calling that
   system's own search, as the person asking. The vendor decides what they may see.
2. Retrieval belongs over the tool catalogue — our own metadata, not somebody's files.

What Stripe actually built

Kai: company-wide, 296 → 5,000+ users in four weeks, 83% weekly active. One engineer, one week, on
LangGraph Deep Agents. Behind it: 500+ MCP tools and 1,000+ skills from 100+ teams, federated.

The mechanism, and it is the whole thing:

• A skill is an on-demand capability in a small folder; a SKILL.md says when to use it and how.
• The agent reads lightweight metadata from every skill at startup. Full content loads only when
  the task needs it.
• Skill selection gates tool context through allowedTools lists. Each skill declares the tools
  it needs, and picking the skill is what loads them.
• Their words: "The LLM is spending effort figuring out which skills to load, so that is what we then
  rely on to load the relevant tools as well."
• Tiered: foundational (pinned), function-specific (by role), user-level additions.

Measured limits worth stealing: frontmatter capped at 1,024 characters; quality degrades past
~150 skills alongside the system prompt; pure LLM selection currently beats RAG when full
context is available, with RAG/classifier planned as a prefilter at larger scale rather than a
replacement for the model's choice.

Note what this is not: not a vendor API feature, and not embeddings-first. Two-pass selection on
their side of the model call, which is why it works whatever model is underneath.

The design for OpenBot

The seam already exists. server/src/plugins/tools.ts → grantedTools({ store, botId, actorId })
is the single place the tool array is built, per Bot per person, before anything reaches a model.
Every provider takes a tools array, so retrieval placed here is model-agnostic by construction.

We already have most of the pieces:

Piece	State
Skills	Exist — slug, title, summary, instructions, owner, origin
Grants	Exist — per Bot, per tool, enforced
Plugin store, policy, audit	Exist
Skills declaring the tools they need	Missing. This is the work.
Two-pass selection in grantedTools	Missing.

The build, in order:

1. Skills declare their tools (allowedTools). The keystone — without it there is no unit of
   retrieval.
2. Two-pass selection in grantedTools. Pass one puts skill summaries in context, not tool
   definitions; the model picks skills; pass two loads only those skills' tools. Both ours.
3. Tier the skills — pinned, by role, personal — so pass one stays under the ~150 that degrades.
4. Prefilter when a deployment outgrows that. BM25 over skill summaries first: no embedding model,
   no vector store, nothing a deployment has to go and acquire. Embeddings only if BM25 measurably is
   not enough, behind an interface so a deployment can bring its own.
5. Namespace tool names by vendor now — google_drive_*, slack_*. Renaming later breaks grants.
6. Write descriptions for retrieval. They are the index.

One layer, no vendor paths. Every deployment gets the same behaviour whatever model it runs.
No branch for a particular provider's API, no capability that exists only on one of them. A second
path is a second thing to test, a second thing to reason about when a boundary misbehaves, and a
quiet way for this to become worse on the models most deployments actually use.

The governance rule, which is ours alone

Deferred loading must never mean deferred authorization. Retrieval decides what the model can
see; the grant, the policy and the audit row decide what it can do.

• The catalogue is already narrowed to what this Bot has been granted before retrieval runs.
  Otherwise it advertises tools the Bot cannot call and every hit is a wasted turn ending in a
  refusal. Narrowing first is also cheaper.
• A discovered tool still goes resolve → decide → audit → act. Discovery is not permission.
• The audit row should record that a tool was discovered, not only called, or "why did it call
  that" cannot be answered.

The tools to build

Search at the vendor, as the asker, read-only first. Writes are a separate decision with their own
grant.

Vendor	Tools	Note
Google Drive	search, recent, read content, metadata	Built in #97, the reference
Microsoft 365 / SharePoint	search, recent, read	Where most enterprises are
Slack	message search, channel history	Per-user token essential — membership is the permission
Confluence / Jira	CQL search, issue search, page read	Atlassian search already applies permissions
Notion	search, page read	Same shape
Email	thread search, message read	Highest custody risk; clearest case for rule 1
Salesforce / ServiceNow	record search	Record-level sharing not reproducible outside the vendor

Ruled out

Ingesting, chunking or embedding customer documents. documents, chunks, document_acls and the
pgvector extension serving them. Any ACL model of our own over customer content. A worker syncing a
corpus on a schedule. Any tool-scaling design that only works on one model vendor.

Status

• Write half of the old document index removed by Answer from Google Drive, as the person asking #97, which also delivers per-person Drive.
• Read half added by Answer from the documents the asker may read #113, taken back by Do not search documents out of our own database #118.
• The three tables and the remaining server/src/knowledge/ modules are read by nothing once both
  land; drop them in a change that says so.
• Skills declaring tools, and two-pass selection, are not built. That is the K3 work.

Sources

• https://www.langchain.com/blog/how-stripe-built-their-knowledge-ai-platform-on-deep-agents
• https://unmesh.dev/post/langgraph-skills/
• https://modelcontextprotocol.io/docs/2026-07-28/develop/clients/client-best-practices
• https://arxiv.org/html/2605.05538v1