Skip to content

Have Knowledge admit no source is connected, instead of citing one - #58

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
Hotragn:have-knowledge-admit-no-source
Aug 20, 2026
Merged

Have Knowledge admit no source is connected, instead of citing one#58
davidmckayv merged 2 commits into
CopilotKit:mainfrom
Hotragn:have-knowledge-admit-no-source

Conversation

@Hotragn

@Hotragn Hotragn commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

What this changes

The Knowledge coworker's role_description in examples/fintech/agents.yaml already says it should "cite sources when available" — correctly hedged, since not every deployment has a knowledge source connected. Its system_prompt, the text actually sent to the model, disagreed: "Answer from authorized company knowledge and cite every source," with no hedge at all. The same unconditional wording is repeated in the agents.yaml example inside docs/configuration.md.

Out of the box, TENANT_PACKAGE_DIR defaults to this fintech example, so a fresh install ships a Knowledge coworker instructed to always cite a source, whether or not one is connected — a citation invented under instruction reads the same as a real one.

This changes the wording in both places to match the hedge already present in the same block, matching a pattern the connectors page already uses on purpose: app/src/routes/_authed/admin/connectors.tsx shows "No setup screen yet" for an unconfigured OneDrive connector rather than a broken or blank control, specifically so the product says plainly what isn't there instead of implying it. The Knowledge prompt should hold itself to the same rule.

This is a narrow fix to the example config and its matching doc block. It does not touch the underlying gap that a Knowledge coworker has no retrieval behind it in this repo yet — I'm filing that separately as an issue, since it's a bigger, cross-cutting change that should be proposed before it's built.

Where it runs

  • New state that outlives a request? None. This is a string in two static files.
  • What happens on the second replica? Identical behavior on every replica; nothing is held in a process.
  • Anything serialised? N/A, no writes.
  • Anything fanned out to a browser? No.
  • New listener, port, or schedule? No.

Boundary and audit

  • N/A — no gateway, policy, or audit path touched.

Proof

This is a two-line text change to a YAML string and its doc mirror; no runtime code, imports, or tests reference the exact string (grep -rn "cite every source" matched only these two files before the change).

I could not get a clean local run of format:check/lint/typecheck in this dev environment: bunx biome format . and bunx biome lint . both crash (access violation) on this machine, and bun run typecheck reports Cannot find module 'yaml'/'zod'/etc. from several files. I confirmed both failures are pre-existing and unrelated to this change by stashing it and reproducing the identical errors on a clean main checkout with zero edits — looks like a package-linking issue local to this Windows setup, not something this PR introduces. bun test tests/fintech-package.test.ts, which exercises this example package, doesn't assert the exact prompt string either way and isn't affected. Happy to have CI be the real signal here.

@davidmckayv
davidmckayv force-pushed the have-knowledge-admit-no-source branch from 93cf23e to 7b81387 Compare August 20, 2026 22:36
@davidmckayv

Copy link
Copy Markdown
Contributor

Rebased onto main and pushed one commit on top: the wording now keeps the instruction to cite alongside the admission that there is nothing to cite.

Your diagnosis was right and the role_description above it is the giveaway: it already hedges with "when available" and the system_prompt did not. But dropping "cite every source" outright leaves nothing telling it to cite when a source is connected, which is the other half of the same promise. So:

Answer from authorized company knowledge and cite your sources. When none is connected, say so plainly rather than inventing a citation.

Both halves, and it still matches the connectors-page rule you cited: say plainly what isn't there.

On the local tooling: nothing to apologise for, and stashing to reproduce on a clean main was the right way to establish it wasn't yours. CI had also not run on this PR at all, which the rebase should fix. Thank you for filing the retrieval gap separately rather than widening this.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Narrow, correct, and the doc mirror is kept in sync.

@davidmckayv
davidmckayv merged commit 6c365f4 into CopilotKit:main Aug 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants