Have Knowledge admit no source is connected, instead of citing one - #58
Conversation
93cf23e to
7b81387
Compare
|
Rebased onto main and pushed one commit on top: the wording now keeps the instruction to cite alongside the admission that there is nothing to cite. Your diagnosis was right and the Both halves, and it still matches the connectors-page rule you cited: say plainly what isn't there. On the local tooling: nothing to apologise for, and stashing to reproduce on a clean |
davidmckayv
left a comment
There was a problem hiding this comment.
Approved. Narrow, correct, and the doc mirror is kept in sync.
What this changes
The Knowledge coworker's
role_descriptioninexamples/fintech/agents.yamlalready says it should "cite sources when available" — correctly hedged, since not every deployment has a knowledge source connected. Itssystem_prompt, the text actually sent to the model, disagreed: "Answer from authorized company knowledge and cite every source," with no hedge at all. The same unconditional wording is repeated in theagents.yamlexample insidedocs/configuration.md.Out of the box,
TENANT_PACKAGE_DIRdefaults to this fintech example, so a fresh install ships a Knowledge coworker instructed to always cite a source, whether or not one is connected — a citation invented under instruction reads the same as a real one.This changes the wording in both places to match the hedge already present in the same block, matching a pattern the connectors page already uses on purpose:
app/src/routes/_authed/admin/connectors.tsxshows "No setup screen yet" for an unconfigured OneDrive connector rather than a broken or blank control, specifically so the product says plainly what isn't there instead of implying it. The Knowledge prompt should hold itself to the same rule.This is a narrow fix to the example config and its matching doc block. It does not touch the underlying gap that a Knowledge coworker has no retrieval behind it in this repo yet — I'm filing that separately as an issue, since it's a bigger, cross-cutting change that should be proposed before it's built.
Where it runs
Boundary and audit
Proof
This is a two-line text change to a YAML string and its doc mirror; no runtime code, imports, or tests reference the exact string (
grep -rn "cite every source"matched only these two files before the change).I could not get a clean local run of
format:check/lint/typecheckin this dev environment:bunx biome format .andbunx biome lint .both crash (access violation) on this machine, andbun run typecheckreportsCannot find module 'yaml'/'zod'/etc. from several files. I confirmed both failures are pre-existing and unrelated to this change by stashing it and reproducing the identical errors on a cleanmaincheckout with zero edits — looks like a package-linking issue local to this Windows setup, not something this PR introduces.bun test tests/fintech-package.test.ts, which exercises this example package, doesn't assert the exact prompt string either way and isn't affected. Happy to have CI be the real signal here.