CLI agent-surface truth + the manifest rail - #1571
Open
aaj3f wants to merge 8 commits into
Open
Conversation
…t 60%) Both live in the corpus embedded into the binary (fluree docs + the MCP docs toolset), so agents consume them as version-exact truth: - policy cookbook + policy-in-transactions: all five f:query examples put the ?$this correlation clause under a '$where' key that no parser reads — unknown keys are silently dropped, so every documented 'restriction' policy matched any identity and allowed everything. Rewritten to the single where-array form the passing tests use (correlate via a shared plain variable), intermediates renamed from ?$user/?$mgr/?$subject to plain variables (the ?$ prefix belongs to the reserved/injected bindings), plus a callout on the silent-allow failure mode and --track-policy verification. - --memory-budget-mb docs claimed auto = 60% of RAM; import.rs:288 is 80%, sized for a dedicated machine. Corrected, with an explicit co-residency warning (live incident: an 8.6GB import at auto budget OOM'd a machine running normal desktop workloads).
.fluree/config.toml carries live access AND refresh tokens after auth
login, and 'fluree config list' dumped them raw — the one command
docs/cli/auth.md claimed never printed refresh tokens. list now routes
the parsed tree through the same redact_json_secrets policy graph-source
info already uses ([redacted] convention, fail-closed), on both the TOML
and JSON-LD config paths, with a stderr note pointing at --reveal.
'config get <key>' keeps raw output (explicit single-key request), and
'auth token' remains the sanctioned scripting path.
SECRET_CONFIG_KEYS gains refresh_token (the graph-source redactor never
met one, but the CLI config stores one per remote). auth token help and
docs/cli/{auth,config}.md updated to match; integration test covers
masked-by-default + --reveal.
fluree __manifest [-o FILE] emits a machine-readable manifest of this binary's CLI surface — command paths, flags, value enums, positionals, compiled features — generated by walking the clap tree, so it cannot drift from what ships. Structure only, deliberately no help text (prose belongs to --help and the embedded docs; including it would churn the manifest on every wording edit and tempt consumers to copy reference content instead of deferring to the binary). cargo-dist extra-artifacts publishes it as fluree-cli-manifest.json on each release. Consumers: fluree/solo CI (validates the centralized ui/lib/cli-snippets.ts against it) and the fluree-cli Claude Code plugin CI (validates its contract file). Hidden commands are excluded from the manifest — they are not teachable surface. Note: dist plan not exercised locally (dist not installed); the first release CI run exercises the extra-artifacts hook. Manifest observation: several --format flags are plain strings, so their value sets are invisible to both --help and the manifest — converting them to clap value enums is a follow-up.
New tests/docs_coverage.rs walks Cli::command() and fails when a non-hidden command lacks a docs/cli page, a page outlives its command, a nested action is never mentioned in its parent page, the README index skips a command, or SUMMARY.md orphans a page from the published book (the embedded corpus ships the whole folder, so an orphan is served to agents while invisible on the website — the worst kind of split). Previously adding a subcommand required zero doc changes to pass CI. Fixes everything the gate caught on main: branch.md gains the missing 'fluree branch revert' section (written from the clap definition); README.md indexes branch, multi-query, iceberg, publish, cache, cluster, context (7 of 42 commands were unlinked); SUMMARY.md gains cli/load.md and cli/server-integration.md.
…ifest rename - remote add now reads the optional cli.min_version block from /.well-known/fluree.json (the field the paired fluree/solo PR serves) and warns when the running binary is older than what the stack's docs and UI teach. Lenient semver compare (pre-release ignored, malformed advertisements never warn); pure helper + unit tests. - The discovery-failure fallback that silently guessed '<url>/fluree' as the API base now says so and tells the user the fix, instead of failing later with 404s on every request. Inputs already ending in /fluree are still used as-is (that path was verified correct — the suffixed form solo teaches is robust on both discovery paths). - __manifest renamed to (hidden) manifest: clap_complete's bash generator uses '__' as its command-path separator in completion function names and panics on a command literally named __manifest (caught by the completions_bash integration test). The name is what consumer CI scripts against, so the collision-free name lands before anything consumes it; dist extra-artifacts + tests updated.
- ai/README.md: the CLI MCP bullet finally mentions the docs toolset (docs_search/get/examples/tree — it listed only memory tools, hiding the version-exact docs surface from the very audience it serves); the vector-search claim now states plainly that the HNSW index is behind the non-default 'vector' feature while the inline similarity functions are in every build (same doc-lie class as the $where fix: standard binaries cannot serve an HNSW claim). - New ai/claude-code.md: the versioned operating guide for agents driving the CLI — probe-first doctrine (docs/--help/hidden manifest, feature-gated command disappearance), machine-output map, remote + device-flow choreography (the human approves /activate), credential hygiene (auth token vs config list --reveal, .fluree/ gitignore), destructive-op confirmation rules, shared-machine resource budgets, policy correlation + --track-policy --direct, and query hygiene (--at WHERE-substring rule, auto-routing vs --direct). Ships in the embedded corpus so agent-side packaging (the fluree-cli Claude Code plugin, Solo's generated build prompt) defers to it instead of copying it.
Contributor
Author
|
First live use of the manifest rail, during plugin-contract authoring: the checker caught that |
…eth, release-day hardening Response to independent review of #1571; every finding verified before fixing (all four confirmed real). - docs/ai/claude-code.md machine-output bullet was wrong three ways: 'list --json' and 'branch merge --preview' don't exist, and --envelope is query-only (not multi-query). Rewritten from the manifest: --json lives on exactly graph list, branch diff, branch revert, and the four docs subcommands; list/info/show are human-only. - manifest gains a root entry (path: []) with the clap-builtin --version exposed (requires cmd.build() — builtins only materialize then), so 'fluree --version' is validatable and fluree/solo#893's declared root entry passes instead of red-lighting the day the release asset ships. Also: possible_values now emitted only when the arg takes values (SetTrue booleans no longer report a spurious true/false enum, so the two real value enums aren't buried under ~60 fake ones), and the module doc states what the manifest deliberately omits (defaults, num_args, relational constraints — 'required' is not the whole truth). - docs_coverage nested-action check was mutation-toothless (bare substring; a prose 'revert' satisfied it). Now matches the full invocation path ('branch revert'); re-ran the reviewer's mutation and the gate fails on it, restored tree passes. Module doc no longer overclaims feature-portability. - dist extra-artifacts drops --release: it runs in the global-artifacts job on the default 4-core/16GB runner, and this workspace's release profile is fat-LTO + codegen-units=1 — the config the same file records as OOMing at link time. Debug is fine; the manifest is pure introspection. Comment also pins the no-features invariant. - auth login now runs the same cli.min_version check as remote add (the recurring touchpoint — a stack that raises its floor after add was previously never heard from again), making the solo-side CliCompat doc comment accurate. - fluree-db-api gains the test tying RemoteAuth's credential fields to SECRET_CONFIG_KEYS across the crate boundary (refresh_token previously had no coverage in the crate that owns the list); features test now pins cluster<->server alongside validate<->shacl, plus a root-entry test and a no-spurious-enum test.
aaj3f
added a commit
to fluree/claude-plugins
that referenced
this pull request
Jul 30, 2026
Response to independent review of #1; every finding verified before fixing. The three text blockers were all real against v4.1.4, the latest installable release: - SKILL.md credential rail was INVERTED: it said config list redacts on 'current versions' when redaction ships only in the first release after fluree/db#1571 — an agent on 4.1.4 would have blessed a command that dumps auth.token AND refresh_token in plaintext. Rewritten to be safe on binaries that exist: treat config list output as secret through v4.1.4; verify --reveal exists before assuming redaction. - workflows.md taught 'branch diff main dev', which does not parse (one positional + --target, defaulting to the source's parent). - The 'start here' pointer to docs get ai/claude-code is now hedged the same way the manifest pointer already was (page ships after #1571), and 'supersedes' narrowed to command-surface authority. Floor moves to 4.1.4 in lockstep with the paired solo review (auth token does not exist at 4.1.3) — SKILL, setup, README, cli-facts. CI honesty (the job could report green while running nothing): - 404 is the only skip; every other curl outcome is now ::error + fail. The skip is a ::warning with an owner and expiry (issue #2 filed). - Marketplace root now validated (the per-plugin loop never saw it); claude-code npm install pinned (2.1.220); .mcp.json syntax-checked in CI (validate --strict provably ignores it) and its nonstandard description key dropped. Checker soundness: - exit 2 (not a stack trace masquerading as 'contract violated') for unreadable/malformed inputs; commands-array guard; version fallback. - min_cli_version is now load-bearing: the checker fails when the manifest is older than the contract's floor. manifest_source is now read by CI (single source of truth for the URL). - hidden entries assert ABSENCE from the manifest (catches a hidden->visible promotion of ); checked-count counts checked commands, not list length. Contract covers the prose it missed: query --at/--policy-class, --json on the four docs subcommands, bare model, branch diff --target/--json, branch revert --json. New: the inbound-MCP section WS-4 asked for (claude mcp add --transport http against a Space's /v1/mcp?space=<id> — the Data Admin tab documents Claude Desktop only); README states the two intentional doctrine-mirror relationships and what wins where; honest failed-server description for a missing binary; .DS_Store untracked + .gitignore added; homepage in plugin.json; skill trigger scoped to terminal contexts + SDK-only check in the body. Declined with reason (recorded in the response doc): flag-VALUE validation — mcp init's --ide/--toolsets and query --format expose no possible_values in the manifest today (plain strings), so the mechanism would be vacuous; riding the 'convert --format flags to value enums' follow-up in fluree/db instead.
aaj3f
added a commit
to fluree/claude-plugins
that referenced
this pull request
Jul 30, 2026
Response to independent review of #1; every finding verified before fixing. The three text blockers were all real against v4.1.4, the latest installable release: - SKILL.md credential rail was INVERTED: it said config list redacts on 'current versions' when redaction ships only in the first release after fluree/db#1571 — an agent on 4.1.4 would have blessed a command that dumps auth.token AND refresh_token in plaintext. Rewritten to be safe on binaries that exist: treat config list output as secret through v4.1.4; verify --reveal exists before assuming redaction. - workflows.md taught 'branch diff main dev', which does not parse (one positional + --target, defaulting to the source's parent). - The 'start here' pointer to docs get ai/claude-code is now hedged the same way the manifest pointer already was (page ships after #1571), and 'supersedes' narrowed to command-surface authority. Floor moves to 4.1.4 in lockstep with the paired solo review (auth token does not exist at 4.1.3) — SKILL, setup, README, cli-facts. CI honesty (the job could report green while running nothing): - 404 is the only skip; every other curl outcome is now ::error + fail. The skip is a ::warning with an owner and expiry (issue #2 filed). - Marketplace root now validated (the per-plugin loop never saw it); claude-code npm install pinned (2.1.220); .mcp.json syntax-checked in CI (validate --strict provably ignores it) and its nonstandard description key dropped. Checker soundness: - exit 2 (not a stack trace masquerading as 'contract violated') for unreadable/malformed inputs; commands-array guard; version fallback. - min_cli_version is now load-bearing: the checker fails when the manifest is older than the contract's floor. manifest_source is now read by CI (single source of truth for the URL). - hidden entries assert ABSENCE from the manifest (catches a hidden->visible promotion of 'fluree manifest'); checked-count counts checked commands, not list length. Contract covers the prose it missed: query --at/--policy-class, --json on the four docs subcommands, bare model, branch diff --target/--json, branch revert --json. New: the inbound-MCP section WS-4 asked for (claude mcp add --transport http against a Space's /v1/mcp?space=<id> — the Data Admin tab documents Claude Desktop only); README states the two intentional doctrine-mirror relationships and what wins where; honest failed-server description for a missing binary; .DS_Store untracked + .gitignore added; homepage in plugin.json; skill trigger scoped to terminal contexts + SDK-only check in the body. Declined with reason (recorded in the response doc): flag-VALUE validation — mcp init's --ide/--toolsets and query --format expose no possible_values in the manifest today (plain strings), so the mechanism would be vacuous; riding the 'convert --format flags to value enums' follow-up in fluree/db instead.
aaj3f
added a commit
to fluree/claude-plugins
that referenced
this pull request
Jul 30, 2026
…ned hedges (1.0.1) Response to a fresh-session live test of v1.0.0 driving the plugin as a real consumer; every finding re-verified against a binary before fixing. - workflows.md insert recipe had two independent bugs: prefixed-name Turtle fails insert's data sniffer (the positional becomes a LEDGER NAME and the CLI waits on stdin — a silent hang in agent sessions), and 'ex:' was never declared (Turtle parse error even routed right). Recipe now uses @Prefix + explicit -e, with a rail note naming both hazards. --explain gets its required --format json; the export recipe gains the required 'fluree index' step. - SKILL Rule 3's --envelope claim corrected: it pairs ONLY with --format ndjson (verified: 'error: --envelope only applies to --format ndjson'). Same bug is being fixed in fluree/db#1571's ai/claude-code.md. - policy.md now carries the crucial caveat the tester proved live: the cookbook EMBEDDED in binaries <= 4.1.4 teaches the broken $where form (their A/B: cookbook policy 2/2 allowed vs corrected 1/2), and this page previously sent readers there without warning. Corrected cookbook ships in releases after fluree/db#1571. - troubleshooting.md's unhedged 'fluree manifest' and ai/claude-code references now carry the same after-#1571 hedge as SKILL.md; the vague 'recent releases only' wording tightened; the docs-search fallback changed to "AI agents" (bare 'agent' ranks HTTP User-Agent docs first). - setup step 2: on a floor failure, check 'which -a fluree' for a shadowed newer install before recommending a reinstall — the tester's own machine had 4.1.4 shadowed by 4.1.2 on PATH. Version 1.0.1 in plugin.json + marketplace so installed copies see the update.
A fresh-session live test of the fluree-cli plugin caught the agent guide implying 'query --format json --envelope' works; the binary says 'error: --envelope only applies to --format ndjson'. Corrected here and in the plugin's SKILL in the same pass — the second machine-output claim in this page that only running the binary caught.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Fluree AI (fluree/solo) already hands its users' AI assistants off to this CLI — a generated build prompt, LLM-targeted served docs — and the embedded
docs/corpus is what agents consume as version-exact truth viafluree docsand the MCPdocstoolset. This PR fixes what that corpus gets wrong, closes the CLI-side credential leak, and builds the rail that lets every downstream teacher of CLI grammar (fluree/solo's UI and docs, the upcomingfluree-cliClaude Code plugin) validate what it teaches against what actually ships.What
Doc truth in the embedded corpus (
2dc6ffb, part of4dfc…docs/aibelow)?$thiscorrelation clause under a nonexistent$wherekey — silently dropped by the parser, so every documented "restriction" policy allowed everything — rewritten to the single correlatedwhere-array form the passing tests use, with a callout on the silent-allow failure mode and--track-policyverification.--memory-budget-mbauto is documented as 80% of RAM (code truth; docs said 60%) with an explicit co-residency OOM warning.Credential hygiene (
c497bbe)fluree config listredacts credentials as[redacted]by default (it dumps.fluree/config.toml, which carries live access and refresh tokens afterauth login) — sameredact_json_secretspolicy and convention the graph-sourceinfopath already uses.--revealis the documented raw escape hatch;config get <key>stays raw;auth tokenremains the scripting path.refresh_tokenjoinsSECRET_CONFIG_KEYS.The manifest rail (
438bd51,071e0ed)fluree manifestemits a machine-readable JSON of the CLI surface — command paths, flags, value enums, positionals, compiled features — generated by walking the clap tree so it cannot drift from the binary. Structure only, deliberately no help text. cargo-distextra-artifactspublishes it asfluree-cli-manifest.jsonon each release for consumer CI (fluree/solo'scli-snippetstest, the plugin's contract check).__manifestand renamed the same day: thecompletions_bashintegration test caught that clap_complete's bash generator uses__as its command-path separator and panics on a command with that literal name. The name is what consumers script against forever, so the collision-free name lands before anything consumes it.fluree remote addreads the optionalcli.min_versionfrom/.well-known/fluree.json(served by the paired fluree/solo PR) and warns when this binary is older than what the stack's docs teach; lenient semver compare, malformed advertisements never warn. The discovery-failure fallback that silently guessed<url>/flureenow says so and names the fix.Docs-coverage gate (
221d01a)tests/docs_coverage.rswalksCli::command()and fails when a command lacks adocs/clipage, a page outlives its command, a nested action is never mentioned in its parent page, the README index skips a command, orSUMMARY.mdorphans a page (orphans are still embedded — served to agents while invisible on the website). Adding a subcommand previously required zero doc changes to pass CI. The gate immediately caught real gaps onmain, all fixed here: the missingfluree branch revertsection, 7 of 43 commands unlinked from the README index, andload.md+server-integration.mdorphaned from the book.Agent guide (last commit)
docs/ai/README.mdun-staled (the CLI MCP bullet omitted the entiredocstoolset; the vector claim now states the HNSW index is behind the non-defaultvectorfeature while inline similarity functions are in every build).docs/ai/claude-code.md: the versioned operating guide for agents driving this CLI — probe-first doctrine, machine-output map, remote/device-flow choreography, credential hygiene, destructive-op confirmation rules, shared-machine budgets, policy correlation. It ships in the embedded corpus precisely so agent-side packaging defers to it instead of copying it.Cross-repo pairing
feat/cli-ai-surface) is the other half: servescli.min_versionin.well-known/fluree.json, fixes the platform-side handoff defects, centralizes its UI's CLI grammar intoui/lib/cli-snippets.ts, and carries an env-gated test that validates those snippets against this PR's manifest once a release ships the asset. Either PR can land first; the handshake and the manifest check both degrade silently until both sides exist. (Post-review: the manifest now emits a rootpath: []entry with--versionexposed — the producer-side resolution both reviews preferred — so solo’s declared root entry validates as-is.)fluree-cliClaude Code plugin (PR to follow) whose contract file validates against the same release-asset manifest and whose skill defers to this PR'sdocs/ai/claude-code.md.Validation
cargo fmtclean;clippy -p fluree-db-cli --all-targets --no-deps0 warnings; full crate suite green (165 lib + 120 integration + 5 docs-coverage);fluree-db-apiredaction tests green. Honest caveat: the distextra-artifactshook is config-only here and gets its first real exercise on the next release CI run.Follow-ups (deliberately not here)
--jsonforlist,info, andshow(the last human-only outputs an agent needs).--formatflags to clap value enums so their choices surface in--helpand the manifest.repository_dispatchfrom release CI to fluree/solo + fluree/claude-plugins so a breaking CLI change surfaces in consumer CI the day it ships.