AS215932 Knowledge Repository

Private OKF knowledge bundle for Servify / Hyrule / AS215932.

This repository is the generated and curated institutional knowledge database for AS215932. It uses Google's draft Open Knowledge Format (OKF): Markdown files with YAML frontmatter, plus deterministic JSONL and SQLite exports for integration. It also provides the read-only foundation for the AS215932 governed learning control plane: authority tiers, machine-checkable claims, policy decisions, context packs, and deterministic eval fixtures.

LLM-Wiki operations & grounding

This repo is an agent-maintained "LLM Wiki": okf/generated + okf/observed are the regenerable raw layer, okf/curated is the reviewed wiki layer, and three named operations run over it — ingest (hyrule-knowledge ingest, nightly loop), query (query/resolve/claims/context-pack, Knowledge MCP), and lint (hyrule-knowledge lint: schema validation + quality findings + claim conflicts + stale claims in one read-only command).

Knowledge is additionally scored by use (the CGS-substrate layer): loop insight decisions cite OKF concepts as evidence, and those citations flow back into per-concept grounding scores, lint findings, and retrieval ranking. The human audit view is Obsidian: open okf/ directly as a vault (docs/obsidian-audit.md).

Insight stream & the IDQ / CGS / LL metrics

Every AS215932 loop (engineering, noc, soc, knowledge) records its surface-or-stay-silent decisions as agent-core InsightDecisionRecords; operator dispositions from the Agentic Observatory inbox become InsightLabels. This repo is where that stream is evaluated, following the three metrics from arXiv 2605.06717 ("Agentic Coding Needs Proactivity, Not Just Autonomy"):

• Insight Decision Quality (IDQ) — did the loop pick the right action at the right time? Computed as the fraction of labeled decisions whose action_selected matches the label's reference action or an acceptable alternative. Deliberate silence (stay_silent, including withheld and quiet-interval samples) counts in the denominator — silence is a decision, not a gap.
• Context Grounding Score (CGS) — was a shown insight supported by the right evidence? Over surfaced, labeled decisions with gold evidence: F1(cited OKF refs vs the label's gold evidence_refs) x faithfulness. Faithfulness precedence: the reviewer's verdict wins, then the record's own, then the deterministic recoverability proxy (every cited concept still exists and is not deprecated, every cited claim id resolves); not_applicable decisions are excluded, and CGS reports null (not zero) when no decision qualifies.
• Learning Lift (LL) — does operator feedback improve later decisions? Defined as IDQ(adapted policy) − IDQ(frozen policy) on the same later decision points. Reported as a stable null placeholder until the frozen-vs-adapted replay harness lands (network-operations#412).

Pipeline: loops emit records to the agent-core collector → hyrule-knowledge insights sync validates/sanitizes/dedupes them into ledger/insights/{decisions,labels}/ (committed, human-reviewed via the nightly PR) → exports (insight-decisions.jsonl, insight-labels.jsonl) → hyrule-knowledge insights metrics [--loop <loop>] prints IDQ/CGS/silence per loop. Those numbers gate autonomy promotions (SOC_MODE rungs, NOC standing grants) per the network-operations runbooks.

Evidence contract: insights cite OKF through SourceRef entries of kind: okf_concept / okf_claim with ref = the bare concept/claim id (the export bundle version travels in tool_versions.knowledge_export), produced by the shared agent_core.adapters.knowledge helper.

Grounding ledger: exports/grounding.jsonl (+ grounding sqlite table) derives per-concept usage from the committed stream — citation counts, distinct loops, first/last cited, faithfulness rate, label-helpfulness rate, downstream-success rate. It feeds:

• grounding lint: never_cited (curated + top-level scope, only once production ledger records exist; grounding_exempt: true opts a concept out), poorly_grounded (≥3 citations with faithfulness < 0.5), and verify_overdue (curated verify_by: date passed);
• an optional retrieval tie-break (retrieval_grounding_tiebreak in knowledge.config.yml, off by default): min(citations,10) x faithfulness applied after exact-match/authority/score ordering, so results change only on exact ties — flip it only after an eval-diff review.

Determinism contract: grounding rows derive exclusively from repo-committed inputs (eval fixtures + ledger/insights/), and every record must carry its own created_at, so export --check stays byte-reproducible.

Source-of-truth model

Source repositories remain canonical for implemented behavior, executable state, APIs, config, schemas, tests, CI/CD, deployment workflows, repo-local docs, and intended infrastructure state committed as code/config.

OKF is derivative for repo-owned facts. Generated concepts summarize, index, explain, and cross-link source material, but cite repository paths and commits. OKF may be canonical only for explicitly authored institutional knowledge that does not naturally belong in a single repo: cross-repo architecture, ADR rationale, postmortems, operator lessons, durable policy, escalation rules, and strategy.

Conflict policy:

• If OKF disagrees with repo-owned facts, the repo wins and OKF is stale.
• If repo behavior disagrees with OKF-owned institutional policy, create an adjudication issue/task.
• If live state disagrees with repo intended state, treat it as drift evidence.
• If an agent summary disagrees with cited source evidence, source evidence wins.
• If an OKF fact has no citation/provenance, treat it as unverified/advisory.

Layout

Path	Purpose
okf/	OKF Markdown bundle (open it as an Obsidian vault for human audit).
okf/generated/	Generated concepts from source repos and GitHub metadata. Do not hand-edit.
okf/curated/	Human-authored institutional knowledge. Reviewed edits live here.
okf/observed/	Evidence-only runtime observations with expires_at.
exports/	Deterministic JSONL and SQLite exports generated from OKF (incl. grounding.jsonl).
ledger/	Human-reviewed learning stream: fixtures, proposed events, reviews, and ledger/insights/ (synced loop decisions + labels).
reports/	Quality/coverage/eval reports regenerated by the loop.
schema/	Frontmatter, claim, context-pack, policy-decision, and SQLite schemas.
evals/	Deterministic baseline eval schemas and fixture cases (incl. fixtures/insights/).
src/hyrule_knowledge/	Ingestion, validation, export, retrieval, policy, context-pack, eval, grounding, insight-sync, and MCP tooling.
knowledge.config.yml	Source list, include/exclude patterns, project mapping, retrieval flags.
knowledge-policy.yml	Built-in YAML policy evaluator configuration.

Common commands

uv run hyrule-knowledge ingest
uv run hyrule-knowledge enrich --target all --dry-run
uv run hyrule-knowledge observe --profile safe-health
uv run hyrule-knowledge quality --write
uv run hyrule-knowledge quality --check
uv run hyrule-knowledge validate okf
uv run hyrule-knowledge lint
uv run hyrule-knowledge export
uv run hyrule-knowledge insights sync --collector-url http://127.0.0.1:8770 --max 200
uv run hyrule-knowledge insights metrics --loop noc
uv run hyrule-knowledge query "POST /v1/vm/create schema" --json
uv run hyrule-knowledge context-pack --task "Engineer change to POST /v1/vm/create" --role engineering_loop
uv run hyrule-knowledge eval --check
uv run hyrule-knowledge ledger --check
uv run hyrule-knowledge ledger lifecycle --check
uv run hyrule-knowledge ledger --review noc_shadow:fixture-shadow-eval-summary --promotion-kind summary
uv run hyrule-knowledge scan-secrets okf exports reports evals ledger schema

Regenerate and verify everything:

uv run hyrule-knowledge ingest
uv run hyrule-knowledge validate okf
uv run hyrule-knowledge quality --check
uv run hyrule-knowledge export --check
uv run hyrule-knowledge eval --check
uv run hyrule-knowledge ledger --check
uv run hyrule-knowledge ledger lifecycle --check
uv run hyrule-knowledge scan-secrets okf exports reports evals ledger schema

Manual LLM enrichment uses OpenRouter Claude Sonnet 4.6 by default and requires OPENROUTER_API_KEY unless --dry-run is set:

uv run hyrule-knowledge enrich --target services --model anthropic/claude-sonnet-4.6

Manual safe-health observation uses whichever read-only endpoints are available in the environment and writes observed/evidence concepts under okf/observed/:

PROMETHEUS_URL=http://[mon]:9090 \
HYRULE_MCP_HEALTH_URL=http://127.0.0.1:8765/health \
uv run hyrule-knowledge observe --profile safe-health

Automation

.github/workflows/ingest.yml runs nightly and on manual dispatch. When source knowledge changes, it opens a refresh PR for review.

.github/workflows/auto-merge.yml is the graduated relaxation of that review gate, controlled by the KNOWLEDGE_AUTO_MERGE repo variable (off default / dry_run / on): bot refresh PRs whose diff is confined to regenerable projections (okf/generated, okf/observed, exports, reports) and whose quality report does not regress are queued behind the branch-protection required checks with gh pr merge --auto. Anything touching okf/curated, ledger/ (the reviewed insight/learning stream), src, tests, evals, schema, or .github never auto-merges. Rollout runbook: network-operations/docs/runbooks/knowledge-auto-merge.md.

Knowledge Loop agent

hyrule-knowledge loop --once is the governed producer loop for the knowledge repo. It is separate from the read-only Knowledge MCP server: the MCP serves context; the loop refreshes, validates, enriches/imports when explicitly enabled, and opens reviewable PRs.

Phase 1 is deterministic by default:

uv run hyrule-knowledge loop --once --dry-run

It acquires a singleton lock, updates a per-day ledger, runs ingest plus the standard validation/export/eval/ledger/secret-scan gates, and reports JSON. Set --create-pr or HYRULE_KNOWLEDGE_LOOP_CREATE_PR=1 to push a branch and open a PR when changes exist.

Phase 2 is opt-in producer work:

uv run hyrule-knowledge loop --once \
  --create-pr \
  --enrich-target infrastructure \
  --enrich-live \
  --max-openrouter-calls-per-day 1 \
  --learning-event /var/lib/engineering-loop/learning-events

Live enrichment requires the Knowledge Loop credential scope (OPENROUTER_API_KEY rendered from kv/knowledge-loop) and an explicit daily OpenRouter call budget. Learning-event inputs may be files, directories, or globs; imported artifacts remain review-gated ledger proposals.

Phase 2 can also pull the production insight stream from the agent-core collector (budgeted by the daily ledger, default 200 records/day):

uv run hyrule-knowledge loop --once --create-pr \
  --insight-sync --insight-collector-url http://127.0.0.1:8770

Synced decisions/labels land under ledger/insights/ and ride the reviewed nightly PR — ledger/ is excluded from the auto-merge allowlist precisely so this stream always crosses human eyes. Env equivalents: HYRULE_KNOWLEDGE_LOOP_INSIGHT_SYNC, HYRULE_KNOWLEDGE_LOOP_INSIGHT_COLLECTOR_URL, HYRULE_KNOWLEDGE_LOOP_MAX_INSIGHT_RECORDS_PER_DAY.

Governed agent consumption

Humans can browse okf/index.md. Agents should first read okf/index.md, then directory indexes, then individual concepts. Machine consumers should prefer exports/knowledge.sqlite and the JSONL exports:

• claims.jsonl / claims table for subject-predicate-object facts with authority tier, freshness, and provenance.
• concepts.jsonl, edges.jsonl, and SQLite FTS for exact + graph + text retrieval.
• context-packs.jsonl / context_packs table for policy-aware task context when explicitly generated.
• policy-decisions.jsonl / policy_decisions table for audited policy outputs.
• eval-cases.jsonl and eval-results.jsonl for deterministic retrieval/grounding/policy baselines.
• learning-events.jsonl / learning_events table for sanitized fixture/local-artifact learning summaries awaiting human review.
• insight-decisions.jsonl / insight_decisions table for loop surface/withhold decisions (fixtures + synced ledger/insights/ records) — the IDQ/CGS input stream.
• insight-labels.jsonl / insight_labels table for operator labels (reference action, gold evidence_refs, faithfulness verdicts) from the Observatory inbox.
• grounding.jsonl / grounding table for per-concept grounding scores derived from that stream (citations, faithfulness/helpfulness/downstream rates).
• ledger/reviews/ for human review decisions that promote or reject learning events.

Authority tiers are ordered A0 source truth, A1 reviewed OKF, A2 reviewed trace summaries, A3 observations, A4 hypotheses, A5 vector hints. Vectors are deliberately not implemented in this tranche; retrieval score objects include null vector fields for future compatibility only.

The optional MCP server is available through the mcp extra and exposes read-only knowledge_* tools for search, resolve, claims, context packs, deployment pins, endpoint schemas, policy decisions, runbooks, conflicts, and staleness checks. It supports stdio for local clients and loopback HTTP transports (streamable-http at /mcp or legacy sse at /sse) for deployment on the loop VM. The HTTP server also exposes GET /health.

Useful read-only commands:

uv run hyrule-knowledge resolve generated/services/hyrule-cloud
uv run hyrule-knowledge claims --subject service:hyrule-cloud --authority-min A0
uv run hyrule-knowledge neighborhood generated/services/hyrule-cloud --depth 2
uv run hyrule-knowledge endpoint-schema POST /v1/vm/create
uv run hyrule-knowledge deployment-pins hyrule-cloud
uv run hyrule-knowledge policy-decision --actor engineering_loop --action knowledge.search
uv run hyrule-knowledge ledger --write
uv run hyrule-knowledge ledger import /path/to/*.learning-event.json
uv run hyrule-knowledge ledger --list
uv run hyrule-knowledge ledger --review engineering_loop:fixture-run-summary --promotion-kind summary
uv run hyrule-knowledge ledger promote-pr engineering_loop:fixture-run-summary --reviewer svag --promotion-kind summary --rationale "reviewed"
uv run hyrule-knowledge ledger lifecycle --write
uv run --extra mcp hyrule-knowledge mcp --transport stdio
uv run --extra mcp hyrule-knowledge mcp --transport streamable-http --host 127.0.0.1 --port 8767

Containerized loopback deployment uses the checked-in Dockerfile:

docker build -t as215932/knowledge-mcp:local .
docker run --rm -p 127.0.0.1:8767:8767 as215932/knowledge-mcp:local
curl http://127.0.0.1:8767/health