ci: insight-sync workflow (Sync-now target) — collector → reviewed ledger PR

[Svaag]

Context

The dispatch target for the Observatory "Sync now" button (agentic-observatory#12), and a manual complement to the nightly on-box sync (network-operations#428). workflow_dispatch runs hyrule-knowledge insights sync and opens a reviewed PR with the ledger/insights/ diff — the durable insight ledger never gets an ad-hoc write.

Change

.github/workflows/insight-sync.yml: checkout → uv sync → insights sync --collector-url http://[loop]:8770 → validate (okf/quality/export/scan-secrets) → peter-evans/create-pull-request on a bot/knowledge-refresh/insight-sync-* branch. Because the diff touches ledger/, the auto-merge allowlist blocks auto-merge → it always opens for human review (correct — operator labels drive gate relaxation).

⚠️ Runner reachability — needs a firewall decision

The collector (http://[loop]:8770) is firewalled to loop/noc/mon; GitHub-hosted runners can't reach it, so this runs on the self-hosted hyrule-infra runner (on the ci VM). That runner is not in the collector allowlist either — a ci → collector:8770 firewall rule (network-operations loop.yml) is required before the sync step can connect. Until then the workflow fails at the sync step.

This is a small security-surface expansion (read-only collector access for a trusted infra runner), so I've left it as your call rather than bundling it. Want me to add that firewall rule? Alternatively the nightly on-box job (#428) remains the primary sync path and the button can wait.

🤖 Generated with Claude Code

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🏅 Score: 75

🧪 No relevant tests

🔒 Security concerns Yes. The collector URL uses HTTP without TLS, exposing data to potential interception on the internal network. Additionally, the fallback to GITHUB_TOKEN with broad contents:</strong><br> write and pull-requests: write permissions widens the attack surface if the PAT is not set. Consider enforcing HTTPS for the collector and removing the fallback token or limiting its scope.

⚡ Recommended focus areas for review Unencrypted collector call The collector URL uses http:// (no TLS). Although the network is firewalled, any attacker with access to the path between the runner and the collector could read or tamper with the insight stream. Use HTTPS or at minimum enforce TLS on the collector endpoint. HYRULE_KNOWLEDGE_LOOP_INSIGHT_COLLECTOR_URL: http://[2a0c:b641:b50:2::f0]:8770 Broad fallback token permissions When KNOWLEDGE_GH_TOKEN secret is unset, the fallback GITHUB_TOKEN (with contents: write and pull-requests: write) is used. This grants the running workflow wide write access to the repository. Consider making the PAT mandatory and removing the fallback, or scoping the GITHUB_TOKEN permissions more tightly. FALLBACK_GH_TOKEN: ${{ github.token }} Missing deterministic check on sync output The workflow runs validate, quality --check, export --check, and scan-secrets but does not verify that the insight sync output is byte-stable or that the ledger diff is deterministic. If the collector returns non-deterministic data (e.g., wall-clock timestamps, unordered keys), the generated PR could churn unnecessarily. Add a step to compare the new ledger content against a known-good baseline or run a --check variant of the sync command. - name: Sync collector insights into the ledger run: | uv run hyrule-knowledge insights sync \ --collector-url "$HYRULE_KNOWLEDGE_LOOP_INSIGHT_COLLECTOR_URL"

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Security	Expand secret scanning to all relevant directories The scan-secrets command is missing the ledger, evals, and schema directories. Since the sync step writes to ledger/insights/, those files must be scanned for secrets to prevent accidental exposure. Add these paths to the command. .github/workflows/insight-sync.yml [55] -uv run hyrule-knowledge scan-secrets okf exports reports +uv run hyrule-knowledge scan-secrets okf exports reports ledger evals schema Suggestion importance[1-10]: 7 __ Why: The suggestion legitimately expands secret scanning coverage to include newly synced directories (ledger, evals, schema), addressing a security gap. The improvement is accurate and contextually relevant, though it assumes these directories exist and contain sensitive data. Score is moderate because the omission is not a critical bug but a proactive hardening step.	Medium

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3afc8f87e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Pass the PAT into create-pull-request

This job sets GH_TOKEN from KNOWLEDGE_GH_TOKEN, but peter-evans/create-pull-request does not consume that environment variable unless it is supplied as the token input; the action metadata defaults token to ${{ github.token }} (see https://raw.githubusercontent.com/peter-evans/create-pull-request/v7/action.yml). With the default token, GitHub's documented recursion guard prevents the PR from firing this repo's on: pull_request validation workflow, so insight-sync PRs can be left without the full ledger/eval/secret-scan checks or be blocked by missing required checks. Pass the selected PAT as with.token here so the generated PR gets the normal PR checks.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Regenerate quality reports after syncing insights

When this dispatch imports ledger insights with OKF evidence refs, insights sync rewrites exports/grounding but does not update reports/coverage.json; quality --check then recomputes the production-stream grounding warnings and can fail with stale_reports before the create-pull-request step ever runs. This especially affects the first real ledger/insights import, because quality only enables never-cited grounding warnings once a production ledger file exists, so run quality --write before checking/committing the refreshed reports.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Scan the synced ledger for secrets

This workflow commits collector-sourced records under ledger/insights/, but the pre-PR secret scan only covers okf, exports, and reports, so the repo redaction regexes for values such as GitHub PATs are never applied to the newly imported ledger files. The in-code sanitizer only checks a narrower set of keys/value patterns before appending rows, so include ledger in this scan to fail the sync before opening a PR with leaked collector data.

Useful? React with 👍 / 👎.

[Svaag]

This is the missing companion to the Observatory "Sync now" button — the config in agentic-observatory#12 points exactly at AS215932/knowledge :: insight-sync.yml on main, so filename/ref line up. Runner label, the loop-host IPv6, and the unauthenticated-reads note all look right, and routing the durable write through a reviewed PR (ledger outside the auto-merge allowlist) is exactly the intended shape.

Heads-up: I'd independently started building the same workflow from the Observatory side and then found this PR — folding my notes in here instead of opening a duplicate. Two things before it'll reliably open a PR on a real sync — one is a hard blocker on the first run, one is defense-in-depth.

[Svaag]

This will fail the workflow on the first real sync, before any PR is opened. quality_check() recomputes evaluate_quality() live and compares warning_count/critical_count/concept_count against the committed reports/coverage.json. The grounding never_cited warnings only activate once ledger/insights/decisions/*.jsonl exists (production_stream = any(... glob('*.jsonl')) in quality.py).

Right now the repo has no committed ledger/insights/decisions/ and coverage.json has warning_count: 44 (computed with production_stream=False). So the first sync that imports any decisions creates that file → production_stream flips True → live warning_count jumps above 44 → count mismatch → stale_reports critical → quality --check exits 1 → the "Open PR" step never runs. insights sync regenerates exports but not quality reports, so nothing here refreshes them.

The nightly loop avoids this by running quality --write (in _run_generation_refresh) before validating. Suggest one of:

• run hyrule-knowledge quality --write (and re-export) before the checks, so the regenerated reports/ ride the PR (matches the nightly path), or
• drop the inline quality --check/export --check — the reviewed PR + validate.yml on merge already gate this.

Either way validate okf + secret scan are worth keeping inline.

[Svaag]

Minor / defense-in-depth: this scans okf exports reports but not ledger — which is exactly where the freshly-synced decisions + operator labels land. The canonical baseline in validate.yml scans okf exports reports evals ledger schema, and bot PRs created with the default GITHUB_TOKEN don't trigger validate.yml (GitHub's workflow loop-prevention), so this inline scan is the only secret gate on the new ledger records before a human sees them.

insight_sync.py already does per-record forbidden-key/secret-value scanning on import, so this is belt-and-suspenders — but adding ledger restores parity and costs nothing: scan-secrets okf exports reports ledger.