Skip to content

ci: insight-sync workflow (Sync-now target) — collector → reviewed ledger PR #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Svaag wants to merge 1 commit into
base: main
Choose a base branch
from
Open

ci: insight-sync workflow (Sync-now target) — collector → reviewed ledger PR #37

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/workflows/insight-sync.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: insight-sync

# On-demand durable sync of the insight stream: pull the collector's decisions
# and operator labels into ledger/insights/ and open a REVIEWED PR. This is the
# target of the Observatory "Sync now" button (agentic-observatory) and a
# manual complement to the nightly on-box knowledge-loop sync
# (network-operations#428).
#
# Runs on the self-hosted hyrule-infra runner because the collector
# (http://[loop]:8770) is firewalled to loop/noc/mon; GitHub-hosted runners
# cannot reach it. REQUIRES a ci -> collector:8770 firewall rule in
# network-operations (loop.yml) — until that lands the sync step will fail to
# connect. Writes nothing directly: the ledger change lands as a reviewed PR.

on:
workflow_dispatch:

permissions:
contents: write
pull-requests: write

concurrency:
group: insight-sync
cancel-in-progress: false

jobs:
insight-sync:
runs-on: [self-hosted, linux, x64, hyrule-infra]
env:
GH_TOKEN: ${{ secrets.KNOWLEDGE_GH_TOKEN }}
FALLBACK_GH_TOKEN: ${{ github.token }}
# GET /v1/insights is unauthenticated (reads are open); no token needed.
HYRULE_KNOWLEDGE_LOOP_INSIGHT_COLLECTOR_URL: http://[2a0c:b641:b50:2::f0]:8770
steps:
- name: Select GitHub token
run: |
if [ -z "${GH_TOKEN:-}" ]; then
echo "GH_TOKEN=${FALLBACK_GH_TOKEN}" >> "$GITHUB_ENV"
fi
- uses: actions/checkout@v6
- uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install dependencies
run: uv sync --group dev --frozen
- name: Sync collector insights into the ledger
run: |
uv run hyrule-knowledge insights sync \
--collector-url "$HYRULE_KNOWLEDGE_LOOP_INSIGHT_COLLECTOR_URL"
- name: Validate the refreshed ledger + exports
run: |
uv run hyrule-knowledge validate okf
uv run hyrule-knowledge quality --check

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Regenerate quality reports after syncing insights

When this dispatch imports ledger insights with OKF evidence refs, insights sync rewrites exports/grounding but does not update reports/coverage.json; quality --check then recomputes the production-stream grounding warnings and can fail with stale_reports before the create-pull-request step ever runs. This especially affects the first real ledger/insights import, because quality only enables never-cited grounding warnings once a production ledger file exists, so run quality --write before checking/committing the refreshed reports.

Useful? React with 👍 / 👎.

@Svaag Svaag Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail the workflow on the first real sync, before any PR is opened. quality_check() recomputes evaluate_quality() live and compares warning_count/critical_count/concept_count against the committed reports/coverage.json. The grounding never_cited warnings only activate once ledger/insights/decisions/*.jsonl exists (production_stream = any(... glob('*.jsonl')) in quality.py).

Right now the repo has no committed ledger/insights/decisions/ and coverage.json has warning_count: 44 (computed with production_stream=False). So the first sync that imports any decisions creates that file → production_stream flips True → live warning_count jumps above 44 → count mismatch → stale_reports critical → quality --check exits 1 → the "Open PR" step never runs. insights sync regenerates exports but not quality reports, so nothing here refreshes them.

The nightly loop avoids this by running quality --write (in _run_generation_refresh) before validating. Suggest one of:

  • run hyrule-knowledge quality --write (and re-export) before the checks, so the regenerated reports/ ride the PR (matches the nightly path), or
  • drop the inline quality --check/export --check — the reviewed PR + validate.yml on merge already gate this.

Either way validate okf + secret scan are worth keeping inline.

uv run hyrule-knowledge export --check
uv run hyrule-knowledge scan-secrets okf exports reports

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scan the synced ledger for secrets

This workflow commits collector-sourced records under ledger/insights/, but the pre-PR secret scan only covers okf, exports, and reports, so the repo redaction regexes for values such as GitHub PATs are never applied to the newly imported ledger files. The in-code sanitizer only checks a narrower set of keys/value patterns before appending rows, so include ledger in this scan to fail the sync before opening a PR with leaked collector data.

Useful? React with 👍 / 👎.

@Svaag Svaag Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor / defense-in-depth: this scans okf exports reports but not ledger — which is exactly where the freshly-synced decisions + operator labels land. The canonical baseline in validate.yml scans okf exports reports evals ledger schema, and bot PRs created with the default GITHUB_TOKEN don't trigger validate.yml (GitHub's workflow loop-prevention), so this inline scan is the only secret gate on the new ledger records before a human sees them.

insight_sync.py already does per-record forbidden-key/secret-value scanning on import, so this is belt-and-suspenders — but adding ledger restores parity and costs nothing: scan-secrets okf exports reports ledger.

- name: Open reviewed insight-sync pull request
uses: peter-evans/create-pull-request@v7

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass the PAT into create-pull-request

This job sets GH_TOKEN from KNOWLEDGE_GH_TOKEN, but peter-evans/create-pull-request does not consume that environment variable unless it is supplied as the token input; the action metadata defaults token to ${{ github.token }} (see https://raw.githubusercontent.com/peter-evans/create-pull-request/v7/action.yml). With the default token, GitHub's documented recursion guard prevents the PR from firing this repo's on: pull_request validation workflow, so insight-sync PRs can be left without the full ledger/eval/secret-scan checks or be blocked by missing required checks. Pass the selected PAT as with.token here so the generated PR gets the normal PR checks.

Useful? React with 👍 / 👎.

with:
branch: bot/knowledge-refresh/insight-sync-${{ github.run_id }}
title: "knowledge: insight-sync ${{ github.run_id }}"
commit-message: "knowledge: sync collector insights into ledger"
body: |
On-demand durable sync of the insight decision/label stream from the
collector into `ledger/insights/`. Deterministic; review the ledger
diff (decisions + operator labels) before merge.
labels: knowledge, automated
delete-branch: true