feat: add GitHub merge queue automation with dashboard UI#72
Open
Ziinc wants to merge 19 commits into
Open
Conversation
- supabase/migrations/003_merge_queue.sql: tables for github_app_installations, github_repositories, merge_queue_configs, merge_queues, merge_queue_entries, and ci_runs; RLS policies; RPCs link_github_installation and get_repo_queue_status - supabase/functions/github-webhook/: Deno edge function that handles installation, pull_request (label-triggered enqueue/dequeue/sync), and check_suite webhooks; HMAC-SHA256 signature verification; GitHub App JWT auth via RS256; parallel merge queue lanes with optimistic batching and bisect mode on CI failure - web/src/pages/integrations/github/callback.tsx: post-install OAuth callback that calls link_github_installation to bind the app to the logged-in user - web/src/pages/dashboard.tsx: integrations tab now shows GitHub App status, Install button, per-repo expandable config panels (target branch, required checks, parallel lanes, batch size, merge method, trigger label), and live queue status via get_repo_queue_status RPC - web/src/lib/constants.ts: GITHUB_APP_NAME and GITHUB_APP_INSTALL_URL constants
- Rust command `get_git_remote_url`: parses .git/config to extract GitHub owner/repo from origin remote (SSH and HTTPS forms) - `src/hooks/useMergeQueueStatus`: React Query hooks for fetching per-workspace queue status via Supabase RPC and invoking the enqueue-workspace edge function to add/remove from the queue - ShowWorkspace: "Add to Queue" / "Remove from Queue" button in the action bar; shows live queue state (position, testing, failed) - WorkspaceSidebarItem: accepts optional queueStatus prop and renders a colored dot indicator (yellow=queued, blue=testing, green=merged) - WorkspaceSidebar: fetches all active queue statuses for the repo once per 30s via get_repo_branch_queue_statuses RPC and distributes to each sidebar item - Supabase: migration 004 adds branch_name column + workspace queue RPCs; enqueue-workspace edge function applies/removes the trigger label via GitHub API; queue-processor updated to store branch_name
Contributor
|
🚀 Web preview: https://preview-72.treq-9zy.pages.dev |
Ziinc
changed the title
- Fix addToast calls: use type: "success"/"error" not variant - Run biome format on all changed files - Run cargo fmt on github.rs - Generate missing ast-grep snapshots for no-e2e-* rules (these were introduced without their snapshot files and blocked CI on any PR touching src/)
Adds a `get_pr_info_via_gh` Tauri command that runs `gh pr view` to retrieve PR metadata (number, title, state, URL, base/head refs, merge state status) without requiring the Supabase edge function. The `usePrInfoViaGh` hook caches the result and feeds it to `useEnqueueWorkspace` as a pre-flight check: if gh reports the branch has no open PR, enqueue is rejected early with a clear error message instead of letting the edge function fail silently.
Rust (src-tauri/src/github.rs): - Extract github logic into a public treq_lib module so it's testable without GTK/Tauri deps and reachable from the NAPI dispatch layer - 14 inline unit tests covering parse_github_remote (SSH, HTTPS, invalid), get_git_remote_url_impl (ssh, https, gitlab, missing dir, non-origin), and get_pr_info_via_gh_impl (success JSON, non-zero exit, bad binary) using a fake gh shell script to avoid requiring a real gh auth session JS (test/integration/useMergeQueueStatus.test.ts): - useGitRemoteInfo: real NAPI against temp git repos with real .git/config - usePrInfoViaGh: verifies the hook never surfaces an error state even when gh is not authenticated - useEnqueueWorkspace: spies on getPrInfoViaGh to inject controlled PrInfo so the MERGED-state pre-flight guard and dequeue bypass are exercisable without a real gh session; supabase edge function is always mocked NAPI dispatch (crates/treq-napi/src/dispatch.rs): - Add get_git_remote_url and get_pr_info_via_gh so the command-consistency integration test continues to pass CI (.github/workflows/ci.yml): - Install gh CLI on both ubuntu-22.04 (via official apt source) and macOS (via brew) so the fake-gh Rust tests and real-gh integration paths work - Add test/** to paths filter so CI runs when test files change
- Add GitHub icon button to WorkspaceSidebar that navigates to github view - Add "github" ViewMode to Dashboard with GitHubPanel overlay - GitHubPanel: split-pane layout with Issues/PRs tabs and open/closed/all filter - Issue CRUD: list, view with comments, create, add comment, close, reopen - PR CRUD: list, view with comments, create, add comment, close, reopen - 12 new Rust impl functions in github.rs calling gh CLI with --repo flag - 12 new Tauri commands in commands/github.rs wired into lib.rs invoke handler - New TypeScript types: GhLabel, GhAuthor, GhIssueComment, GhIssue, GhPullRequest - Split into github-panel/ subdirectory (shared.tsx, IssueDetail.tsx, PrDetail.tsx) to stay within the 500-line component limit
…ueue leases - Enable pgmq and create merge_queue_commands + merge_queue_dead_letters (durable basic queues; browser roles have no access) - Service-role-only SECURITY DEFINER wrappers for send/read/archive/delete/ set_vt/metrics so Edge Functions can use PGMQ without exposing the schema - github_webhook_receipts for GitHub delivery deduplication and audit - merge_queue_command_executions with a partial unique index enforcing one success per operation_key - merge_queue_execution_leases + acquire/renew/release RPCs for per-queue single-writer orchestration across GitHub API calls - ci_run_entries join table replacing ci_runs.entry_ids as source of truth (array still written for compatibility reads) - get_or_create_merge_queue and enqueue_merge_queue_entry RPCs for atomic queue creation and position assignment - reserve_next_merge_queue_lane RPC for transactional lane reservation - Observability views for stuck CI runs and failed commands
…ared modules New supabase/functions/_shared/merge-queue/ decomposing the former queue-processor into explicit layers: - messages.ts: versioned discriminated-union command contracts and operation-key builders (enqueue/start-lane/merge-pr/delete-branch) - schemas.ts: runtime validation; unknown versions/shapes are terminal - errors.ts: RetryableError/TerminalError, error classification from GitHub/network failures, bounded exponential backoff schedule - state-machine.ts: pure entry/CI-run transitions, required-check evaluation (neutral/skipped are not success), merge precondition checks against the tested head SHA, lane failure policy - scheduler.ts: pure lane capacity/batch/base planning, deterministic test branch naming, entry-set hashing - github-adapter.ts: all GitHub side effects behind an interface; merge-PR forwards the expected head SHA, branch delete treats not-found as success, comments carry idempotency markers - commands-queue.ts: PGMQ producer/consumer plumbing via the service-role wrappers (archive on success, set_vt reschedule) - leases.ts / idempotency.ts: per-queue execution leases, command claims and durable operation-key records - queue-repository.ts / ci-run-repository.ts: domain persistence, ci_run_entries as source of truth for lane membership - command-handler.ts: per-command orchestration (enqueue, dequeue, drive, ci.completed with chained lane merging, installation.sync) Also adds a 'merged' ci_run_status enum value so lanes whose entries have landed no longer block higher lanes.
The webhook function is now ingest-only: - Requires GITHUB_WEBHOOK_SECRET; unsigned webhooks are never accepted - Verifies the HMAC signature before JSON parsing - Persists a github_webhook_receipts row per delivery: exact redeliveries of accepted events return 200 without reprocessing, a reused delivery id with a different payload hash is rejected 409 and logged as a security event, and failed ingestions are safely reprocessed on GitHub's redelivery - Upserts directly observable installation/repository facts (and no longer auto-links installations to users — linking is reserved for the authenticated intent flow) - Publishes entry.enqueue / entry.dequeue / ci.completed / installation.sync commands to PGMQ and returns 202 - Queue configuration is resolved by the PR's actual base branch - Optionally nudges the worker fire-and-forget; cron remains the recovery mechanism Removes the synchronous queue-processor and the recursive processing paths that ran inside webhook handling. enqueue-workspace now imports GitHub App auth from the shared adapter. Adds per-function verify_jwt config (webhook is signature-authenticated, worker/reconciler require JWTs).
…lows New merge-queue-worker Edge Function consuming merge_queue_commands: - Reads a bounded batch (5) with a 120s visibility timeout - Validates messages against the versioned schema; malformed or unknown-version messages dead-letter immediately - Claims a merge_queue_command_executions row per command; commands that already succeeded (or logical operations already satisfied via operation key) are archived without re-execution - Acquires the per-queue execution lease before mutating queue state so concurrent workers can progress different queues but never the same one - Successful messages are archived (not deleted) for auditability - Retryable failures reschedule the message with bounded exponential backoff (15s/60s/5m/15m) via set_vt; exhausted or terminal commands are published to merge_queue_dead_letters with a sanitised error envelope - Structured logs carry command_id, pgmq_message_id, github_delivery_id, queue_id, worker_id, attempt, duration and outcome Correctness does not depend on immediate invocation: schedule this function via cron; the webhook nudge is only a latency optimisation.
…space The trigger label and enabled flag were looked up with only repo_id, which breaks (multiple rows) or silently picks the wrong config once a repository has queue configurations for more than one target branch. The open PR for the workspace branch is now resolved first and its base ref keys the config lookup. Label manipulation remains the single ingestion path — the resulting webhook publishes the enqueue command.
merge-queue-reconciler (run every 5-15 minutes via cron) publishes corrective commands instead of performing GitHub side effects: - resets entries stuck in 'testing' with no active CI run back to queued - fails CI runs stuck beyond a 2h timeout by publishing ci.completed with a timed_out conclusion - deletes expired execution leases - publishes queue.drive for every queue that still has queued or merging entries, so capacity is used even if a drive command was lost
Replaces the browser-callable link_github_installation RPC (which let any authenticated user claim any installation_id) with a verified intent flow: - github_install_intents table stores only the SHA-256 hash of a single-use state with a 15-minute expiry (service-role only, RLS on) - create-github-install-intent mints 256-bit state for the signed-in user before redirecting to GitHub's install page - complete-github-installation atomically consumes the intent (single use, unexpired, same user), verifies the installation against GitHub with app-level auth, and links it server-side - dashboard install button now requests an intent and appends the state to the install URL; the callback page posts installation_id + state to the completion function instead of calling the dropped RPC - returning from 'manage access' without a state no longer attempts any linking
…and messages 49 pure-function tests over the shared merge-queue modules: - message schema versioning: valid commands per type, unknown version/ type rejection, missing-field and enum validation, operation-key derivation (queue.drive intentionally keyless) - state machine: entry lifecycle transitions, required-check evaluation (pending on missing/incomplete checks, neutral/skipped not success by default, latest re-run wins), suite-conclusion fallback, obsolete ci.completed detection, merge precondition violations (closed PR, retargeted base, moved head), lane failure/bisect policy - scheduler: bisect batch sizing, lane capacity, deterministic batch composition and lane numbering, optimistic base selection, lane merge ordering and chain-merging, test branch naming (current + legacy), order-insensitive entry-set hashing - retry policy: explicit and inferred error classification (429/5xx/rate limit retryable, other 4xx terminal, network retryable), bounded backoff schedule and attempt exhaustion
…ions Covers the PGMQ command flow, identity/concurrency model, merge safety invariants, retry/dead-letter policy, deployment steps, cron setup for the worker and reconciler via pg_cron + pg_net + Vault, observability surfaces, dead-letter replay and the security posture.
CI was failing on two pre-existing issues from the GitHub integration panel work: - crates/treq-napi/src/dispatch.rs was missing match arms for all 12 gh_* Tauri commands (list/view/create/comment/close/reopen for issues and PRs), so command-consistency.test.ts's invoke()/dispatch parity check failed. Added arms mirroring src-tauri/src/commands/ github.rs exactly, plus a local gh_bin() helper. - test/integration/useMergeQueueStatus.test.ts referenced mockEdgeFn inside a vi.mock() factory, but vi.mock is hoisted above the top-level const declaration, causing "Cannot access 'mockEdgeFn' before initialization". Moved the fn into vi.hoisted().
The vi.hoisted() fix from the previous commit surfaced a latent bug in these tests: waitFor(() => result.current.remoteInfo != null) never throws when the condition is false, so testing-library's waitFor resolved on the very first (unmet) check instead of polling until the query settled. Tests then called mutateAsync() before remoteInfo had loaded and hit the hook's own "Repository or branch not detected" guard. Switched to waitFor(() => expect(...).toBeTruthy()), which throws (and so actually retries) until the assertion holds. Verified locally by building the treq-napi addon and running the full suite.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a complete GitHub merge queue system with automated parallel CI testing and merging. It includes a web dashboard for configuration, Supabase backend infrastructure, GitHub webhook handlers, and desktop app integration for workspace-level queue status.
Key Changes
Web Dashboard (
web/src/pages/dashboard.tsx)Backend Infrastructure
supabase/migrations/003_merge_queue.sql,004_workspace_queue.sql):github_app_installations- tracks GitHub App installations per user/orggithub_repositories- synced repos from GitHubmerge_queue_configs- per-repo, per-branch configurationmerge_queues- active queue state with pause/bisect mode supportmerge_queue_entries- individual PR entries with status trackingci_runs- parallel CI test lanes with entry batchingQueue Processing Engine (
supabase/functions/github-webhook/queue-processor.ts)GitHub Integration
webhook-handlers.ts):github-client.ts):Edge Functions
enqueue-workspace- API endpoint for adding/removing PRs from queue via label manipulationweb/src/pages/integrations/github/callback.tsx) - OAuth flow completion and installation linkingDesktop App Integration
src-tauri/src/commands/github.rs) - Parse git remote URLs to identify repossrc/hooks/useMergeQueueStatus.ts) - Query queue status and enqueue/dequeue operationsConfiguration
GITHUB_APP_INSTALL_URLconstant for app installation linkNotable Implementation Details
https://claude.ai/code/session_01SJinHrSBMhX9HmdGiEMva4