From 0422657f42dda8fb1dfd79ff44ba019f6d1c5243 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Tue, 16 Jun 2026 20:58:41 +0000 Subject: [PATCH 1/4] doc(runway): add workflow RFC for runway domain Runway is a consumer-only landing service that owns VCS operations (mergeability checking and landing) on behalf of SubmitQueue. This RFC describes the two independent queue flows (check and land), branch serialization via partition key, statelessness, and ownership model. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/index.md | 4 ++ doc/rfc/runway/workflow.md | 84 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+) create mode 100644 doc/rfc/runway/workflow.md diff --git a/doc/rfc/index.md b/doc/rfc/index.md index aa95dc19..e60d0180 100644 --- a/doc/rfc/index.md +++ b/doc/rfc/index.md @@ -17,3 +17,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting ## Stovepipe - [Stovepipe Workflow](stovepipe/workflow.md) - Post-merge trunk-validation pipeline: ingest trunk push events (webhook + fallback poll), batch since last green, build to validate, record per-commit health, bisect to the offending commit, hand off to a remediation extension + +## Runway + +- [Runway Workflow](runway/workflow.md) - Landing service: check request mergeability, land scored batches, publish results back to SubmitQueue diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md new file mode 100644 index 00000000..a3a86cf2 --- /dev/null +++ b/doc/rfc/runway/workflow.md @@ -0,0 +1,84 @@ +# Runway Workflow + +Runway is the landing service: it owns VCS operations — mergeability checking and landing (resolve, apply, push, finalize) — on behalf of SubmitQueue. Unlike SQ and Stovepipe, it is a consumer-only domain with no gateway and no proto API; work arrives via topic queues, results leave via topic queues. The orchestrator subscribes to two inbound topics (runway-check, runway-land) and publishes to two outbound topics (sq-check-result, sq-land-result). There are no internal hops between stages and no cycles — each controller consumes one inbound message and produces one outbound result. + +## Check and land + +The two queues operate at different granularities: + +- **check** is request-level. SubmitQueue publishes a check message to determine whether a request's changes can merge cleanly against the target branch. Runway performs a read-only trial merge and publishes per-change mergeability results back. + +- **land** is batch-level. SubmitQueue publishes a job message to land a batch. The job carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message must be self-contained). Runway pre-validates, lands, finalizes, and publishes a result with per-item outcomes back. + +These are independent input→output flows. Check can run without land ever running (SubmitQueue may skip external mergeability checks for some queues), and land does not depend on a prior check having passed through runway. + +## Branch serialization + +The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order — at most one check or land operation is in flight for any given branch at any time. + +This replaces the LandQueue extension from the synchronous design — no Redis, no MySQL queue table, no distributed lock. The tradeoff: serialization granularity is fixed at the partition key level. If pipelining becomes necessary (overlapping check N+1 with land N for the same branch), it can be implemented within the controllers without changing the queue topology. + +The outbound topics partition by SubmitQueue queue name (not repo/branch), matching SubmitQueue's existing fan-out model where state updates for the same queue must be serialized. + +## Workflow + +``` + ┌─────────────────────────────────────────────┐ + │ submitqueue orchestrator │ + │ │ + │ │ │ │ + └───────┼───────────────────────┼─────────────┘ + │ │ + Check (per request) Job (per batch) + │ │ + ▼ ▼ + [runway-check] [runway-land] + │ │ + check ctrl land ctrl + (read-only) (pre-validate + push) + │ │ + CheckResult Result + │ │ + ▼ ▼ + [sq-check-result] [sq-land-result] + │ │ + ▼ ▼ + ┌───────┼───────────────────────┼─────────────┐ + │ check-result ctrl land-result ctrl │ + │ (update request (update batch state, │ + │ mergeability) fan out to conclude) │ + │ │ + │ submitqueue orchestrator │ + └─────────────────────────────────────────────┘ +``` + +## Per-controller summary + +| Controller | In | Out | One-line role | +|---|---|---|---| +| **check** | Check | CheckResult → sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | +| **land** | Job | Result → sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | + +The check controller always publishes a result — even when all changes are mergeable — so SubmitQueue receives a definitive answer. On infrastructure error it nacks for retry. + +The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. Conflicts are expected outcomes, infrastructure errors are not — this matches the pattern established by SubmitQueue's merge controller. + +## Idempotency + +Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. The message queue provides at-least-once delivery; idempotent consumers handle duplicates. + +## DLQ reconciliation + +Runway does not implement DLQ reconciliation controllers. Unlike SubmitQueue and Stovepipe — where a stuck message leaves a request or commit in a non-terminal state forever — runway is stateless. A check or land message that exhausts retries and lands in the DLQ simply means SubmitQueue never receives a result for that request or batch. SubmitQueue's own timeout and retry logic handles the missing response (re-publishing the check or re-batching the requests). Adding DLQ controllers to runway would duplicate that recovery logic without adding safety, since there is no persistent state to reconcile. + +If runway later gains persistent state (e.g., a job store for observability), DLQ controllers should be added following the same pattern as SubmitQueue and Stovepipe. + +## Ownership by service + +### Orchestrator + +The orchestrator is the only service. It subscribes to two inbound topics (runway-check, runway-land), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (sq-check-result, sq-land-result). It owns no persistent data. + +### Shared: the messaging queue + +Runway communicates with SubmitQueue only through the messaging queue, consistent with the ownership model established by SubmitQueue and Stovepipe. The inbound topics are owned by runway; the outbound topics are owned by SubmitQueue. This keeps the dependency direction clean: runway depends on the shared entity package for message payloads, but not on SubmitQueue's internal state management. From 468b32a6fa0fa0b99aa71ddf567647791f7de092 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Tue, 16 Jun 2026 22:11:59 +0000 Subject: [PATCH 2/4] fix(rfc): clean up workflow diagram box-drawing characters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove dangling pipe characters and empty lines inside diagram boxes that render as strikethroughs in GitHub rich diff. Change connector characters from cross (┼) to tee (┬) at box edges. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/runway/workflow.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md index a3a86cf2..cf072a97 100644 --- a/doc/rfc/runway/workflow.md +++ b/doc/rfc/runway/workflow.md @@ -25,9 +25,7 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match ``` ┌─────────────────────────────────────────────┐ │ submitqueue orchestrator │ - │ │ - │ │ │ │ - └───────┼───────────────────────┼─────────────┘ + └───────┬───────────────────────┬─────────────┘ │ │ Check (per request) Job (per batch) │ │ @@ -43,11 +41,10 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match [sq-check-result] [sq-land-result] │ │ ▼ ▼ - ┌───────┼───────────────────────┼─────────────┐ + ┌───────┬───────────────────────┬─────────────┐ │ check-result ctrl land-result ctrl │ │ (update request (update batch state, │ │ mergeability) fan out to conclude) │ - │ │ │ submitqueue orchestrator │ └─────────────────────────────────────────────┘ ``` From 8292a9580292e41b2e125ecdbea71a335b6e80d3 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Tue, 16 Jun 2026 22:27:55 +0000 Subject: [PATCH 3/4] docs(rfc): revise runway workflow to match existing RFC style Remove historical context, implementation rationale, and defensive justifications. Match the factual tone and structure of the submitqueue and stovepipe workflow RFCs. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/runway/workflow.md | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md index cf072a97..61b960ac 100644 --- a/doc/rfc/runway/workflow.md +++ b/doc/rfc/runway/workflow.md @@ -1,24 +1,22 @@ # Runway Workflow -Runway is the landing service: it owns VCS operations — mergeability checking and landing (resolve, apply, push, finalize) — on behalf of SubmitQueue. Unlike SQ and Stovepipe, it is a consumer-only domain with no gateway and no proto API; work arrives via topic queues, results leave via topic queues. The orchestrator subscribes to two inbound topics (runway-check, runway-land) and publishes to two outbound topics (sq-check-result, sq-land-result). There are no internal hops between stages and no cycles — each controller consumes one inbound message and produces one outbound result. +Runway is the landing service: it owns VCS operations — mergeability checking and landing — on behalf of SubmitQueue. The orchestrator subscribes to two inbound topics (`runway-check`, `runway-land`) and publishes results to two outbound topics (`sq-check-result`, `sq-land-result`). It is a consumer-only service with no gateway; work arrives via topic queues and results leave via topic queues. ## Check and land The two queues operate at different granularities: -- **check** is request-level. SubmitQueue publishes a check message to determine whether a request's changes can merge cleanly against the target branch. Runway performs a read-only trial merge and publishes per-change mergeability results back. +- **check** is request-level. A check message carries a request's changes and merge strategy. Runway performs a read-only trial merge and publishes per-change mergeability results back. -- **land** is batch-level. SubmitQueue publishes a job message to land a batch. The job carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message must be self-contained). Runway pre-validates, lands, finalizes, and publishes a result with per-item outcomes back. +- **land** is batch-level. A job message carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message is self-contained). Runway pre-validates, lands, and publishes a result with per-item outcomes back. -These are independent input→output flows. Check can run without land ever running (SubmitQueue may skip external mergeability checks for some queues), and land does not depend on a prior check having passed through runway. +These are independent input-output flows. Check can run without land ever running, and land does not depend on a prior check. ## Branch serialization -The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order — at most one check or land operation is in flight for any given branch at any time. +The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order, so at most one check or land operation is in flight for any given branch at any time. -This replaces the LandQueue extension from the synchronous design — no Redis, no MySQL queue table, no distributed lock. The tradeoff: serialization granularity is fixed at the partition key level. If pipelining becomes necessary (overlapping check N+1 with land N for the same branch), it can be implemented within the controllers without changing the queue topology. - -The outbound topics partition by SubmitQueue queue name (not repo/branch), matching SubmitQueue's existing fan-out model where state updates for the same queue must be serialized. +The outbound topics partition by SubmitQueue queue name, matching SubmitQueue's fan-out model where state updates for the same queue are serialized. ## Workflow @@ -42,9 +40,9 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match │ │ ▼ ▼ ┌───────┬───────────────────────┬─────────────┐ - │ check-result ctrl land-result ctrl │ - │ (update request (update batch state, │ - │ mergeability) fan out to conclude) │ + │ check-result ctrl land-result ctrl │ + │ (update request (update batch state, │ + │ mergeability) fan out to conclude) │ │ submitqueue orchestrator │ └─────────────────────────────────────────────┘ ``` @@ -53,29 +51,23 @@ The outbound topics partition by SubmitQueue queue name (not repo/branch), match | Controller | In | Out | One-line role | |---|---|---|---| -| **check** | Check | CheckResult → sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | -| **land** | Job | Result → sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | +| **check** | Check | CheckResult -> sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | +| **land** | Job | Result -> sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | The check controller always publishes a result — even when all changes are mergeable — so SubmitQueue receives a definitive answer. On infrastructure error it nacks for retry. -The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. Conflicts are expected outcomes, infrastructure errors are not — this matches the pattern established by SubmitQueue's merge controller. +The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. ## Idempotency -Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. The message queue provides at-least-once delivery; idempotent consumers handle duplicates. - -## DLQ reconciliation - -Runway does not implement DLQ reconciliation controllers. Unlike SubmitQueue and Stovepipe — where a stuck message leaves a request or commit in a non-terminal state forever — runway is stateless. A check or land message that exhausts retries and lands in the DLQ simply means SubmitQueue never receives a result for that request or batch. SubmitQueue's own timeout and retry logic handles the missing response (re-publishing the check or re-batching the requests). Adding DLQ controllers to runway would duplicate that recovery logic without adding safety, since there is no persistent state to reconcile. - -If runway later gains persistent state (e.g., a job store for observability), DLQ controllers should be added following the same pattern as SubmitQueue and Stovepipe. +Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. ## Ownership by service ### Orchestrator -The orchestrator is the only service. It subscribes to two inbound topics (runway-check, runway-land), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (sq-check-result, sq-land-result). It owns no persistent data. +The orchestrator is the only service. It subscribes to two inbound topics (`runway-check`, `runway-land`), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (`sq-check-result`, `sq-land-result`). It owns no persistent data. ### Shared: the messaging queue -Runway communicates with SubmitQueue only through the messaging queue, consistent with the ownership model established by SubmitQueue and Stovepipe. The inbound topics are owned by runway; the outbound topics are owned by SubmitQueue. This keeps the dependency direction clean: runway depends on the shared entity package for message payloads, but not on SubmitQueue's internal state management. +Runway communicates with SubmitQueue only through the messaging queue. The inbound topics are owned by runway; the outbound topics are owned by SubmitQueue. From dafd04f22eaace131d48a1acb075372d72d0ba73 Mon Sep 17 00:00:00 2001 From: "kevin.new" Date: Thu, 18 Jun 2026 21:34:58 +0000 Subject: [PATCH 4/4] docs(rfc): update runway workflow for landed queue contract Update queue names, entity names, and controller names to match the merge contract that landed in #244: merge-conflict-checker/merger topic keys, MergeRequest/MergeResult payloads, unified contract serving both dry-run checks and committing merges. Co-Authored-By: Claude Opus 4.6 (1M context) --- doc/rfc/index.md | 2 +- doc/rfc/runway/workflow.md | 73 +++++++++++++++++++------------------- 2 files changed, 38 insertions(+), 37 deletions(-) diff --git a/doc/rfc/index.md b/doc/rfc/index.md index e60d0180..6654b39b 100644 --- a/doc/rfc/index.md +++ b/doc/rfc/index.md @@ -20,4 +20,4 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting ## Runway -- [Runway Workflow](runway/workflow.md) - Landing service: check request mergeability, land scored batches, publish results back to SubmitQueue +- [Runway Workflow](runway/workflow.md) - Landing service: merge-conflict checking and merging on behalf of SubmitQueue diff --git a/doc/rfc/runway/workflow.md b/doc/rfc/runway/workflow.md index 61b960ac..01357eb8 100644 --- a/doc/rfc/runway/workflow.md +++ b/doc/rfc/runway/workflow.md @@ -1,72 +1,73 @@ # Runway Workflow -Runway is the landing service: it owns VCS operations — mergeability checking and landing — on behalf of SubmitQueue. The orchestrator subscribes to two inbound topics (`runway-check`, `runway-land`) and publishes results to two outbound topics (`sq-check-result`, `sq-land-result`). It is a consumer-only service with no gateway; work arrives via topic queues and results leave via topic queues. +Runway is the landing service: it owns VCS operations — mergeability checking and landing — on behalf of SubmitQueue. The orchestrator subscribes to two inbound topics (`merge-conflict-checker`, `merger`) and publishes results to two outbound topics (`merge-conflict-checker-signal`, `merger-signal`). It is a consumer-only service with no gateway; work arrives via topic queues and results leave via topic queues. -## Check and land +## Merge-conflict check and merge The two queues operate at different granularities: -- **check** is request-level. A check message carries a request's changes and merge strategy. Runway performs a read-only trial merge and publishes per-change mergeability results back. +- **merge-conflict-check** is request-level. A merge request carries an ordered sequence of steps (changes + merge strategy). Runway performs a read-only trial merge and publishes per-step mergeability results back. -- **land** is batch-level. A job message carries the resolved content for each request in the batch (runway has no access to SubmitQueue's request store, so the message is self-contained). Runway pre-validates, lands, and publishes a result with per-item outcomes back. +- **merge** is batch-level. A merge request carries the same payload but Runway commits the result and reports the revisions it produced (per-step output IDs). -These are independent input-output flows. Check can run without land ever running, and land does not depend on a prior check. +These are independent input-output flows. A merge-conflict check can run without a merge ever running, and a merge does not depend on a prior check. ## Branch serialization -The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order, so at most one check or land operation is in flight for any given branch at any time. +The partition key `repo/target` on both inbound topics serializes all VCS operations for a given branch. The message queue delivers messages with the same partition key to the same consumer in order, so at most one merge-conflict check or merge operation is in flight for any given branch at any time. The outbound topics partition by SubmitQueue queue name, matching SubmitQueue's fan-out model where state updates for the same queue are serialized. ## Workflow ``` - ┌─────────────────────────────────────────────┐ - │ submitqueue orchestrator │ - └───────┬───────────────────────┬─────────────┘ - │ │ - Check (per request) Job (per batch) - │ │ - ▼ ▼ - [runway-check] [runway-land] - │ │ - check ctrl land ctrl - (read-only) (pre-validate + push) - │ │ - CheckResult Result - │ │ - ▼ ▼ - [sq-check-result] [sq-land-result] - │ │ - ▼ ▼ - ┌───────┬───────────────────────┬─────────────┐ - │ check-result ctrl land-result ctrl │ - │ (update request (update batch state, │ - │ mergeability) fan out to conclude) │ - │ submitqueue orchestrator │ - └─────────────────────────────────────────────┘ + ┌─────────────────────────────────────────────────────┐ + │ submitqueue orchestrator │ + └──────────┬───────────────────────────┬──────────────┘ + │ │ + MergeRequest (dry run) MergeRequest (commit) + │ │ + ▼ ▼ + [merge-conflict-checker] [merger] + │ │ + merge-conflict-check ctrl merge ctrl + (read-only) (apply + commit) + │ │ + MergeResult MergeResult + │ │ + ▼ ▼ + [merge-conflict-checker-signal] [merger-signal] + │ │ + ▼ ▼ + ┌──────────┬───────────────────────────┬──────────────┐ + │ merge-conflict-check- merge-signal ctrl │ + │ signal ctrl (update batch state, │ + │ (update request fan out to conclude) │ + │ mergeability) │ + │ submitqueue orchestrator │ + └─────────────────────────────────────────────────────┘ ``` ## Per-controller summary | Controller | In | Out | One-line role | |---|---|---|---| -| **check** | Check | CheckResult -> sq-check-result | Check mergeability of a request's changes against the target branch (read-only) | -| **land** | Job | Result -> sq-land-result | Pre-validate, land, and finalize a batch's changes on the target branch | +| **merge-conflict-check** | MergeRequest | MergeResult -> merge-conflict-checker-signal | Dry-run merge: check mergeability of ordered steps against the target branch (read-only) | +| **merge** | MergeRequest | MergeResult -> merger-signal | Apply, commit, and report per-step output IDs | -The check controller always publishes a result — even when all changes are mergeable — so SubmitQueue receives a definitive answer. On infrastructure error it nacks for retry. +The merge-conflict-check controller always publishes a result — even when all steps are mergeable — so SubmitQueue receives a definitive answer. On infrastructure error it nacks for retry. -The land controller publishes a conflict result (and acks) when pre-validation detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-item outcomes (commit SHAs, whether new commits were produced) so SubmitQueue can update its request state. +The merge controller publishes a conflict result (and acks) when the merge detects a conflict; SubmitQueue handles rebatching. On infrastructure error it nacks for retry. On success it publishes per-step outcomes (output IDs of the revisions produced) so SubmitQueue can update its request state. ## Idempotency -Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: land detects already-pushed changes (commit SHAs reachable from HEAD) and treats them as already-landed; closing an already-closed PR is a no-op. Check is read-only and naturally idempotent. +Runway has no persistent state — no request store, no job store, no database. Idempotency is achieved through the VCS contract: merge detects already-pushed changes (revisions reachable from HEAD) and treats them as already-landed. Merge-conflict check is read-only and naturally idempotent. ## Ownership by service ### Orchestrator -The orchestrator is the only service. It subscribes to two inbound topics (`runway-check`, `runway-land`), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (`sq-check-result`, `sq-land-result`). It owns no persistent data. +The orchestrator is the only service. It subscribes to two inbound topics (`merge-conflict-checker`, `merger`), performs VCS operations through a pluggable extension, and publishes results to two outbound topics (`merge-conflict-checker-signal`, `merger-signal`). It owns no persistent data. ### Shared: the messaging queue