Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/release-management/atr-release-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,30 @@ tired human at 2am, enforced the mechanics.
> Coordinate on `dev@tooling.apache.org` / `#apache-trusted-releases`
> on ASF Slack before cutting a *real* release through ATR.

> [!IMPORTANT]
> **Hybrid mode (Magpie's current stance): SVN hosts, ATR votes.**
> Because ATR is still alpha, Magpie does **not** yet let it host or
> publish releases — but it *does* use ATR's automated checks and vote
> administration. This is expressed as two independent config keys in
> [`release-management-config.md`](../../projects/magpie/release-management-config.md):
> `release_dist_backend = svnpubsub` **and** `release_vote_backend = atr`.
> Under this mode:
> - **Compose** — the signed artefacts are staged to SVN `dist/dev`
> (canonical download location) **and** uploaded to ATR so it runs the
> policy checks. Artefacts live in both places.
> - **Vote** — ATR sends the `[VOTE]` to `dev@` and tabulates, exactly as
> the *Vote* rows below describe. The `[VOTE]` body points voters at the
> SVN `dist/dev` URL for downloads.
> - **Finish** — **not done through ATR.** Promotion is `svn mv dist/dev →
> dist/release` per the [`svnpubsub` runbook](svn-release-runbook.md);
> ATR's Finish/publish is skipped until `release_dist_backend` itself
> becomes `atr`.
>
> So in hybrid mode the **Compose** and **Vote** rows of the table below
> apply, but **Finish** falls back to the SVN runbook. The rest of this
> document describes the *full* ATR flow (`release_dist_backend = atr`),
> which is the post-ratification target.

## The three ATR phases vs the 14-step lifecycle

ATR organises a release into three phases. They map onto the
Expand Down
49 changes: 43 additions & 6 deletions projects/_template/release-management-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,15 @@ ship in the same adopter directory and are referenced from here:

## Backends

Three switches select the backend each skill in the family emits
Four switches select the backend each skill in the family emits
commands against. See
[`process.md` § Adopter backends](../../docs/release-management/process.md#adopter-backends)
for the dimensions.

| Key | Value | Allowed values |
|---|---|---|
| `release_dist_backend` | `svnpubsub` | `svnpubsub`, `atr`, `github-releases`, `s3`, `self-hosted` |
| `release_vote_backend` | `manual` | `manual`, `atr` |
| `release_approval_mechanism` | `dev-list-vote` | `dev-list-vote`, `github-discussion`, `pr-approval`, `maintainer-roster` |
| `release_announce_backend` | `announce-list` | `announce-list`, `github-release-notes`, `site-post`, `discord-channel` |

Expand All @@ -86,17 +87,34 @@ and `announce-list` (mandatory per
`release-vote-tally` and `release-announce-draft` refuse to run an
ASF TLP release against any other value.

`atr` selects the **[Apache Trusted Releases](https://release-test.apache.org/)**
platform: the RM composes a signed candidate in ATR (which runs the
signature/checksum/license/notice/source-header checks on upload),
ATR drives the `dev@` `[VOTE]` and tabulation, and *finishing*
`release_dist_backend = atr` selects the
**[Apache Trusted Releases](https://release-test.apache.org/)** platform
for the *whole* flow: the RM composes a signed candidate in ATR (which
runs the signature/checksum/license/notice/source-header checks on
upload), ATR drives the `dev@` `[VOTE]` and tabulation, and *finishing*
publishes to `dist.apache.org`. It is an ASF-only backend and an
alternative to `svnpubsub` for the same `dev-list-vote` /
`announce-list` approval and announce mechanisms. See the
[ATR release runbook](../../docs/release-management/atr-release-runbook.md)
for the phase-by-phase flow. ATR is in alpha; until a PMC ratifies it,
`svnpubsub` remains the ratified default.

`release_vote_backend` **decouples vote administration from artefact
hosting**, so an adopter can use ATR's automated checks + vote *without*
letting it host or publish the release yet:
- `manual` (default) — the RM sends the `[VOTE]` email and tallies
replies from the mail archive by hand (the classic `svnpubsub` flow).
- `atr` — the signed artefacts are *also* uploaded to ATR (Compose) so
it runs the policy checks and then **sends and tabulates** the `[VOTE]`.
Combine `release_vote_backend = atr` with `release_dist_backend =
svnpubsub` for the **hybrid** flow: SVN hosts (`dist/dev`) and promotes
(`svn mv` to `dist/release`), while ATR only checks and drives the vote
— ATR's Finish/publish is skipped. This is the recommended stance while
ATR is alpha (checks + vote automation are useful; hosting/publishing is
not yet trusted). `release_vote_backend` is ignored when
`release_dist_backend = atr` (ATR already owns the vote in the full
flow).

Non-ASF adopters set the values their workflow uses; the skills
emit backend-shaped paste-ready commands per
[spec § Per-skill specifications](../../docs/release-management/spec.md#per-skill-specifications).
Expand All @@ -108,14 +126,18 @@ The state-change boundaries are backend-independent.
|---|---|
| `release_dist_url_template` | `https://dist.apache.org/repos/dist/<bucket>/airflow/<version>/` |
| `archive_url_template` | `https://archive.apache.org/dist/airflow/` |
| `atr_platform_url` | *(set when `release_vote_backend = atr` or `release_dist_backend = atr`; ASF alpha host `https://release-test.apache.org/`, production `https://release.apache.org/`)* |
| `release_publish_command_template` | *(`svnpubsub` default; non-ASF adopters override with backend-specific command, e.g. `gh release upload <version> <artefacts>` for `github-releases`, `aws s3 cp --recursive <local> s3://<bucket>/<version>/` for `s3`)* |

`<bucket>` resolves to `dev` (staging) or `release` (promoted)
depending on the lifecycle step the skill is executing. The
`<bucket>` semantics are `svnpubsub`-shaped; backends that have no
staging area (`github-releases` draft, `s3` versioned prefix) use
the analogous draft/promote convention and document it in
`release_publish_command_template`.
`release_publish_command_template`. When `release_vote_backend = atr`
(with `release_dist_backend = svnpubsub`), the artefacts are staged to
the `dist/dev` URL **and** uploaded to `atr_platform_url` for the
checks + vote; promotion is still `svn mv` to the `release` bucket.

## Signing

Expand Down Expand Up @@ -160,6 +182,21 @@ shorten.
(e.g. require 5 binding +1 instead of 3). Attempts to weaken the
baseline are refused by `release-vote-tally`.

When `release_vote_backend = atr`, the same `vote_*` keys apply, but the
ATR platform *sends* the `[VOTE]` (from the uploaded candidate) and
*tabulates* the replies instead of the RM doing it by hand:
`release-vote-draft` emits an `atr vote start -m <vote_dev_list>
--subject "<vote_subject_template rendered>" --no-auto-publish <project>
<version>` command (with the drafted body pointing at the download URL)
rather than an email to paste, and `release-vote-tally` reads ATR's
tabulation (`atr vote tabulate`) and cross-checks it against
`release_approver_roster_path`. `--no-auto-publish` keeps ATR from
publishing when the dist backend (not ATR) owns promotion. The window and
pass rule are unchanged. ATR verb/flag names may shift between releases —
the skills confirm with `atr <cmd> --help` and do not depend on a
`revisions` lookup (the vote targets the latest uploaded revision by
default).

### Approval, non-list variants

| Mechanism | Required keys | Notes |
Expand Down
87 changes: 62 additions & 25 deletions projects/magpie/release-management-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,30 @@ mandatory ASF approval + announce mechanisms (`dev-list-vote`,
`announce-list`).

> [!IMPORTANT]
> **Distribution backend = `svnpubsub`** (the ASF-ratified default),
> per the [`svnpubsub` runbook](../../docs/release-management/svn-release-runbook.md).
> **Apache Trusted Releases (ATR) is the intended direction** and is
> fully documented in the [ATR release runbook](../../docs/release-management/atr-release-runbook.md),
> but ATR is in **alpha** and its adoption is **pending a PMC
> ratification vote on `dev@`**. Until that vote passes,
> `release_dist_backend` stays `svnpubsub`. After ratification, switch
> the value below to `atr` (and see `atr_platform_url`); no other change
> to this file is needed, since the approval and announce mechanisms are
> backend-independent.
> **Hybrid backend while ATR is in alpha: SVN for artefacts, ATR for the
> vote.** The two concerns are decoupled:
> - **`release_dist_backend = svnpubsub`** — the signed artefacts are
> staged to `dist/dev/` and promoted to `dist/release/` on
> `dist.apache.org` by `svn mv`, per the
> [`svnpubsub` runbook](../../docs/release-management/svn-release-runbook.md).
> SVN remains the **durable, ASF-hosted, canonical** home of the release
> bits. ATR's **Finish**/publish is **not** used while this is
> `svnpubsub`.
> - **`release_vote_backend = atr`** — the mandatory `dev@` `[VOTE]` is
> administered by the [ATR platform](../../docs/release-management/atr-release-runbook.md):
> the signed artefacts are *also* uploaded to ATR (Compose) so it runs
> the signature / checksum / licence / source-header checks and then
> **sends and tabulates** the `[VOTE]`. This is why artefacts land in
> **both** places during the RC.
>
> **Why the split:** ATR is in **alpha**, so we do not yet trust it to
> *host or publish* the release (that stays on SVN, ratified). But its
> automated checks and vote administration are useful now. Full adoption
> (flipping `release_dist_backend` to `atr`, so ATR also hosts/publishes
> via Finish) is **pending a PMC ratification vote on `dev@`** and a move
> of ATR from alpha to beta/GA. After that, set `release_dist_backend =
> atr` and drop the SVN staging/promote steps; the approval and announce
> mechanisms are backend-independent and need no change.

## Identifiers

Expand All @@ -64,32 +78,50 @@ mandatory ASF approval + announce mechanisms (`dev-list-vote`,
| Key | Value | Allowed values |
|---|---|---|
| `release_dist_backend` | `svnpubsub` | `svnpubsub`, `atr`, `github-releases`, `s3`, `self-hosted` |
| `release_vote_backend` | `atr` | `manual`, `atr` |
| `release_approval_mechanism` | `dev-list-vote` | `dev-list-vote`, `github-discussion`, `pr-approval`, `maintainer-roster` |
| `release_announce_backend` | `announce-list` | `announce-list`, `github-release-notes`, `site-post`, `discord-channel` |

As an ASF TLP, Magpie is pinned to `dev-list-vote` (mandatory per
[release-policy § release approval](https://www.apache.org/legal/release-policy.html#release-approval))
and `announce-list` (mandatory per
[release-policy § announcements](https://www.apache.org/legal/release-policy.html#release-announcements)).

`release_dist_backend = svnpubsub` stages the RC under `dist/dev/` and
promotes to `dist/release/` on `dist.apache.org`; see the
promotes to `dist/release/` on `dist.apache.org` by `svn mv`; see the
[`svnpubsub` runbook](../../docs/release-management/svn-release-runbook.md).
Setting it to `atr` (after PMC ratification) instead drives compose /
check / vote / finish through the ATR platform; see the
[ATR release runbook](../../docs/release-management/atr-release-runbook.md).
Flipping it to `atr` (after PMC ratification) instead drives compose /
check / vote / **finish** — including hosting and publishing — through the
ATR platform; see the [ATR release runbook](../../docs/release-management/atr-release-runbook.md).

`release_vote_backend` selects how the mandatory `dev-list-vote` is
*administered*, independently of where the artefacts are hosted:
- `manual` — the RM sends the `[VOTE]` email by hand and tallies replies
from the mail archive (the classic flow the `svnpubsub` runbook
describes).
- `atr` — the signed artefacts are uploaded to ATR (Compose) so it runs
the automated policy checks and then **sends and tabulates** the `[VOTE]`
on `dev@`. **`release_vote_backend = atr` with `release_dist_backend =
svnpubsub` is the current hybrid** (see the callout above): SVN hosts and
promotes; ATR only checks and drives the vote. ATR's Finish/publish is
not used until `release_dist_backend` itself becomes `atr`.

## Distribution URLs

| Key | Value |
|---|---|
| `release_dist_url_template` | `https://dist.apache.org/repos/dist/<bucket>/magpie/<version>/` |
| `archive_url_template` | `https://archive.apache.org/dist/magpie/` |
| `atr_platform_url` | `https://release-test.apache.org/` *(only used once `release_dist_backend = atr`; alpha host, production will be `release.apache.org`)* |
| `atr_platform_url` | `https://release-test.apache.org/` *(used whenever `release_vote_backend = atr` or `release_dist_backend = atr`; alpha host, production will be `release.apache.org`)* |

On the `svnpubsub` default, `<bucket>` resolves to `dev` while the RC
is staged for the vote and `release` after promotion. On the `atr`
backend (post-ratification) the RC lives in ATR's draft/candidate area
during Compose+Vote and **Finish** publishes to `dist/release/magpie/`.
On the `svnpubsub` dist backend, `<bucket>` resolves to `dev` while the
RC is staged for the vote and `release` after promotion. Under the
current hybrid (`release_dist_backend = svnpubsub`, `release_vote_backend
= atr`) the artefacts live in `dist/dev/magpie/<version>-rcN/` **and** are
uploaded to ATR's candidate area for the checks + vote — but promotion is
still `svn mv dist/dev → dist/release`, not ATR Finish. Only once
`release_dist_backend = atr` (post-ratification) does the RC live solely
in ATR and **Finish** publish to `dist/release/magpie/`.

## Signing

Expand All @@ -100,9 +132,10 @@ during Compose+Vote and **Finish** publishes to `dist/release/magpie/`.
| `rm_key_fingerprint` | *(per-RM; lives in the RM's `user.md` under `release_manager.gpg_fingerprint`)* |

The RM signs each artefact and the public key must be in `KEYS` (and,
once `release_dist_backend = atr`, also registered in the ATR platform,
which validates candidate signatures during Compose — see the ATR
runbook, Step B). The agent never holds the private key half.
whenever `release_vote_backend = atr` or `release_dist_backend = atr`,
also registered in the ATR platform, which validates candidate signatures
during Compose — see the ATR runbook, Step B). The agent never holds the
private key half.

## Vote

Expand All @@ -119,9 +152,13 @@ runbook, Step B). The agent never holds the private key half.

`vote_window_hours` is a floor per
[release-policy § release approval](https://www.apache.org/legal/release-policy.html#release-approval).
The ≥72h window and the binding-vote rule are backend-independent; on
the `atr` backend the platform sends the `[VOTE]` and tabulates, but the
window and rule are unchanged.
The ≥72h window and the binding-vote rule are backend-independent; when
`release_vote_backend = atr` the platform sends the `[VOTE]` to
`vote_dev_list` and tabulates replies, but the window and rule are
unchanged (ATR *drives* the mandatory `dev@` vote, it does not replace the
PMC's binding decision). The `[VOTE]` body still points voters at the SVN
`dist/dev/magpie/<version>-rcN/` staging URL for downloads while
`release_dist_backend = svnpubsub`.

## Announce

Expand Down
9 changes: 9 additions & 0 deletions skills/release-promote/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ This skill emits the backend-shaped promotion command set for a release
that has passed its vote. It is Step 10 of the
[release-management lifecycle](../../docs/release-management/process.md).

**Promotion follows `release_dist_backend`, not the vote backend.** Under
the hybrid (`release_dist_backend = svnpubsub`, `release_vote_backend =
atr`), ATR administered the *vote* but SVN owns *hosting and promotion*:
this skill emits the `svn mv dist/dev → dist/release` sequence, and it
does **not** emit `atr release finish` / any ATR publish command. ATR's
Finish phase is only used once `release_dist_backend` itself is `atr`. So
`release_vote_backend` has no effect here — the promotion path is chosen
solely by `release_dist_backend`.

The skill **never runs the promotion command itself** and **never publishes
the release**. This is
[Boundary 2](../../docs/release-management/spec.md#boundary-2-agent-never-publishes-the-release):
Expand Down
37 changes: 37 additions & 0 deletions skills/release-rc-cut/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ Read the following from `<project-config>/release-build.md` and
| `expected_artefacts` | `release-build.md` | `expected_artefacts` list |
| `digest_set` | `release-build.md` | `digest_set` list |
| `backend` | `release-management-config.md` | `release_dist_backend` |
| `vote_backend` | `release-management-config.md` | `release_vote_backend` (`manual` default, or `atr`) — when `atr`, Step 3 also emits an `atr upload` block |
| `staging_url` | `release-management-config.md` | `release_dist_url_template` rendered with `<version>-<rcN>` at `dist/dev/<project>/` (for `release_dist_backend = svnpubsub`) |
| `signing_key_fingerprint` | user.md or `release-management-config.md` | `rm_key_fingerprint` |
| `release_branch` | `release-management-config.md` | `release_branch_base` (or `--release-branch` override) |
Expand All @@ -240,6 +241,7 @@ Return ONLY valid JSON with this structure:
"expected_artefacts": ["<artefact filename pattern>"],
"digest_set": ["sha512"] | ["sha512", "sha256"],
"backend": "svnpubsub" | "github-releases" | "s3" | "self-hosted",
"vote_backend": "manual" | "atr",
"staging_url": "<URL>",
"signing_key_fingerprint": "<fingerprint or empty string>",
"release_branch": "<branch>",
Expand Down Expand Up @@ -367,6 +369,35 @@ The `release_publish_command_template` from
`release-management-config.md` rendered with `<version>` and `<rcN>`
substituted.

**Additionally, when `release_vote_backend = atr`** (the hybrid flow —
`release_dist_backend = svnpubsub` hosts + promotes, ATR runs the checks
and drives the vote), emit an ATR-upload block **after** the dist-backend
staging block. The signed artefacts must reach ATR so its Compose checks
run and there is a candidate revision to vote on. The artefacts land in
**both** the dist backend (above) and ATR (here); ATR's Finish/publish is
**not** emitted (promotion stays with `release_dist_backend`).

```text
# Upload the SAME signed + checksummed artefacts to ATR (Compose) so it
# runs signature/checksum/licence/source-header checks and can drive the
# [VOTE]. Verb/flag names may shift between ATR releases; confirm the
# current shape with `atr <cmd> --help`.
atr upload <project> <version> <artefact> <artefact>
atr upload <project> <version> <artefact>.asc <artefact>.asc
atr upload <project> <version> <artefact>.sha512 <artefact>.sha512
atr check status <project> <version> --verbose # confirm checks pass
atr check concerns <project> <version> # list any concern-group keys to ack at vote time
```

The uploaded candidate is the one `release-vote-draft` votes on;
`atr vote start` targets the latest uploaded revision by default (its
`--revision` flag is optional), so there is no separate revision id to
capture here.

This block is a proposal like the rest; the RM runs it on their machine
under their own ATR credentials. `atr_platform_url` from
`release-management-config.md` is the target platform.

Present the staging command block to the RM and ask for confirmation
before proceeding to Step 4.

Expand All @@ -375,13 +406,19 @@ Return ONLY valid JSON with this structure:
```json
{
"backend": "svnpubsub" | "github-releases" | "s3" | "self-hosted",
"vote_backend": "manual" | "atr",
"staging_commands": ["<command 1>", "<command 2>"],
"atr_upload_commands": ["<atr upload …>", "..."],
"staging_url": "<URL>",
"dist_dev_only": true,
"proposed": true
}
```

`atr_upload_commands` is populated only when `vote_backend = atr` (empty
list otherwise); it never contains an `atr release finish` / publish
command while `release_dist_backend = svnpubsub`.

`dist_dev_only` is always `true` for `svnpubsub` (`release_dist_backend = svnpubsub`); it confirms that no
`dist/release/` path (`release_dist_backend = svnpubsub`) was emitted. For non-`svnpubsub` backends it is
`false` (the field is not meaningful but must be present). `proposed`
Expand Down
Loading