diff --git a/.gitignore b/.gitignore index 4618e843..4304cf24 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,8 @@ tsl msalcpp design-docs 1ES-Pipelines +AuthClientAndroidPipelines +IdentityWiki.wiki nativeauthsample # Gradle files @@ -59,4 +61,17 @@ ehthumbs_vista.db out/ -plugins/buildsystem/bin \ No newline at end of file +plugins/buildsystem/bin + +ICM-investigation/ +# Release Orchestrator generated run-state (per-release, ephemeral) +.release-runs/ +.sim-runs/ + +# Python bytecode +__pycache__/ +*.pyc + + +# Personal step-mock overrides for release-agent (local testing only) +release-agent/mocks.local.yaml diff --git a/build.gradle b/build.gradle index a3e5b735..90a3f6b3 100644 --- a/build.gradle +++ b/build.gradle @@ -67,7 +67,7 @@ buildscript { dependencies { classpath "com.android.tools.build:gradle:${rootProject.ext.gradleVersion}" classpath "org.javassist:javassist:${rootProject.ext.javaAssistVersion}" - classpath "com.microsoft.intune.mam:android-build-plugin:${rootProject.ext.intuneAppSdkVersion}" + // classpath "com.microsoft.intune.mam:android-build-plugin:${rootProject.ext.intuneAppSdkVersion}" classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${rootProject.ext.kotlinVersion}" // classpath "net.serenity-bdd:serenity-gradle-plugin:1.9.6" // classpath 'com.google.gms:google-services:3.2.1' diff --git a/release-agent/EXTERNAL-REFERENCES.md b/release-agent/EXTERNAL-REFERENCES.md new file mode 100644 index 00000000..0269d9f7 --- /dev/null +++ b/release-agent/EXTERNAL-REFERENCES.md @@ -0,0 +1,66 @@ +# External References + +Everything the Release Orchestrator depends on that lives **outside this codebase**. +If any of these change (URL moved, DL renamed, pipeline re-IDed, template edited, +access revoked), the orchestrator can silently break — so they're catalogued here. +Review this list when something stops working or when onboarding a new release owner. + +Legend for **Access**: `anon` = no auth · `az` = Azure CLI signed-in user · +`AAD-SSO` = browser Microsoft sign-in · `MCP` = via an MCP server · `Google` = Google account (not automatable in Scout). + +## Azure DevOps orgs & projects (we work across TWO) + +| Project | Org | Host aliases | How we read it | +|---|---|---|---| +| **Engineering** | identitydivision | `identitydivision.visualstudio.com` = `dev.azure.com/identitydivision` | ADO **MCP** (bound here) or `az` | +| **One** | msazure | `msazure.visualstudio.com` = `dev.azure.com/msazure` | **`az` only** — the ADO MCP can't reach it | + +> The ADO **MCP is bound to identitydivision/Engineering**. Calling it for project **One** fails with `TF200016: project does not exist`. Reads against **msazure/One** must use the **`az` CLI** (verified working as the signed-in user, no Conditional-Access 401): `az pipelines build show`, and `az devops invoke --area build --resource timeline|logs`. Engineering resources: pipeline 3038, build def 2828. One resources: pipeline 405133 (localization), build def 397224, CG repo 104410, the Auth App git repo. + +## Systems of record (read/write) + +| Ref | What | Used by | Access | Notes | +|---|---|---|---|---| +| ADO pipeline **3038** | "Code Complete Calendar Checker" — CCD source of record | CCD seed, `set-ccd`, `skip-release`, Phase-0 `cron` (verify scheduled) | az | org identitydivision / project Engineering. Real writes gated by --confirm. `cron` step verifies a recent `schedule`-reason run. | +| ADO pipeline **405133** | Localization build (org msazure / project One) | Phase-1 `localization` | MCP (ADO) / az | triggered at noon on CCD with `isCreatePrSelected=true`; polled every 10 min (3h timeout → email engineer); its **OneLocBuild@3** task logs `Pull request created with ID ''` — that PR (`…/pullrequest/`) is posted to the Code reviews chat for review | +| ADO build def **2828** | Auth Client Android build (org identitydivision / project Engineering) | readiness `build_access` | az | access check only | +| ADO build def **397224** | Android Build Release (org msazure / project One) | readiness `build_access` | az | access check only | +| ADO wiki **IdentityWiki.wiki** page **59148** | "Monthly Releases Payloads History" (parent) | Phase-0 `wiki` agent | az (`az devops wiki`) | child page ` Release`; dup-safe numbering | +| **ICM team 78848** | "Auth Client Android Shield" on-call roster | readiness `oncall_now` | MCP (ICM) | primary = index 0 of currentOnCallContacts | +| **ADX cluster** idsharedeus2.eastus2 / db d496be22d62a46b0a3cf67ea2e736fd8 | release telemetry | readiness `adx_access` | MCP (Kusto) | `print 1` access probe | + +## External web pages (scraped / linked) + +| Ref | URL | Used by | Access | Notes | +|---|---|---|---|---| +| CCOA No-Fly Zones | https://prod.change-manager.msidentity.com/ccoa-periods | Phase-0 `lockdown` | AAD-SSO | scraped via browser; only Production-env periods block | +| **Component Governance alerts** (governed repo **104410** = AD-MFA-phonefactor-phoneApp-android, branch `working`) | https://msazure.governance.visualstudio.com/{One projId}/_apis/ComponentGovernance/GovernedRepositories/104410/Branches/working/Alerts | Phase-0 `cg` | az (`az rest`) | read-only report; active alerts by severity. projId=b32aa71e-…, resource=499b84ac-… | +| **`release` variable group 40** | https://identitydivision.visualstudio.com/Engineering/_library?...variableGroupId=40&path=release | Phase-0 `flight_reminder` (link only) | az/web | feature owners update local flights here — release engineer does NOT | +| Flight pre-mortem example doc | https://microsoft-my.sharepoint-df.com/:w:/p/rapong/cQpEZp0cXp1sQYo4A4M3PQWCEgUCDj364FJa-rq-msg59WlBsw | Phase-0 `flight_reminder` (link only) | AAD-SSO | example shared with feature owners | +| Localization instructions | https://eng.ms/docs/.../combined-release-checklist/localization | Phase-0 `flight_reminder` (link) · Phase-1 `localization` (manual-steps fallback + timeout email) | AAD-SSO | confirmed valid 2026-07-29 | +| **Teams chat: "Android Core Team"** | thread `19:976a859f167f44e59c4ceca8b1d23581@thread.v2` | Phase-0 `flight_reminder` target | MCP (WorkIQ) | real target; redirect for tests via the `send_to` mock knob | +| **Teams chat: "Code reviews"** | thread `19:meeting_Y2Y3OGRjZGMtZGVkYi00MTkzLThhZjktNDAxYWVkMjZlMmE3@thread.v2` | Phase-1 `pr_reminder` target | MCP (WorkIQ) | CCD PR-merge reminder; redirect for tests via the `send_to` mock knob | +| **CCD-delay / cherry-pick approver: Moumita Ghosh** | moghosh@microsoft.com | Phase-1 `pr_reminder` (named in message) | — | both a CCD delay and a post-CCD cherry-pick require her approval | +| **EcsFlight.kt** (Auth App ECS flights) | https://msazure.visualstudio.com/One/_git/AD-MFA-phonefactor-phoneApp-android?path=/.../ecs/entities/EcsFlight.kt&version=GBworking | Phase-0 `flight_reminder` bullet 4 (link only) | az/web | reviewers check its history since last code complete | +| Early code-complete notice template | https://eng.ms/docs/.../combined-release-checklist/early-code-complete-notice-email-template | Phase-0 `notice` | AAD-SSO | copied locally to `templates/early-code-complete-notice.md` — **re-sync if upstream edits** | +| Hotfix cherry-pick guide | https://eng.ms/docs/.../release/cherry-pick-to-hotfix-guidelines | link inside notice email body | AAD-SSO | referenced, not fetched | +| common-for-android changelog | https://raw.githubusercontent.com/AzureAD/microsoft-authentication-library-common-for-android/dev/changelog.txt | Phase-0 `breaking` | anon | breaking = `[MAJOR]` in `vNext` | +| Play Console vitals | (Google Play Console) | Phase-0 `vitals` (#8) | Google | **NOT automatable in Scout** (Google auth wall) | + +## Outbound email recipients + +> Runs are real — these recipients receive the notice. To test without emailing +> the DL, redirect with the `send_to` mock knob (`mocks.local.yaml`). + +| Step | To | Notes | +|---|---|---| +| Phase-0 `notice` (early code-complete) | androididentity@microsoft.com ("Azure Identity Android SDK"), jialh@microsoft.com | provided by release owner 2026-07-29 | +| Phase-1 `final_reminder` (CCD-day code-complete) | androididentity@microsoft.com ("Azure Identity Android SDK"), jialh@microsoft.com | same DL as `notice`; CCD-day "update" variant | + +## Tooling / infra (provisioned by bootstrap) + +| Ref | What | Notes | +|---|---|---| +| Agency CLI | provides the **ICM** and **Kusto** MCP servers | `agency mcp icm` / `agency mcp kusto`; auto-registered into `~/.scout/m-mcp-servers.json` by `cli infra` | +| Azure CLI (`az`) + `azure-devops` extension | pipeline + wiki reads/writes | signed-in user is the release owner | +| Scout | host for the skill + automations | `~/.scout`; bootstrap checks presence | diff --git a/release-agent/README.md b/release-agent/README.md new file mode 100644 index 00000000..9a6400e1 --- /dev/null +++ b/release-agent/README.md @@ -0,0 +1,260 @@ +# Release Orchestrator (`/release-agent`) + +The **conductor backbone** for the Android monthly release (ADO items **X4** + **X5**). +It drives the whole release as a state-aware smart checklist: it knows the phases, +runs each step's agent, and **holds at gates** for the release engineer to decide. + +> **Build status.** **Phase 0 (pre-flight) has real, tested agents** (early-notice, +> flight/string reminders, BREAKING-OneAuth detection, CG alerts, cron verify, wiki +> payload); the later phases are still **stubs** (mock actions) and get filled in one +> at a time (see the Release-Stabilization roadmap). Building on this shared backbone — +> not 50 one-off scripts — is what makes the "agent carries the knowledge" model real. + +## Architecture (thin skill over a deterministic engine) + +``` + you ──/release-agent──▶ SKILL (conversation layer) ──shell──▶ ENGINE (Python, deterministic) + presents gate briefs, state machine + dispatch + run-state + relays your approve/deny the BRAIN — fully unit-tested +``` + +- **Engine = the brain.** Decides what's next, runs stubbed agents, holds at gates, persists run-state. No LLM — unit-tested and replayable. +- **Skill = the mouth & ears.** Presents the gate, collects your decision, relays it. Never decides the flow. + +## Layout + +``` +release-agent/ COMMITTED (distributed with android-complete) +├─ config/ +│ ├─ phases.yaml the state machine COMPOSITION (phases → steps → gates, order, deps), as data +│ ├─ readiness.yaml the entry-gate checklist, as data +│ ├─ knowledge.yaml per-step help (what/where/how/links/faqs) for `step-info` — data (a step module may override via KNOWLEDGE) +│ ├─ schedule.yaml where CCD comes from (pipeline 3038 coords), as data +│ └─ requirements.yaml external dependencies (CLIs, extensions, MCP servers) — single source of truth +├─ orchestrator/ three layers: logic → data → presentation +│ ├─ engine.py the conductor: state machine + dispatch (resolves steps via steps.get_step) + gates + status_report +│ ├─ readiness.py ReadinessGate: entry-gate logic (verify/sign/decline) → structured data +│ ├─ schedule.py CCD math (2nd Wednesday, override, phase anchors) — pure, no IO +│ ├─ mocks.py local test overlay loader (mocks.local.yaml): outcome / readiness / stepresult +│ ├─ knowledge.py step knowledge resolver (config/knowledge.yaml + module KNOWLEDGE overlay) +│ ├─ infra.py infra preflight: check CLIs + register/verify MCP servers into Scout config +│ ├─ render.py presentation only: structured data → text/markdown (swap for other UIs) +│ ├─ state.py Release State Record / run-state (X5); StepState carries note + links +│ ├─ discovery.py find releases (none / one / many) +│ ├─ registry.py automation registry (track provisioned automations for teardown) +│ ├─ eventlog.py per-release interaction + event log +│ ├─ cli.py thin entry point: builds the parser from commands/, dispatches +│ ├─ cli_common.py shared CLI plumbing (load state, emit, event log, advance block) +│ └─ commands/ one module per command domain (self-registering subparsers) +│ ├─ release.py lifecycle + overrides: init/list/status/next/approve/deny/done/skip/reopen/halt/resume/activate +│ ├─ readiness.py entry gate: checklist/verify/sign/decline +│ ├─ pipeline.py real pipeline writes (gated): set-ccd / skip-release +│ ├─ notify.py daily phase digest (tick advances + reports; notify = read-only) + set-owner +│ ├─ lockdown.py CCOA overlap recorder: check-lockdown +│ ├─ step_action.py generic scout-step dispatcher + mock-spec + step-info +│ ├─ notice.py record-step (scout-step recorder) +│ ├─ logs.py event log: log / journal +│ ├─ automation.py automation registry command +│ └─ infra_cmd.py infra preflight command +├─ steps/ THE STEP HOME — one self-contained module per step (auto-discovered) +│ ├─ __init__.py discover()/get_step(): scans steps//*.py — NO hand-maintained registry +│ ├─ lib/ shared step helpers (context, templating, agent adapter, mock-input ctx) +│ └─ preflight/ Phase-0 step modules — each declares ID/KIND/build + optional MOCKABLE/KNOWLEDGE/CONFIG +│ ├─ notice.py flight_reminder.py lockdown.py (scout) +│ ├─ breaking.py cg.py cron.py wiki.py (agent — run in-engine) +│ └─ confirm_reminders.py vitals.py (attest) +├─ phases/ +│ ├─ stub_runner.py generic runner for steps that have no module yet (later phases) +│ └─ readiness_verifiers.py auto verifiers for the entry gate (pass/fail) +├─ tools/checks.py real IO (az / http), isolated +├─ skill/SKILL.md the /release-agent Scout skill +├─ mocks.local.example.yaml template → copy to mocks.local.yaml (gitignored) for local testing +├─ setup/bootstrap.ps1 one-time setup (infra preflight, installs skill) +└─ tests/test_engine.py unit + full flow-replay + structural guardrail tests + +.release-runs// GENERATED, gitignored (per-release working state) +├─ release-state.json the per-release metadata + run-state (owner, CCD, steps, gates, …) +├─ events.jsonl the per-release event/interaction log +└─ _automations.json registry of THIS release's provisioned Scout automations (owned by the release; removed at close) +.release-runs/_automations.json GENERATED, gitignored — registry of SHARED (machine-wide) automations only +``` + +## Adding a step (the modular contract) + +A step is **one self-contained module**; everything else is derived, so there are no +hand-maintained registries to forget. Adding a Phase-0 step touches **2 files**: + +1. **`steps/preflight/.py`** — the step. Declare `ID`, `KIND` (`agent`|`scout`| + `attest`), `build(state) -> Outcome`, and optionally `MOCKABLE` (mock knobs), + `KNOWLEDGE` (help), `CONFIG` (the step's config). Agent steps add `run = legacy_run(build)`. + It is **auto-discovered** — no registry edit. +2. **`config/phases.yaml`** — one line placing the step in the flow (id, name, order, + deps, gate/attest/source flags). + +That's it. Mocking works automatically (`outcome`/knobs); `step-info` shows its knowledge; +`step-action`/`mock-spec` find it. The **`test_step_modules_and_config_stay_in_sync`** +guardrail fails loudly if a module and `phases.yaml` drift, so nothing silently breaks. + +**Two homes for data (by lifetime):** +- **Release metadata + run-state** → `.release-runs//release-state.json` (per-release; the `ReleaseState` record). Holds `owner_email`/`owner_name` (the release owner, resolved from the signed-in `az` user at `init`; reminders email this person), `ccd`/`ccd_source`/`ccd_conflict`, step completion, gate decisions, `last_notified_date`, etc. Add release-scoped fields here. +- **Tool config** → `release-agent/config/*.yaml` (not release-specific; committed): `phases.yaml`, `readiness.yaml`, `schedule.yaml`, `requirements.yaml`. + +## Architecture — three layers (so it adapts to other interfaces) + +1. **Logic** (`engine.py`, `readiness.py`, `schedule.py`, `state.py`) — pure, deterministic, returns **structured data**. No formatting, no IO. +2. **Presentation** (`render.py`) — pure functions: structured data → text/markdown. A different interface (web UI, TUI) swaps this layer and reuses everything else. +3. **Interface** (`cli.py` + `cli_common.py` + `commands/` + `skill/SKILL.md`) — the CLI is a thin assembler: `cli.py` builds the parser from the self-registering modules in `commands/` (one per domain), and shared plumbing lives in `cli_common.py`. Adding a command is a localized change to one module. + +IO lives in `tools/` and `phases/` (pluggable). Config is data in `config/`. + + +## Run-state: two kinds (the X5 idea) + +- **Derived** — recomputed from systems of record (ADO/Git/Play Console/ADX). Never stored ⇒ never stale. *(reconcilers are stubbed for now.)* +- **Persisted** — decisions/intent, step completion, pending human actions. Stored in `release-state.json`. + +The conductor is **stateless**: on each invocation it loads the record, (later) reconciles against live systems, decides, acts, writes back. That's what lets a release resume across days/sessions. + +## Quick start + +Run the one-time setup from the **`release-agent` folder of your `android-complete` clone**, +using PowerShell 7 (`pwsh`): + +```powershell +# one-time — from the release-agent folder +cd C:\repos\android-complete\release-agent # adjust to your clone location +pwsh .\setup\bootstrap.ps1 +``` + +`bootstrap.ps1` runs an **infrastructure preflight** first (`python -m orchestrator.cli infra`), +driven by **`config/requirements.yaml`** (the single source of truth for external +dependencies). It checks each CLI/host dependency and prints an `install:` hint for +anything missing, then **registers any required MCP servers into Scout's config** +(backing the file up first) and tells you to **restart Scout** so they load. Keep +`requirements.yaml` up to date whenever a new dependency (CLI, package, or MCP +server) is introduced. + +You can run the preflight any time on its own: + +```powershell +python -m orchestrator.cli infra # check + auto-register MCP servers (restart Scout after) +python -m orchestrator.cli infra --no-register # report only +``` + +```powershell +# drive a release (runs are real; keep a mocks.local.yaml for safe testing) +cd release-agent +python -m orchestrator.cli init --release 2026-07 +python -m orchestrator.cli next --release 2026-07 # runs until the first gate +python -m orchestrator.cli approve --release 2026-07 --comment "flags reviewed" +python -m orchestrator.cli status --release 2026-07 +``` + +Or in Scout: **`/release-agent`**. + +## Time anchoring — phases open relative to the Code Complete Date (CCD) + +Phases don't fire on demand; they're anchored to the **CCD**. **The CCD is +canonically the 2nd Wednesday of the month.** The orchestrator still reads ADO +pipeline **3038 "Code Complete Calendar Checker"**, but it does **not** silently +adopt the pipeline's `overrideCodeCompleteDate`: if that override is a *different* +in-month date, the tool flags a **conflict** (`ccd_conflict`) and asks the user +which date is real — the default or the pipeline's. `init` computes the default +and reports any conflict. + +- **Phase 0 opens at `CCD-7`** (declared as `anchor: "CCD-7"` on the phase in + `phases.yaml`). Before then the release is **`scheduled`** — the engine runs + nothing and status shows *"opens `` (in N days)"*. Other phases are + dependency-driven for now; add an `anchor:` to any phase to time-gate it too. +- **Simulated clock:** every read/advance command takes `--as-of YYYY-MM-DD` so a + `--as-of` can jump to CCD-7 and prove a phase opens on schedule. Normal runs use today. +- **Resolving a conflict / changing the CCD.** `set-ccd` and `skip-release` + **write back** to pipeline 3038 (override / `skipRelease`) — real production + changes, so they're gated: preview first, then re-run with `--confirm` (a + `--reason` is always required and audited). Pick the default → `set-ccd --default` + clears the pipeline override so they match; pick the pipeline date → `set-ccd + --date `. `status` re-reads the pipeline and re-flags any new conflict. + +```powershell +python -m orchestrator.cli set-ccd --release 2026-07 --date 2026-07-15 --reason "more bake time" # preview +python -m orchestrator.cli set-ccd --release 2026-07 --date 2026-07-15 --reason "more bake time" --confirm +python -m orchestrator.cli status --release 2026-07 --as-of 2026-07-01 # jump the clock +python -m orchestrator.cli done --release 2026-07 --note "China upload complete" # clear a reminder hold +``` + +## Push reminders — daily phase digest (reaching you when Scout is closed) + +Everything the engine surfaces is **pull** — you see it when you open Scout. The +**push** layer is a **daily phase status digest** emailed to the release owner: + +- **Setup is interactive → no push.** Readiness + establishing the CCD happen in + Scout, so they're never emailed (unsigned / blocked / halted = silent). +- **First push = a phase opening** (Phase 0 at CCD‑7). Nothing before it. +- **Daily while a phase is open with outstanding work** — once/day, progress + + what still needs you, until the phase's actions are done; then the next phase's + digest takes over when it opens (each phase notifies on open). + +```powershell +python -m orchestrator.cli tick --json # advance to today + {message,subject,owner_email,...} +python -m orchestrator.cli tick --as-of 2026-08-06 # simulate a date (debug) +python -m orchestrator.cli notify --json # read-only: report WITHOUT advancing (manual check) +``` + +A **Scout automation** runs **`tick --json` hourly** and, when `message` is non‑empty, +emails it to `owner_email` (subject from the JSON). `tick` both **advances** the release +to the current date and reports; running hourly means a tick missed while the machine +was off is picked up by the next one, and a once-per-calendar-day guard +(`last_notified_date`) keeps it to one advance-effect and one email per day. `notify` is +the **read-only** variant (report without advancing); `--as-of`/`--force` are debug overrides. + +**Automation registry.** Every automation the orchestrator provisions is recorded via +`cli automation register` so it can be torn down cleanly. **Per-release** automations +(`--release `, the default) live in `.release-runs//_automations.json` — +co-located with that release's state so ownership is explicit and they're removed with +the release folder at close (`automation list --release ` → delete each → +`automation deregister`). Push reminders are per-release too. A `--shared` scope (stored +machine-wide at `.release-runs/_automations.json`) exists for the rare automation meant +to outlive every release. + +## Two kinds of human step + +- **Gate** (`gate: true`) — a *decision*: the conductor holds and you `approve`/`deny`. +- **Reminder** (`owner: human`, no gate) — a *to-do*: the conductor holds + ("ACTION NEEDED"), you go do it, then `done` it. Not a decision — just done / not-yet. + +## Event log (for analysis & improvement) + +Every action is recorded to an append-only JSONL event log so we can improve the +process across engineers and months. The highest-value signal is the **decision +driver** — the reason attached to each gate approve/deny/decline. + +- Per-release trace: `.release-runs//events.jsonl` (one log per release; there is no machine-wide aggregate). + +```powershell +python -m orchestrator.cli log --release 2026-07 # this release's trace +python -m orchestrator.cli log --release 2026-07 --analyze # rolled-up summary +``` + +Events captured include: `release_started`, `readiness_verified/signed/declined`, +`step_ran`, `gate_hold` + `gate_approved`/`gate_denied` (with `driver`), +`reminder_hold`/`reminder_done`, `scheduled_hold`, `ccd_changed`, +`release_skip_set`/`release_skip_cleared`, `step_skipped`/`step_reopened`, +`release_halted`/`release_resumed`, `release_complete`, plus interaction events +(what Scout showed / what the user chose). Logging never breaks the flow (best-effort). + +> The log lives under the gitignored `.release-runs/`, so it's per-machine. Shipping +> logs to a shared store (Kusto/ADO/wiki) for cross-engineer analysis is a future step. + +## Tests + +```powershell +cd release-agent +python tests/test_engine.py # unit + full flow replay + readiness + eventlog +``` + +## Design constraints honored (from §7.1 of the stabilization plan) +1. Real-by-default with a personal `mocks.local.yaml` (skip/redirect/inject per step) is the test method — never blast the real DL from a test (use a `send_to` redirect). +2. Run-state schema defined once, upfront (X5), shared by all agents. +3. Sequence by risk/value — agents are independent plug-ins on the backbone. +4. Manual overrides are first-class (approve/deny gates; activate conditional phases). +5. Conductor is stateless; minimize persisted state, derive the rest. diff --git a/release-agent/config/automations.yaml b/release-agent/config/automations.yaml new file mode 100644 index 00000000..a36903d3 --- /dev/null +++ b/release-agent/config/automations.yaml @@ -0,0 +1,66 @@ +# Per-release automation definitions — the SINGLE source of truth for which Scout +# automations a release provisions and WHICH STEPS each one drives (traceability). +# +# This is DATA, not code. The engine/CLI never call Scout's automation API; the +# skill does (m_create_automation / m_delete_automation). `automation plan +# --release ` reads this file + the release CCD and emits the concrete +# automations to create; each is then recorded in _automations.json WITH its steps, +# so automation<->step linkage is queryable and torn down cleanly at release close. +# +# Timing is NOT specified here — it is DERIVED from each step module's +# `fire_at_local` (steps//.py CONFIG). All steps in one automation must +# share the same fire_at_local; a guardrail test enforces that every scheduled step +# (one that declares fire_at_local) is owned by exactly ONE automation here. +# +# Fields: +# slug : stable short id for this automation (used in the registry + names) +# name : display name; "{release}" is filled with the release id (YYYY-MM) +# phase : the phase these steps belong to (all steps must be in it) +# steps : "." ids this automation runs, in order +# every : OPTIONAL. If set (e.g. "10 minutes"), this is an INTERVAL/poller +# automation on that cadence instead of a one-shot at a step's +# fire_at_local. Interval automations may share a step with a +# time-of-day one (e.g. noon triggers localization, a poller watches it) +# and their steps need not declare fire_at_local. +# purpose : one-line description (stored in the registry) + +version: 1 + +automations: + - slug: ccd-morning + name: "Release {release} — CCD morning reminders" + phase: ccd + steps: + - ccd.final_reminder # email: code complete is TODAY + - ccd.pr_reminder # Teams: merge required PRs before the 11 PM cut + purpose: "Send the CCD-day code-complete email + the Code reviews PR-merge reminder" + + - slug: ccd-noon + name: "Release {release} — CCD noon localization" + phase: ccd + steps: + - ccd.localization # trigger pipeline 405133 (isCreatePrSelected=true) + purpose: "Trigger the localization pipeline at noon" + + - slug: ccd-localization-poller + name: "Release {release} — localization poller" + phase: ccd + every: "10 minutes" + steps: + - ccd.localization # poll the triggered run (check-localization) + purpose: "Poll the localization run every 10 min; email on 3h timeout, post the PR on completion" + + # ON-DEMAND (not provisioned at release start). Created only when Build & RC + # Verification is holding on an in-flight re-triggered RC (the human ran + # `rc-retriggered`); torn down when `poll-rc` reports `resolved`. Interval automation, + # so its step needs no fire_at_local. + - slug: build-verify-rc-poller + name: "Release {release} — RC verification poller" + phase: build_verify + every: "30 minutes" + on_demand: true + steps: + - build_verify.rc_report # poll the in-flight RC (poll-rc): re-gate on completion, nudge at 6h + purpose: "Poll a re-triggered RC every 30 min; hold while in-flight, re-apply the gate on completion, courtesy-nudge the owner at 6h" + + diff --git a/release-agent/config/knowledge.yaml b/release-agent/config/knowledge.yaml new file mode 100644 index 00000000..44bf6a7c --- /dev/null +++ b/release-agent/config/knowledge.yaml @@ -0,0 +1,658 @@ +# Step knowledge base — reference/help content the skill uses to answer a user's +# questions about a step ("what does this do?", "where do I find X?", "how do I +# resolve it?"). This is DATA (not code): edit here to keep answers accurate. +# +# Keyed by ".". Fields (all optional): +# summary : one-line description +# what : what the step does / why it matters +# who : who owns / is responsible for the step's action +# where : navigation / where to look (list of precise paths) +# how : how to complete or resolve it +# links : [{name, url}] reference links +# faqs : [{q, a}] common questions with accurate answers +# +# A step MODULE may also declare KNOWLEDGE (co-located); the resolver overlays the +# module on top of this file, so code and this data compose. Steps without an entry +# report "no knowledge yet" honestly rather than guessing. + +version: 1 + +# ============================== Phase 0 · preflight ============================== + +preflight.notice: + summary: "Email the early code-complete notice to the Android distribution list." + what: > + Sends a heads-up that code complete for the month's release is approaching, so + teams stop landing feature work and switch to the hotfix cherry-pick process + after code complete. Filled from a local template with the CCD and release owner. + how: > + Scout resolves the email (subject, HTML body, recipients) and sends it via + WorkIQ, then records the step. Recipients are the configured distribution list; + for a test, redirect with the `send_to` mock knob. + links: + - name: "Hotfix cherry-pick guide (linked in the email)" + url: "https://eng.ms/docs/microsoft-security/identity/entra-developer-application-platform/auth-client/microsoft-authenticator/microsoft-authenticator/release/cherry-pick-to-hotfix-guidelines" + faqs: + - q: "Who receives it?" + a: "The configured distribution list (androididentity@microsoft.com, jialh@microsoft.com). Redirect for a test with preflight.notice.send_to." + - q: "What's the difference between the 'initial' and 'update' variant?" + a: "'initial' is the CCD-7 heads-up; 'update' is the CCD-day reminder ('code complete is Today'). Set via --param variant=update." + +preflight.flight_reminder: + summary: "Post the combined feature-owner reminders to the Android Core Team Teams chat." + what: > + One Teams message with four reminders as code complete approaches: update local + flights, write flight pre-mortem docs, merge user-facing strings by CCD-7, and + complete the feature-flag freeze / default-OFF review. + how: > + Scout posts the message to the Android Core Team group chat and records the step. + For a test, redirect to your own chat with the `send_to: me` mock knob. + links: + - name: "release variable group (local flights)" + url: "https://identitydivision.visualstudio.com/Engineering/_library?itemType=VariableGroups&view=VariableGroupView&variableGroupId=40&path=release" + - name: "Localization instructions" + url: "https://eng.ms/docs/microsoft-security/identity/entra-developer-application-platform/auth-client/authn-sdk-msal-android/android-auth-libraries/releases/combined-release-checklist/localization" + - name: "EcsFlight.kt history (flag-freeze review)" + url: "https://msazure.visualstudio.com/One/_git/AD-MFA-phonefactor-phoneApp-android?path=/PhoneFactor/ExperimentationLibrary/src/main/java/com/microsoft/authenticator/experimentation/ecs/entities/EcsFlight.kt&version=GBworking" + faqs: + - q: "Why CCD-7 for strings?" + a: "Strings merged inside the 1-week window aren't guaranteed to be localized in time, so user-facing strings must land by CCD-7." + +preflight.confirm_reminders: + summary: "Attest that feature owners completed the reminded pre-code-complete work." + what: > + Sending the reminders doesn't prove the work got done. This step holds until the + owner confirms feature owners actually: updated local flights, wrote pre-mortem + docs, merged user-facing strings by CCD-7, and left features default-OFF (or got + default-ON approved in the release wiki). + how: "Confirm with the feature owners, then mark the step done. If you can't confirm, leave it holding." + +preflight.lockdown: + summary: "Detect whether a CCOA lockdown / holiday no-fly zone overlaps the release window." + what: > + Reads the CCOA 'No-Fly Zones' and checks whether any PRODUCTION-environment + lockdown overlaps the release window (CCD-7 through CCD+14). Banner-only + advisories don't block; a Production overlap holds the step so the owner can + shift the CCD past it. + where: + - "CCOA periods page: the change-manager site — 'Upcoming CCOA periods' and the current-year 'Past NoFly Zones' table." + how: > + Scout scrapes the page (AAD SSO), the engine decides overlap deterministically. + On an overlap it names the clashing lockdown(s); shift the CCD with set-ccd if + you want to proceed. On no overlap it passes. + links: + - name: "CCOA periods (change-manager)" + url: "https://prod.change-manager.msidentity.com/ccoa-periods" + faqs: + - q: "Which lockdowns actually block?" + a: "Only Production-environment CCOA periods that intersect CCD-7..CCD+14. Banner-only advisories are ignored." + +preflight.breaking: + summary: "Detect BREAKING-OneAuth changes in the common-for-android changelog and draft comms." + what: > + Scans the unreleased 'vNext' section of the common-for-android changelog for + [MAJOR] (breaking) entries — the changes shipping in this release that OneAuth + consumers must handle — and drafts the notification comms. Read-only. + where: + - "Source: the common-for-android changelog.txt (dev branch), 'vNext' section." + how: "Review any detected [MAJOR] entries and send the drafted comms to the OneAuth team. No breaking changes → nothing to do." + links: + - name: "common-for-android changelog" + url: "https://raw.githubusercontent.com/AzureAD/microsoft-authentication-library-common-for-android/dev/changelog.txt" + +preflight.cg: + summary: "Report active Component Governance alerts; block on High/Critical." + what: > + Reads Component Governance alerts for the governed Android repo and reports + active alerts by severity. It BLOCKS (holds) when there are active High/Critical + alerts — the release engineer fixes them and reruns the check. Overriding to ship + WITHOUT a fix is allowed only with the release engineer's MANAGER's approval + (recorded in the skip reason). + who: > + The RELEASE ENGINEER (release owner) is responsible for remediating Component + Governance alerts — this includes creating the fix PR (e.g. bumping the flagged + dependency to a patched version) and driving it to merge, not just pinging another + team. CG remediation is a release-owner duty, not something delegated to the + component owner by default. The ONE exception to fixing is an override to ship as-is, + which is not the engineer's call alone — it requires their MANAGER's approval. + where: + - "Component Governance alerts page for the repo (see link) — review the active High/Critical items." + - "The flagged dependency's version is pinned in the repo's version catalog (gradle/libs.versions.toml) or the module build.gradle — that's where the fix PR changes it." + how: > + Preferred path — FIX: as release engineer, open the alert, identify the flagged + dependency and patched version, raise the fix PR (update the pin, e.g. + httpcore5 5.3 -> 5.4.3), get it merged, then rerun this step to re-check. + Override path — SHIP WITHOUT FIXING: this needs your MANAGER's explicit approval + first. Once the manager signs off, skip the step with a reason that NAMES the + approving manager and why (audited), e.g. + `skip --release --phase preflight --step cg --reason "High CG alert X — ship + as-is approved by on ; fix tracked in "`. + Never override on your own authority. + faqs: + - q: "Who fixes the alert / creates the PR?" + a: "The release engineer (release owner). Remediating CG alerts — including authoring and landing the fix PR — is a release-owner responsibility, not the component owner's." + - q: "Can I ship with an open High/Critical CG alert?" + a: "Only as an override, and only with your MANAGER's approval. The release engineer cannot waive a High/Critical CG block on their own — get manager sign-off, then skip the step with a reason that records who approved it and links the follow-up fix. If there's no manager approval, fix the alert first." + - q: "Does a Medium/Low alert block?" + a: "No — only active High/Critical alerts block. Medium/Low are reported but don't hold the release." + links: + - name: "Component Governance alerts" + url: "https://msazure.visualstudio.com/One/_componentGovernance/AD-MFA-phonefactor-phoneApp-android" + +preflight.vitals: + summary: "Attest that Google Play Console app health (Android vitals + Policy status) is acceptable." + what: > + Play Console has no API for policy status, so this is a manual confirmation: + review the app's technical health (crash rate / ANR rate) and its policy standing + (open issues or warnings), and confirm both are acceptable before code complete. + where: + - "Android vitals: Play Console → left panel → 'Monitor and improve' → 'Android vitals' → Overview." + - "Policy status: Play Console → left panel → 'Monitor and improve' → 'Policy and programs' → 'Policy status'." + how: > + Open both pages, confirm crash/ANR are within acceptable bounds (no regression) + and there are no open policy issues/warnings, then mark the step done. An + unresolved policy issue or a vitals regression → leave it holding. + links: + - name: "Google Play Console" + url: "https://play.google.com/console" + faqs: + - q: "Where do I find Android vitals?" + a: "Play Console → left panel → Monitor and improve → Android vitals → Overview." + - q: "Where is policy status?" + a: "Play Console → left panel → Monitor and improve → Policy and programs → Policy status." + +preflight.cron: + summary: "Verify the Code Complete Calendar Checker pipeline is scheduled and firing." + what: > + Pipeline 3038's cron lives in YAML (not exposed in its triggers), so the check + proves it's actually firing by confirming a recent schedule-reason run in its + build history. Blocks if there's no recent scheduled run or it's stale. + how: "If blocked, investigate the pipeline's schedule (it may be disabled/broken), fix it, and rerun the step — or skip to override." + +# =============================== Phase 1 · ccd =============================== + +ccd.final_reminder: + summary: "Email the Android DL that code complete is TODAY (morning of CCD)." + what: > + The CCD-day final code-complete reminder email — the 'update' variant of the + same notice template Phase 0 sends a week earlier at CCD-7. Tells everyone code + complete is today and that any check-ins after it need the hotfix cherry-pick + process + EM approval. + who: > + Scout sends it automatically (a scout step) on the morning of CCD, driven by the + per-release CCD-morning automation. The release owner owns the content/recipients. + how: > + Runs via step-action → workiq_send_email. To test, redirect with the send_to + mock knob (keeps the send real, points it at you). Sent ~09:00 local on CCD. + faqs: + - q: "How is this different from the Phase 0 notice?" + a: "Same template, different variant/timing: Phase 0 'notice' sends the 'initial' variant at CCD-7; this sends the 'update' variant ('Today') on CCD morning." + +ccd.pr_reminder: + summary: "Teams reminder to the 'Code reviews' chat: merge required PRs before the 11 PM branch cut." + what: > + The morning-of-CCD Teams message telling engineers every required PR must be + merged today before the release branch is cut automatically at 11:00 PM; if a + required PR won't make it, request a CCD delay or a post-CCD cherry-pick; and + merge any PR with user-facing strings before the localization pipeline runs at + noon. + who: > + Scout posts it automatically (a scout step) on CCD morning. CCD-delay and + cherry-pick requests must be approved by Moumita Ghosh (moghosh@microsoft.com). + where: + - "Target: the 'Code reviews' Teams chat (fixed chat id in the step CONFIG / EXTERNAL-REFERENCES.md)." + how: > + Runs via step-action → workiq_send_chat_message. To test, redirect with the + send_to mock knob ('me' = your own chat). + faqs: + - q: "Who approves a CCD delay or cherry-pick?" + a: "Moumita Ghosh (moghosh@microsoft.com) — both a Code Complete delay and a post-CCD cherry-pick require her approval." + - q: "Why merge string PRs before noon?" + a: "The localization pipeline is triggered at noon; strings merged after it aren't guaranteed to be localized this release." + +ccd.localization: + # Knowledge for this step is co-located in the step module (steps/ccd/localization.py + # KNOWLEDGE) — the trigger→poll→timeout/complete→post-PR flow. Kept there so it + # can't drift from the logic. No yaml override needed. + +preflight.wiki: + summary: "Create the per-release payload wiki subpage under the history parent page." + what: > + Creates ' Release' under the standing Monthly Releases Payloads + History page; Phase 2 later fills in the built versions. Duplicate-safe: if the + month's page already exists it's left untouched and a numbered page is created. + how: "Runs automatically; the created page link is stored on the step and shown in the status Details column." + +# ======================= Phase 2 · build_verify (RC verification) ======================= +# Read-only checks over the Engineering release pipelines: the checker (3038) fired the +# orchestrator (2828), the orchestrator is healthy + parked at Remove RC Tags, and both +# MRWP (2519) RC-testing runs (ECS + Local) ran to completion. Each blocks with the +# recovery TSG + escalation chat. + +build_verify.checker_fired: + summary: "Confirm the Code Complete Calendar Checker fired the release on the CCD." + what: > + The checker (def 3038) runs daily; its 'Trigger Monthly Release' job is SKIPPED on + ordinary days and only runs (succeeds) on the actual Code Complete Day, when it + launches the Release Orchestrator. This step scans the month's checker runs for the + one whose 'Trigger Monthly Release' job succeeded. + where: + - "Code Complete Calendar Checker (def 3038): https://identitydivision.visualstudio.com/Engineering/_build?definitionId=3038" + how: > + If it blocks: before Code Complete Day this is EXPECTED (nothing has triggered yet). + After the CCD, a missing/failed 'Trigger Monthly Release' means the release wasn't + kicked off — check the checker's latest run and its schedule, fix, and rerun the step. + faqs: + - q: "It's blocked but the CCD hasn't arrived — is that a problem?" + a: "No. The trigger job only runs on Code Complete Day; before then there's nothing to confirm. It clears once the checker fires the release." + +build_verify.orchestrator_health: + summary: "Confirm the Release Orchestrator is healthy and parked at the Remove RC Tags gate." + what: > + Finds this release's orchestrator run (def 2828) by its AuthenticatorBranch tag and + confirms the pre-gate stages — Validate Branch and Versions, Create Release Branches, + Trigger RC Testing — all succeeded, and that it is PAUSED at 'Remove RC Tags' (a manual + approval the owner clears in a LATER phase). This step never approves that gate. Reports + the RC versions (Common/MSAL/Broker). + who: > + Scout verifies it automatically (read-only). If a pre-gate stage failed, the RELEASE + ENGINEER follows the failed stage's guidance + the recovery TSG, retries the stage, then + reruns this step. Approving Remove RC Tags is a separate, later step — not here. + where: + - "Release Orchestrator (def 2828): https://identitydivision.visualstudio.com/Engineering/_build?definitionId=2828" + - "Recovery TSG: https://eng.ms/docs/microsoft-security/identity/entra-developer-application-platform/auth-client/authn-sdk-msal-android/android-auth-libraries/releases/internal-release-checklist/release-orchestrator-recovery" + how: > + Each stage is descriptive — the failed stage's output explains the root cause and + suggested fix. Follow it, click Retry on the failed stage, then rerun this step. If + unresolved within 2h, escalate to the release Teams chat. + faqs: + - q: "It says 'parked at Remove RC Tags' — is that bad?" + a: "No — that's the healthy expected state. The orchestrator pauses at Remove RC Tags for a human approval that happens in a later phase. Phase 2 only confirms it got there cleanly; it never approves it." + - q: "A pre-gate stage failed — what do I do?" + a: "Open the failed stage; its output describes the root cause + corrective action. Fix it, click Retry on that stage, then rerun this step. Recovery steps: the release-orchestrator-recovery TSG. Escalate to the release chat if stuck > 2h." + +build_verify.mrwp_ecs: + summary: "Confirm the ECS Monthly Release Work Pipeline run ran to completion + report its tests." + what: > + The orchestrator triggers MRWP (def 2519) twice — once per flight provider. This checks + the ECS run 'ran to completion': EVERY stage executed (state completed, result + succeeded/succeededWithIssues/failed). Red (failed) and yellow (warnings) stages are + acceptable — they're triaged later. The ONLY blocker is a stage that never ran (pending, + skipped, or canceled — i.e. the pipeline aborted partway). Also reports the Test-tab + results (unit / instrumented / UI-automation totals). + where: + - "Monthly Release Work Pipeline (def 2519): https://identitydivision.visualstudio.com/Engineering/_build?definitionId=2519" + how: > + If it blocks, a stage never ran — the pipeline aborted. Open the run, find the stage that + didn't execute, follow its output + the recovery TSG, Retry, then rerun this step. Failing + tests do NOT block here; they're triaged in the bug-bash phase. + faqs: + - q: "The run has red stages / failing tests — why did it still pass?" + a: "By design. A run is valid as long as every stage EXECUTED. Red/yellow stages and failed tests are expected at this checkpoint and triaged later. Only a stage that never ran (aborted pipeline) blocks." + - q: "How is the ECS run identified?" + a: "From the orchestrator run's RC-ECS= tag (or, for runs predating that tag, the orchestrator's trigger-task log). The run is then read directly by build id." + +build_verify.mrwp_local: + summary: "Confirm the Local-flighting MRWP run ran to completion + report its tests." + what: > + Local-flighting counterpart to the ECS check: confirms the Local MRWP (def 2519) run 'ran + to completion' (every stage executed; skipped/canceled/pending = block) and reports its + Test-tab results. Red/yellow stages and failed tests do NOT block — triaged later. + where: + - "Monthly Release Work Pipeline (def 2519): https://identitydivision.visualstudio.com/Engineering/_build?definitionId=2519" + how: > + Same as the ECS check: only a stage that never ran (aborted pipeline) blocks. Follow the + aborted stage's output + the recovery TSG, Retry, rerun the step. Failing tests are triaged + in the bug-bash phase. + faqs: + - q: "What's the difference between the ECS and Local runs?" + a: "The orchestrator triggers MRWP twice with different flight providers — ECS (server-driven flights) and Local (local flight overrides). Both must run to completion; they're verified separately." + +build_verify.rc_report: + summary: "Email the RC verification report + apply the three-tier 90% UI gate (the Phase-2 go/no-go)." + what: > + Once the four verification steps have resolved the chain, this composes the Phase-2 RC report — + checker fired, orchestrator healthy/parked, both MRWP runs' stage completion, and each run's + failing test suites — from LIVE pipeline data and emails it to the release owner (on CCD+1). It + is the terminal Phase-2 step and the go/no-go: after sending, `record-rc-report` applies a + three-tier gate on the combined UI-automation pass rate across both MRWP runs — 100% is a clean + pass, >=90% passes with a warning (investigate the failing tests in parallel, bug bash not + blocked), and <90% BLOCKS for owner investigation. There is no separate approval gate. + Scout-assisted (source: scout): the engine composes it; the skill sends via WorkIQ. + who: > + Scout composes the email; the skill sends it via workiq_send_email to the release owner + (state.owner_email). Redirect for testing with the step's `send_to` payload knob. + where: + - "Release Orchestrator run (versions + parked gate): via the run link in the email + the step's Details" + - "MRWP ECS / Local runs (Test tab): via the run links in the email + the step's Details" + how: > + If it blocks with 'no release owner email', set it with `set-owner --email ` + and rerun. On a clean/warn pass (>=90% UI) the release auto-advances into bug bash. On a <90% + block, the owner investigates and rules on it (patch a real bug + re-trigger RC, or proceed as an + automation flake): fix + re-run then `next`, or `skip --reason` to override. + faqs: + - q: "Who gets the email?" + a: "The release owner on record (owner_email). It's a personal review copy, not the broad DL — the owner reviews the RC failures; the 90% UI gate then decides go/no-go automatically." + - q: "Does the sim send this email?" + a: "No. In a sim the step is marked done without sending; only the real skill flow composes + sends it via WorkIQ." + +# =============================== Phase 3 · bug_bash =============================== + +bug_bash.clone_plans_broker: + summary: "Copy the Broker master test plan to a new plan for this release." + what: > + Each release COPIES the Broker master template (ADO test plan #2007357, area + 'Engineering\\Auth Client\\Broker\\Android') to a new plan named + 'Android Monthly Release - ' (e.g. 'Android Monthly Release - Aug 2026'), + REFERENCING the existing test cases (the ADO clone default — it shares test cases, + it does not duplicate them). This is the Broker half of the old clone_plans step. + who: "Scout runs it automatically during `next` (an agent step, via the ADO test-plan clone API)." + how: > + Idempotent — the new plan id is stashed on the step; a re-run re-confirms that plan + exists and reports done without cloning again. If the clone fails (auth/API), the + step blocks with the ADO error; fix and `next`, or `skip --reason` to override. + links: + - name: "Broker master test plan (#2007357)" + url: "https://identitydivision.visualstudio.com/Engineering/_testPlans/define?planId=2007357&suiteId=2008656" + - name: "Broker test-plans doc" + url: "https://eng.ms/docs/microsoft-security/identity/entra-developer-application-platform/auth-client/authn-sdk-msal-android/android-auth-libraries/releases/internal-release-checklist/test-plans" + faqs: + - q: "Does it duplicate the test cases?" + a: "No — it references the existing test cases (the 'Reference existing test cases' clone option). Edit test cases only in the master template." + +bug_bash.clone_plans_auth: + summary: "Create the Authenticator bug-bash query-based test suite for this release." + what: > + Creates a NEW query-based (dynamic) test suite under the standing 'MSAuthenticator + Test Passes' plan (#714514, root suite 714515), named 'Android release/MM/DD/YYYY' + from the CCD (e.g. CCD 2026-08-13 -> 'Android release/08/13/2026'). Its WIQL selects the Android bug-bash test cases + (tag 'Android' + 'ReleaseBugBash', area 'Engineering\\ISP\\Identity Apps', not Closed + — matches the current prod suite 3728419 'Android release/08/13/2026'). This is the + Authenticator half of the old clone_plans step. It STOPS after creating the suite — + assigning testers is a later, manual step. + who: "Scout runs it automatically during `next` (an agent step, via the ADO test-suite create API)." + how: > + Idempotent — the created suite id is stashed on the step, and it also reuses an + existing same-named suite rather than making a duplicate. If the create fails + (auth/API), the step blocks with the ADO error; fix and `next`, or `skip --reason`. + links: + - name: "MSAuthenticator Test Passes plan (#714514)" + url: "https://identitydivision.visualstudio.com/Engineering/_testManagement?_a=tests&planId=714514&suiteId=714515" + - name: "How to make a test suite for bug bash (IDWiki 33580)" + url: "https://identitydivision.visualstudio.com/IdentityWiki/_wiki/wikis/IdentityWiki.wiki/33580/How-to-make-test-suite-for-bug-bash-" + faqs: + - q: "Does it assign testers?" + a: "No — it stops after creating the query-based suite. Assigning testers is done later (a separate, manual step)." + - q: "What names the suite?" + a: "From the CCD: 'Android release/MM/DD/YYYY' (e.g. CCD 2026-08-13 → 'Android release/08/13/2026'), matching prod." + - q: "Which test cases does it include?" + a: "The curated bug-bash set — Android test cases tagged 'ReleaseBugBash' (the tag also matches month-specific 'ReleaseBugBash' tags), not Closed. This matches prod; a broad 'all Android' filter would over-include (e.g. ComposeTesting/DarkMode cases)." + +# ========================== Entry gate · readiness ========================== +# Keyed "readiness.". Queried via `gate-info --item `. Same fields as the +# step entries (summary/what/who/where/how/links/faqs). + +readiness.build_access: + summary: "You have permission to both ADO orgs the release builds from." + what: > + Confirms you can reach the two build definitions this release drives — the + BROKER pipelines in Engineering (Auth Client Android #2828, org + identitydivision) and the AUTHENTICATOR pipelines in One (Android Build Release + #397224, org msazure). Auto-verified via `az pipelines build definition show`; + passes only if BOTH are accessible. + who: > + Scout verifies it automatically (a Python verifier — no user action) as long as + you're signed into az with access to both orgs. + where: + - "Broker (Engineering #2828): https://identitydivision.visualstudio.com/Engineering/_build?definitionId=2828" + - "Authenticator (One #397224): https://msazure.visualstudio.com/One/_build?definitionId=397224" + how: > + If it fails: run `az login`, then confirm you have access to BOTH orgs/projects + — Engineering (identitydivision) for the Broker pipelines and One (msazure) for + the Authenticator pipelines. The two orgs authenticate separately, so a token + good for one may not cover the other; request access to whichever org is + failing. Re-run the gate once fixed. + faqs: + - q: "Why check two organizations?" + a: "The Android release spans both: the Broker libraries build in Engineering (identitydivision) and the Authenticator app builds in One (msazure). You need permission to both." + - q: "It failed with a 401 — what now?" + a: "Usually you're not signed into az, or you lack access to one of the two orgs (they're separate). Sign in / request access to the failing org, then re-run the gate." + +readiness.ccd_confirmed: + summary: "The Code Complete Date is a valid future date, reconciled with the pipeline." + what: > + Validates the CCD end-to-end, because every phase is CCD-relative and a wrong + CCD misaligns the whole run. Two hard checks (either BLOCKS): the CCD must be + FUTURE-DATED — a past CCD (e.g. a current-month release whose 2nd-Wednesday + default already passed) can't be code-completed — and it must AGREE with + pipeline 3038's override. Plus one advisory: if the CCD leaves Phase 0 + (CCD-7→CCD) with less than its normal 7-day window, Scout WARNS about the + compressed prep window but lets you proceed. + who: > + Scout runs the check (`check-ccd`, reads the pipeline via az). On a past CCD or a + differing pipeline override, Scout shows you the options and YOU decide the real + date; Scout writes it with `set-ccd`. On a compressed Phase 0 window Scout warns + and you choose to proceed or move the CCD out. If the pipeline can't be read, you + confirm the CCD manually (attest fallback). + where: + - "CCD override variable — Broker pipeline (Engineering #3038): https://identitydivision.visualstudio.com/Engineering/_build?definitionId=3038" + how: > + Scout runs `check-ccd --json` and acts on the status: `past` → the CCD is before + today; reschedule to a future date with `set-ccd --date --confirm` + (adopt a future pipeline override if one exists), re-check. `conflict` → pipeline + override differs; pick which is right, `set-ccd`, re-check. `unreadable` → confirm + the CCD by hand (attest). `match` → record the item pass. On `compressed: true` + (any non-past status) Scout warns that Phase 0 has only `runway_days` of the + normal 7 and lets you proceed or move the CCD. + faqs: + - q: "I started a release this month and the CCD is in the past — why?" + a: "The default CCD is the 2nd Wednesday of the release month. If that day has already passed, the default is in the past and invalid. The gate blocks until you set a future Code Complete Date (or target a later release month)." + - q: "Why does a CCD mismatch block the release?" + a: "The phase schedule is computed from the CCD. If the pipeline is building to a different date than the orchestrator thinks, every reminder, gate, and deadline is off. Reconciling once up front keeps them in lockstep." + - q: "My CCD is only a couple of days out and Scout warned about Phase 0 — is that a blocker?" + a: "No. Phase 0 normally runs CCD-7→CCD (7 days). A nearer CCD squeezes that window, so Scout flags how many prep days remain, but you can still proceed (e.g. for a hotfix) or move the CCD out with set-ccd." + - q: "The pipeline has no override — is that a problem?" + a: "No. No override means the pipeline uses the same 2nd-Wednesday default the orchestrator does, so they already agree — that's a `match` (as long as the date is still in the future)." + - q: "Scout couldn't read the pipeline — can I still proceed?" + a: "Yes. On an unreadable pipeline the item falls back to attestation: you confirm the CCD manually and Scout records it with a note. It never silently passes." + +readiness.mcp_servers: + summary: "The MCP servers the release needs are registered in Scout." + what: > + Confirms the command-based MCP servers this release depends on are registered in + your Scout config — ICM (on-call / incident lookups), Kusto/ADX (telemetry + the + adx_access check), and Teams (comms gaps WorkIQ can't fill: group chats, channel + posts). Auto-verified read-only against config/requirements.yaml; passes only if + ALL are present. + who: > + Scout verifies it automatically (a Python verifier). Fixing a missing server is a + one-time bootstrap step (register + restart Scout), not something the release does. + where: + - "Scout's registered servers: ~/.scout/m-mcp-servers.json" + - "The required-servers manifest: config/requirements.yaml" + how: > + If it fails: run `python -m orchestrator.cli infra` (or setup/bootstrap.ps1) to + auto-register the missing server(s), then RESTART Scout so it loads them, and + re-run the gate. These are provided by the Agency CLI (agency mcp icm|kusto|teams); + if Agency isn't installed, install it first. infra won't register a server whose + launcher is missing. + faqs: + - q: "What happens if a server is missing?" + a: "infra registers it (backing up your config first) and tells you to restart Scout. It refuses to register a server whose launcher isn't installed." + - q: "Why is the Teams MCP required now?" + a: "To cover Teams gaps WorkIQ can't (group-chat creation, adding members, rich channel messaging) — WorkIQ is 1:1-only." + - q: "What's the empty-tools gotcha?" + a: "Command servers must ship a populated `tools` allowlist; an empty one makes Scout silently drop the server. Our requirements.yaml ships icm/kusto/teams with populated lists." + +readiness.silent_perms: + summary: "Scout's permissions allow fully unattended (silent) runs." + what: > + Confirms shell, WorkIQ (email + Teams), the browser (Playwright), and the ICM + + Kusto MCP servers are all auto-approved in Scout settings — so scheduled / + background work (the daily digest, Teams reminders, CCOA-lockdown browser checks, + the on-call & telemetry checks) runs WITHOUT stalling on a permission prompt when + Scout isn't focused. + who: > + The skill checks it (reads m_get_settings — the engine can't see Scout's own + settings). Enabling requires the user to flip one master toggle (only the user + can). OPT-OUT: you may proceed without. + where: + - "Scout → Settings → Permissions — the per-server auto-approve flags." + - "The master toggle: 'When blocked, let Scout ask instead of stopping' (allowModelPermissionsChange)." + how: > + If not all auto-approved, the skill offers 'Enable silent runs' — it requests + auto-approval for WorkIQ/browser/ICM/Kusto (you click Allow once); if the master + toggle is off you turn it on first. Or choose 'Proceed without' → recorded as + `degraded`, which STILL clears the gate (⚠️ 'Proceeding (not silent)') but + background checks will prompt and can stall until you open Scout. + faqs: + - q: "Does this block the release?" + a: "No — it's soft/opt-out. `degraded` satisfies the gate; you just risk occasional prompts on unattended runs." + - q: "Exactly what gets auto-approved?" + a: "shell, workiq, playwright, kusto, icm." + +readiness.teams_notify: + summary: "The daily digest can also reach you over the Scout Teams bot." + what: > + Confirms the Teams relay is connected and the Scout bot is reachable, so the + hourly `tick` digest can be delivered to you in Teams IN ADDITION to email. Email + is the guaranteed channel; Teams is a bonus. + who: > + The skill checks it (m_relay_status / m_relay_connect + a silent + m_send_teams_message handshake — the engine can't reach the relay). OPT-OUT. + where: + - "Channels are configured in config/notifications.yaml (channels.teams, teams.target: scout)." + - "The Scout bot: open the Microsoft Scout chat in Teams." + how: > + If Teams is enabled in notifications.yaml, the skill connects the relay and sends + a handshake. If the relay won't connect or the bot has no conversation yet (you've + never messaged it), it records `degraded` (email-only) — which STILL clears the + gate. To enable Teams later: open the Microsoft Scout chat in Teams and send it + any message once, then re-run. + faqs: + - q: "Does this block the release?" + a: "No — opt-out. If Teams is unreachable it records `degraded` and you get the digest by email only." + - q: "How do I turn Teams delivery off?" + a: "Set channels.teams: false in config/notifications.yaml — then this item passes as 'not requested'." + +readiness.adx_access: + summary: "You can query the ADX release-telemetry cluster." + what: > + Confirms you can query the ADX release dashboard's backing Kusto cluster (used for + monitoring / Play vitals / crash metrics later in the release). Scout runs a + `print 1` probe against the cluster. + who: > + The skill runs it (Kusto MCP — the engine can't reach the MCP) and records + pass/fail. Fully verified (not attested). + where: + - "ADX release dashboard: https://dataexplorer.azure.com/dashboards/ab7abd7e-c36f-4ff0-88a7-aaa3ec014bd7" + - "Backing cluster: https://idsharedeus2.eastus2.kusto.windows.net (db d496be22d62a46b0a3cf67ea2e736fd8)" + how: > + If it fails, you lack access to the cluster (or the Kusto MCP isn't reachable). + Request access to the ADX cluster / dashboard, confirm the Kusto MCP is registered + (see mcp_servers), then re-run the gate. + faqs: + - q: "What is the probe?" + a: "A trivial `print 1` query against the cluster — it only proves you can connect and run a query, nothing about the data." + +readiness.oncall_now: + summary: "You are not the current Android primary on-call." + what: > + Confirms you are NOT the current primary on-call (OCE) for 'Auth Client Android + Shield' (ICM team 78848) — a primary on-call shouldn't also be driving a release. + A backup/secondary is fine; only the current PRIMARY blocks. + who: > + The skill checks it via ICM (get_on_call_schedule_by_team_id, team 78848 — the + engine can't reach the MCP) and records pass/fail. + where: + - "On-call schedule: https://aka.ms/androidoncall (team: Auth Client Android Shield, ICM 78848)." + how: > + If you ARE the current primary, hand the release to someone who isn't (or wait + until your shift ends), then re-run. Being a backup/secondary does NOT block — the + skill records those as pass. + faqs: + - q: "I'm the backup on-call — does that block me?" + a: "No. Only the current PRIMARY (first-listed current OCE) blocks. Backups are free to run the release." + - q: "Why can't the primary on-call run the release?" + a: "Primary on-call is expected to be responsive to incidents; combining it with release duty risks both. It's a required (non-opt-out) check." + +readiness.play_console_access: + summary: "You can open the Google Play Console app dashboard." + what: > + You ATTEST (Scout can't verify — Google auth wall) that you can sign into the + Play Console and load the Authenticator app dashboard. Needed later for the Play + vitals / policy-status review (Phase 0 `vitals`) and rollout management. + who: "You confirm it — Scout cannot prove Google sign-in." + where: + - "Play Console app dashboard: https://play.google.com/console/u/0/developers/6720847872553662727/app/4972501392087484764/app-dashboard" + how: > + Open the dashboard and confirm it loads (you're signed into the right Google + account with access to this app), then attest. If you can't load it, request Play + Console access following the how-to doc below. + links: + - name: "How to access Google Play Console" + url: "https://eng.ms/docs/microsoft-security/identity/entra-developer-application-platform/auth-client/authn-sdk-msal-android/android-auth-libraries/releases/combined-release-checklist/how-to-access-google-play-console" + faqs: + - q: "Why can't Scout verify this?" + a: "Play Console is behind Google auth, which Scout can't automate — so you attest." + - q: "How do I get access?" + a: "Follow the 'How to access Google Play Console' doc (linked). Then open the dashboard to confirm." + +readiness.oncall_window: + summary: "You're not scheduled on-call during the release window." + what: > + You ATTEST you are NOT scheduled Android on-call at any point across the release + window — CCD-7 through CCD+14 (the full stabilization period). This is the future + counterpart to oncall_now (which only checks today); Scout can only see the + current rotation, so you confirm the upcoming window. + who: "You confirm it — Scout can't reliably read the future rotation." + where: + - "On-call schedule: https://aka.ms/androidoncall (team: Auth Client Android Shield). Check your shifts across the window." + how: > + Open the schedule, verify none of your on-call shifts fall in CCD-7 … CCD+14, then + attest. If you ARE scheduled in that window, arrange coverage / a swap, or hand off + the release. + faqs: + - q: "Why is this separate from 'Not on-call now'?" + a: "oncall_now is auto (Scout reads today's roster); this is attest because Scout can't see the future schedule reliably." + - q: "What's the window?" + a: "CCD-7 to CCD+14 — pre-flight through the rollout / monitoring tail." + +readiness.saw_ame: + summary: "Your SAW machine and AME account are accessible." + what: > + You ATTEST you can log into your SAW (Secure Admin Workstation), the desktop + loads, and you can sign in with your AME (Azure Managed Environment) credentials — + the secure environment required for privileged release actions. + who: "You confirm it — Scout can't verify SAW/AME sign-in." + where: + - "SAW: https://aka.ms/saw" + how: > + Log into SAW (link below), confirm the desktop loads, and sign in with your AME + credentials, then attest. If SAW or AME isn't working, resolve access via the SAW + portal before starting the release. + links: + - name: "SAW" + url: "https://aka.ms/saw" + faqs: + - q: "Why is SAW/AME needed?" + a: "Privileged release steps must run from the secure admin workstation with AME credentials — Scout can't prove that, so you attest." + +readiness.yubikey: + summary: "You have a YubiKey in hand." + what: > + You ATTEST you physically have your YubiKey — the hardware security key required + for the release's privileged sign-ins. Scout can't verify a physical device, so + you confirm. + who: "You confirm it — it's a physical device." + where: + - "No YubiKey? Pick one up at Studio A 1909, the Bldg 34/35 cafe, or order a DSR USB-C thumbdrive (link below)." + how: > + Have your YubiKey with you, then attest. If you don't have one, get one first + (Studio A 1909 / Bldg 34-35 cafe) or order a DSR USB-C security key via the link. + links: + - name: "Order a DSR USB-C security key" + url: "https://cloudmfa-support.azurewebsites.net/SecurityKeyServices/SecurityKey" + faqs: + - q: "I don't have a YubiKey — where do I get one?" + a: "Studio A 1909, the Bldg 34/35 cafe, or order a DSR USB-C thumbdrive (linked)." diff --git a/release-agent/config/notifications.yaml b/release-agent/config/notifications.yaml new file mode 100644 index 00000000..c639949a --- /dev/null +++ b/release-agent/config/notifications.yaml @@ -0,0 +1,31 @@ +# Release digest delivery channels — DATA (not code) for how the daily `tick` +# digest reaches the release owner. The engine builds ONE digest; this file says +# where it goes. Read by `tick`/`notify`, which add a `channels` map and (when a +# digest is due) a `teams` delivery descriptor to their payload. +# +# Purpose: keep the RELEASE OWNER (the engineer running the release) aware of status +# and pull them in when a step needs them. Broadcasting to anyone else only happens +# when a specific step requires it (that's the step-driving automations' job, e.g. +# the CCD reminders) — NOT this digest. +# +# Fields: +# channels.email : email the digest to owner_email (existing behavior). +# channels.teams : ALSO deliver the digest over Teams. +# teams.target : WHERE the Teams copy goes — +# 'scout' (default) → the Scout Teams bot DM, delivered with +# m_send_teams_message (the plain-text +# digest). This is the release owner's +# Scout notification channel. +# → an explicit Teams chat, delivered with +# workiq_send_chat_message (rich HTML). Use +# only when you deliberately want a shared +# chat instead of the Scout bot. + +version: 1 + +channels: + email: true + teams: true + +teams: + target: scout diff --git a/release-agent/config/phases.yaml b/release-agent/config/phases.yaml new file mode 100644 index 00000000..96675dd9 --- /dev/null +++ b/release-agent/config/phases.yaml @@ -0,0 +1,135 @@ +# Release Orchestrator — state machine definition (data, not code). +# The conductor reads this to know phases -> steps -> gates -> transitions. +# This file is pure COMPOSITION: which steps run, in what order, their deps and +# flags. A step's LOGIC lives in its co-located module `steps//.py` +# (auto-discovered); steps without a module yet run the generic stub. +# +# Field reference: +# phases[].id/name : phase identifier + label +# phases[].checklist_phase : maps to the human checklist Phase number +# phases[].execution : sequential (default) | parallel (independent steps) +# steps[].id/name : step identifier + label (id matches the step module's ID) +# steps[].gate : if true, conductor HOLDS at this step for human approval +# steps[].owner : agent | human (who acts; gates are always human-decided) +# steps[].source : scout -> the skill runs it via MCP + record-step +# steps[].attest : if true, a human confirms it (owner: human) +# steps[].depends_on : step ids in this phase that must finish first +# steps[].maps_to : the Release-Stabilization action-item ID(s) this step uses + +version: 1 + +phases: + - id: preflight + name: "Pre-flight & Code Complete" + checklist_phase: 0 + anchor: "CCD-7" # opens 7 days before the Code Complete Date (start window) + execution: parallel # steps are independent; a hold on one doesn't block the others + steps: + - { id: notice, name: "Send early release notice", owner: agent, source: scout, maps_to: [S0] } + - { id: flight_reminder, name: "Send feature-owner reminders (flights · strings · flag-freeze)", owner: agent, source: scout, maps_to: [S3, S4, S5] } + - { id: confirm_reminders, name: "Confirm feature owners completed flight / string / flag-freeze work", owner: human, attest: true, depends_on: [flight_reminder], maps_to: [S3, S4, S5, S6] } + - { id: lockdown, name: "Detect lockdown/holiday overlap", owner: agent, source: scout, maps_to: [S1] } + - { id: breaking, name: "Detect BREAKING-OneAuth + draft comms", owner: agent, maps_to: [S2] } + - { id: cg, name: "Report critical CG alerts", owner: agent, maps_to: [S8] } + - { id: vitals, name: "Confirm Play Console vitals & policy status reviewed", owner: human, attest: true, maps_to: [S9] } + - { id: cron, name: "Verify Calendar Checker scheduled", owner: agent, maps_to: [S10] } + - { id: wiki, name: "Create release payload wiki subpage", owner: agent, maps_to: [S0] } + + - id: ccd + name: "Code Complete Day" + checklist_phase: 1 + anchor: "CCD" # opens ON the Code Complete Date — these are CCD-DAY comms; must not fire early + steps: + - { id: final_reminder, name: "CCD final code-complete reminder (email)", owner: agent, source: scout, maps_to: [P1-1] } + - { id: pr_reminder, name: "Remind Code Reviews chat of the CCD PR-merge deadline", owner: agent, source: scout, maps_to: [P1-1b] } + - { id: localization, name: "Trigger localization pipeline (PR)", owner: agent, source: scout, maps_to: [P1-2] } + + - id: build_verify + name: "Build & Lib Verification" + checklist_phase: 2 + anchor: "CCD+1" # opens the day AFTER Code Complete — the engineer wakes to a resume + show_pipeline_runs: true # surface the RC pipeline-run one-liner (checker/orchestrator/MRWP) in status + digest + steps: + - { id: checker_fired, name: "Verify Code Complete Checker fired the release", owner: agent, maps_to: [B0] } + - { id: orchestrator_health, name: "Verify Release Orchestrator health (parked at Remove RC Tags)", owner: agent, maps_to: [B1] } + - { id: mrwp_ecs, name: "Verify MRWP (ECS) ran to completion + tests", owner: agent, maps_to: [B2] } + - { id: mrwp_local, name: "Verify MRWP (Local) ran to completion + tests", owner: agent, maps_to: [B3] } + # rc_report is the Phase-2 go/no-go: it emails the RC report AND applies the 90% UI + # gate — clean/warn (>=90%) proceed to bug bash; <90% BLOCKS for owner investigation. + # No separate human approval step: the gate IS the decision. + - { id: rc_report, name: "Email RC verification report + apply the 90% UI gate", owner: agent, source: scout, maps_to: [B4] } + + - id: bug_bash + name: "Test / Bug Bash" + checklist_phase: 3 + steps: + - { id: clone_plans_broker, name: "Clone Broker test plan (copy master template)", owner: agent, maps_to: [T3] } + - { id: clone_plans_auth, name: "Create Authenticator bug-bash test suite", owner: agent, maps_to: [T3] } + - { id: coordinate, name: "Bug Bash coordinator (invite/monitor/aggregate)", owner: agent, maps_to: [T2] } + - { id: ui_failures, name: "Surface Phase 2 UI failure list", owner: human, maps_to: [T4] } + - { id: signoffs, name: "Chase DID/Dublin sign-offs + telemetry", owner: agent, maps_to: [T5] } + - { id: bash_done, name: "Bug bash complete + signed off", owner: human, gate: true } + + - id: finalize + name: "Finalize & Publish" + checklist_phase: 4 + steps: + - { id: gate_watch, name: "Watch orchestrator gates (1-click approve)", owner: human, gate: true, maps_to: [F1] } + - { id: integ_prs, name: "Auto-create integration PRs", owner: agent, maps_to: [F2] } + - { id: verify_pub, name: "Verify Maven Central + GitHub publication", owner: agent, maps_to: [F3] } + - { id: final_comms, name: "Send final broker email + Teams", owner: agent, maps_to: [F4] } + - { id: nonrc_pin, name: "Final non-RC pin PR + kick + retain", owner: agent, maps_to: [F5] } + - { id: tag, name: "Tag the release commit", owner: agent, maps_to: [F6] } + - { id: backmerge, name: "Back-merge to working", owner: agent, maps_to: [F7] } + + - id: rollout_start + name: "Rollout Start" + checklist_phase: 5 + steps: + - { id: notice, name: "Send initial Auth App release notice", owner: agent, maps_to: [R1] } + - { id: artifact, name: "Verify final non-RC artifact is used", owner: agent, maps_to: [R2] } + - { id: signoff_start, name: "Start Release Sign Off (1-click)", owner: human, gate: true, maps_to: [R3] } + + - id: monitor + name: "Monitoring & Ring Advancement" + checklist_phase: 6 + steps: + - { id: health_report, name: "Crash-% health report (halt/hotfix/proceed)", owner: agent, maps_to: [M1] } + - { id: adoption, name: "Adoption % vs ring threshold", owner: agent, maps_to: [M2] } + - { id: guards, name: "Check advancement guard rules", owner: agent, maps_to: [M4] } + - { id: advance, name: "Ring advance / halt decision", owner: human, gate: true, maps_to: [M3] } + - { id: progress_email, name: "Send progression email on advance", owner: agent, maps_to: [M5] } + - { id: dashboard, name: "Update ADX dashboard params", owner: agent, maps_to: [M6] } + - { id: crash_annot, name: "Flag new vs existing crashes", owner: human, maps_to: [M7] } + - { id: hotfix_eval, name: "Surface hotfix-trigger criteria", owner: human, maps_to: [M8] } + + - id: partner + name: "Partner Stores" + checklist_phase: 7 + steps: + - { id: china, name: "China publish flow", owner: human, maps_to: [PT1] } + - { id: samsung, name: "Samsung portal upload", owner: human, maps_to: [PT1s] } + - { id: ngms, name: "Verify nGMS upload", owner: agent, maps_to: [PT2] } + - { id: teams_dev, name: "Teams Devices announcement", owner: agent, maps_to: [PT3] } + - { id: approvals, name: "Track partner approvals + final notice", owner: agent, maps_to: [PT4] } + - { id: review_times, name: "Maintain review-times table", owner: agent, maps_to: [PT5] } + + - id: hotfix + name: "Hotfix (conditional)" + checklist_phase: 8 + conditional: true + steps: + - { id: cherry, name: "Cherry-pick PRs", owner: agent, maps_to: [H1] } + - { id: rebuild, name: "Rebuild + retain hotfix build", owner: agent, maps_to: [H2] } + - { id: smoke, name: "P0 smoke subset", owner: human, gate: true, maps_to: [H3] } + - { id: expedite, name: "Expedited rollout", owner: agent, maps_to: [H4] } + - { id: alpha_orgs, name: "Track alpha-mitigation orgs to remove", owner: agent, maps_to: [H5] } + + - id: close + name: "Release Close" + checklist_phase: 9 + steps: + - { id: closeout, name: "Close-out checklist (approvals/comms/bugs)", owner: agent, maps_to: [C1] } + - { id: done, name: "Release complete", owner: human, gate: true } + + diff --git a/release-agent/config/readiness.yaml b/release-agent/config/readiness.yaml new file mode 100644 index 00000000..39d3eeea --- /dev/null +++ b/release-agent/config/readiness.yaml @@ -0,0 +1,151 @@ +# Release readiness checklist — the ENTRY GATE. +# The engine blocks (status = readiness_gate) until EVERY item is satisfied. +# All items are equally required — there is no per-item priority. +# +# The ONLY distinction is WHO resolves an item (the `verify` field): +# auto -> Scout resolves it (verifies programmatically). Result is pass or fail; +# if it can't be fully proven, it is NOT auto. Two execution sources: +# * (default) a Python verifier in phases/readiness_verifiers.py (uses az/http) +# * source: scout -> the SKILL runs the check via its MCP tools (e.g. ICM) +# and records the result with `record-check`; the Python engine skips it. +# attest -> The engineer resolves it (confirms). Used for anything Scout cannot +# fully prove (physical devices, portal sign-in). +# +# If any item is unsatisfied the gate stays closed. If the engineer cannot satisfy +# an attest item, they resolve it or hand the release to someone who can. +# +# Edit this file to change the entry checklist — it is data, not code. + +title: "Release readiness — entry gate" +instructions: > + Clear every item before Phase 0 can start. Scout verifies the auto items; you + attest the rest. Every item is required — the release cannot start until all are + satisfied. +blocked_message: > + This item is required and is not satisfied, so the release cannot start. Resolve + it — or, if you cannot, hand the release to another engineer who can (notify your + manager / the release team). The new owner runs /release-agent and completes the + checklist themselves. + +items: + # ---- AUTO: fully verified by Scout (pass/fail, no half-measures) ---- + # `label` = short name for the table; `detail` = PLAIN cell text (no inline links — + # markdown links don't render inside Scout table cells). Links go in `links` + # (list of {name,url}) and are rendered as a clickable reference list BELOW the table. + # For build_access the detail + links are generated from the live check results. + - id: build_access + label: "Build definitions accessible" + text: "Release build definitions are accessible" + verify: auto + verifier: build_defs + checks: + - type: ado_build_def + name: "Auth Client Android (Engineering #2828)" + org: "https://identitydivision.visualstudio.com/" + project: "Engineering" + id: 2828 + url: "https://identitydivision.visualstudio.com/Engineering/_build?definitionId=2828" + - type: ado_build_def + name: "Android Build Release (One #397224)" + org: "https://msazure.visualstudio.com/" + project: "One" + id: 397224 + url: "https://msazure.visualstudio.com/One/_build?definitionId=397224" + + # Everything the tool needs to run UNATTENDED (machine on, Scout not focused): + # the MCP servers registered, and the permissions set so scheduled work (the + # daily digest, Teams reminders, browser checks) runs without a prompt. + - id: ccd_confirmed + label: "CCD reconciled with pipeline" + text: "The Code Complete Date is a valid future date, reconciled with the pipeline (no unresolved override)" + detail: "Scout validates the CCD end-to-end: it must be future-dated (a past CCD — e.g. a current-month release whose 2nd-Wednesday default already passed — is invalid and BLOCKS until rescheduled), and it must agree with pipeline 3038's override (a differing override BLOCKS until reconciled). Because every phase is CCD-relative, a wrong CCD misaligns the whole release. If the CCD leaves Phase 0 (CCD-7→CCD) with less than its normal 7-day window, Scout WARNS about the compressed prep window but still lets you proceed." + verify: auto + source: scout # the SKILL runs check-ccd (reads the pipeline via az) + may reconcile interactively + verifier: ccd_confirmed + + - id: mcp_servers + label: "MCP servers registered" + text: "The MCP servers the release needs (ICM on-call, Kusto/ADX telemetry, Teams comms) are registered in Scout" + detail: "Scout checks the ICM + Kusto/ADX + Teams MCP servers are registered in your Scout config (needed for the on-call and telemetry checks, and to fill Teams comms gaps WorkIQ lacks)." + verify: auto + verifier: mcp_servers + + - id: silent_perms + label: "Silent-run permissions" + text: "Permissions allow fully unattended runs — shell, WorkIQ (email + Teams), the browser, and the ICM + Kusto/ADX MCP servers are auto-approved so scheduled work and the on-call/telemetry checks never stall on a prompt" + detail: "Scout checks shell, WorkIQ (email + Teams), the browser, and the ICM + Kusto/ADX MCP servers are all auto-approved so the daily automation and the readiness on-call/telemetry checks run silently when Scout isn't focused." + verify: auto + source: scout # only the skill can read Scout's own settings (m_get_settings) + verifier: silent_perms + # The servers that must be auto-approved for a fully-silent run (kept as data). + # Includes the MCP servers the readiness auto-checks call: `kusto` (adx_access) and + # `icm` (oncall_now) — otherwise those checks prompt on first run. + required_servers: [shell, workiq, playwright, kusto, icm] + # OPT-OUT (soft): enabling silent runs needs the user to turn on the Scout master + # toggle "Allow AI to request permission changes" (allowModelPermissionsChange) — + # which ONLY the user can flip in the UI. The skill offers to enable silent runs; + # if the user declines, it records `degraded` (proceed WITHOUT silent runs) and the + # gate still clears. Downside recorded: the daily digest / Teams reminders / browser + # checks will PROMPT when Scout isn't focused and can stall until the user opens Scout. + opt_out: true + + - id: teams_notify + label: "Scout Teams notifications" + text: "The daily release digest can reach you over the Scout Teams bot (in addition to email)" + detail: "Scout checks the Teams relay is connected and the Scout bot is reachable, so the hourly `tick` digest can be delivered to you in Teams as well as email. Falls back to email-only if Teams isn't reachable." + verify: auto + source: scout # only the skill can reach the relay (m_relay_status/connect) + bot (m_send_teams_message) + verifier: teams_notify + # OPT-OUT (soft): email is the guaranteed channel, so Teams is a bonus. If the relay + # won't connect or the Scout bot has no conversation yet (never messaged), the skill + # records `degraded` (email-only) and the gate still clears — it never blocks a release. + opt_out: true + + # ---- ATTEST: the engineer confirms (Scout cannot fully prove these) ---- + # `detail` is the table-cell text and MAY contain inline markdown links + # (they render fine inside Scout table cells). + - id: adx_access + label: "ADX release dashboard" + text: "You can query the ADX release-telemetry cluster (Scout verifies via Kusto)" + detail: "Scout verifies you can query the [ADX release dashboard](https://dataexplorer.azure.com/dashboards/ab7abd7e-c36f-4ff0-88a7-aaa3ec014bd7)'s cluster." + verify: auto + source: scout # the SKILL queries Kusto (the Python engine can't reach the MCP) + verifier: kusto_access + cluster_uri: "https://idsharedeus2.eastus2.kusto.windows.net" + database: "d496be22d62a46b0a3cf67ea2e736fd8" + + - id: oncall_now + label: "Not on-call now" + text: "You are NOT currently the Android on-call (Scout verifies via ICM)" + detail: "Scout verifies you are not the current Android [on-call](https://aka.ms/androidoncall) for Auth Client Android Shield." + verify: auto + source: scout # the SKILL checks ICM (the Python engine can't reach the MCP) + verifier: oncall + team_id: 78848 + team_name: "Auth Client Android Shield" + + - id: play_console_access + label: "Play Console access" + text: "You can open the Play Console app dashboard (sign in and confirm it loads)" + detail: "Open the [Play Console dashboard](https://play.google.com/console/u/0/developers/6720847872553662727/app/4972501392087484764/app-dashboard) and confirm it loads." + verify: attest + + - id: oncall_window + label: "Free during release window" + text: "You are NOT scheduled on-call during the release window" + detail: "Confirm you are NOT scheduled Android [on-call](https://aka.ms/androidoncall) during the release window." + verify: attest + window_start_anchor: "CCD-7" # window opens 7 days before Code Complete + window_end_anchor: "CCD+14" # …through 14 days after + + - id: saw_ame + label: "SAW + AME" + text: "SAW machine and AME account are accessible — log into SAW, confirm the desktop loads, and sign in with your AME credentials" + detail: "Log into [SAW](https://aka.ms/saw), confirm the desktop loads, and sign in with your AME credentials." + verify: attest + + - id: yubikey + label: "YubiKey in hand" + text: "YubiKey in hand (no YubiKey? pick one up at Studio A 1909, Bldg 34/35 cafe, or order a DSR USB-C thumbdrive)" + detail: "YubiKey in hand — no YubiKey? Studio A 1909, Bldg 34/35 cafe, or [order a DSR USB-C](https://cloudmfa-support.azurewebsites.net/SecurityKeyServices/SecurityKey)." + verify: attest diff --git a/release-agent/config/requirements.yaml b/release-agent/config/requirements.yaml new file mode 100644 index 00000000..2f167a39 --- /dev/null +++ b/release-agent/config/requirements.yaml @@ -0,0 +1,234 @@ +# Release Orchestrator — external dependency manifest (single source of truth). +# +# Lists everything the tool needs on a machine, so a DIFFERENT release engineer +# can verify their setup is complete. `bootstrap.ps1` reads this file and checks +# each entry (or, for MCP servers, prints them for the engineer to confirm in Scout). +# +# KEEP THIS UPDATED: whenever a feature or phase agent introduces a new dependency +# (a CLI, a Python package, or an MCP server), add it here in the same change. +# +# Entry fields: +# id unique short id +# name human name +# type cli | python | host | mcp +# required_by which part(s) of the tool need it (traceability) +# check shell command that succeeds (exit 0) when present [cli/python/host] +# install how to install it (shown when the check fails) [cli/python/host] +# note guidance (used for `mcp` entries — not shell-checkable) + +requirements: + - id: scout + name: "Microsoft Scout (host app)" + type: host + required_by: [everything — the skill runs inside Scout] + # Detected by the presence of the Scout profile dir (~/.scout). This is a soft + # check surfaced by `infra`; if absent, the engineer must install Scout FIRST + # (MCP servers can't be registered without it). + check: "cmd /c \"if exist \"%USERPROFILE%\\.scout\" (exit 0) else (exit 1)\"" + install: "Install Microsoft Scout first, then re-run bootstrap. (Get it from your team's Scout distribution / the internal Scout install page.)" + + - id: python + name: "Python 3.9+" + type: host + required_by: [engine] + check: "python --version" + install: "Install Python 3.9 or newer from https://www.python.org/downloads/" + + - id: pyyaml + name: "PyYAML" + type: python + required_by: [engine (reads config/*.yaml)] + check: "python -c \"import yaml\"" + install: "python -m pip install pyyaml" + + - id: tzdata + name: "tzdata (IANA timezone database)" + type: python + required_by: [engine (owner-timezone due-ness + fire_at_local gating)] + check: "python -c \"from zoneinfo import ZoneInfo; ZoneInfo('America/Los_Angeles')\"" + install: "python -m pip install tzdata" + + - id: tzlocal + name: "tzlocal (detect the owner's local timezone)" + type: python + required_by: [init (captures the owner's IANA timezone)] + check: "python -c \"import tzlocal\"" + install: "python -m pip install tzlocal" + + - id: azure-cli + name: "Azure CLI (az)" + type: cli + required_by: [readiness.build_access] + check: "az version" + install: "https://learn.microsoft.com/cli/azure/install-azure-cli" + + - id: azure-devops-ext + name: "Azure CLI 'azure-devops' extension" + type: cli + required_by: [readiness.build_access] + check: "az extension show --name azure-devops" + install: "az extension add --name azure-devops" + + - id: az-login + name: "Signed in to Azure DevOps (az login)" + type: cli + required_by: [readiness.build_access] + check: "az account show" + install: "az login (must have access to the release build definitions)" + + - id: agency-cli + name: "Agency CLI (provides the ICM MCP server)" + type: cli + required_by: [readiness.oncall_now, mcp.icm] + check: "cmd /c \"%APPDATA%\\agency\\CurrentVersion\\agency.exe --version\"" + install: "Install the Agency platform: https://aka.ms/agency (provides agency.exe under %APPDATA%\\agency)" + +# MCP servers the skill needs inside Scout. Unlike CLIs these live in Scout's own +# config (~/.scout/m-mcp-servers.json) and load at startup — so `bootstrap.ps1` +# REGISTERS any missing ones (create-if-absent, backing up the file first) and then +# tells the engineer to RESTART Scout for them to load. Each entry: +# id short id +# name human name +# scout_key the key to use under "servers" in m-mcp-servers.json +# provider a shell check that the launcher exists (so we don't register a broken server) +# command absolute launcher path (supports %APPDATA% etc.; expanded at register time) +# args launcher args +# required_by which part(s) of the tool need it +# note guidance +mcp_servers: + - id: icm + name: "ICM MCP server (on-call / incidents)" + scout_key: icm + provider: "cmd /c \"%APPDATA%\\agency\\CurrentVersion\\agency.exe --version\"" + command: "%APPDATA%\\agency\\CurrentVersion\\agency.exe" + args: ["mcp", "icm"] + required_by: [readiness.oncall_now, on-call lookups] + # Populated allowlist (Scout auto-discovers on a good first load, but an empty + # list risks a first-load DROP for command servers — so ship the tools). + tools: + - get_contact_by_id + - get_incident_details_by_id + - search_incidents_by_owning_team_id + - mitigate_incident + - get_impacted_azure_priority0_customers + - update_incident_severity + - transfer_incident + - get_incident_discussion_entries_and_insights + - get_incident_context + - reactivate_incident + - get_teams_by_public_id + - post_discussion_entry + - get_ai_summary + - get_teams_by_name + - acknowledge_incident + - get_team_by_id + - update_incident + - request_assistance + - is_specific_customer_impacted + - get_incident_customer_impact + - get_similar_incidents + - get_contact_by_alias + - get_services_by_names + - search_teams_or_services + - get_support_requests_crisit + - get_support_answer + - get_incident_location + - get_impacted_services_regions_clouds + - get_impacted_ace_customers + - get_on_call_schedule_by_team_id + - post_incident_insight + - get_mitigation_hints + - resolve_incident + - get_outage_high_priority_events + - get_my_icm_context + - get_impacted_s500_customers + - search_incidents + - get_impacted_subscription_count + note: "Provided by the Agency CLI. After registering, RESTART Scout so it loads; then Scout auto-discovers the ICM tools." + + - id: kusto + name: "Kusto / ADX MCP server (query telemetry)" + scout_key: kusto + provider: "cmd /c \"%APPDATA%\\agency\\CurrentVersion\\agency.exe --version\"" + command: "%APPDATA%\\agency\\CurrentVersion\\agency.exe" + args: ["mcp", "kusto"] + # Multi-cluster: infra appends '--known-services ' built from the + # `kusto_clusters` list below, so ALL our clusters are queryable through one + # MCP. Add a cluster there (data only) to make it available — no code change. + known_services_from: kusto_clusters + required_by: [monitoring/ADX dashboards, Play vitals, crash metrics, readiness.adx_access] + # Populated allowlist (non-empty avoids a first-load DROP; Scout still discovers + # any additional tools the MCP advertises). readiness.adx_access needs kusto_query. + tools: + - kusto_query + - kusto_command + - kusto_list_entities + - kusto_describe_database + - kusto_describe_database_entity + - kusto_sample_entity + - kusto_show_queryplan + - kusto_graph_query + - kusto_diagnostics + - kusto_ingest_inline_into_table + - kusto_deeplink_from_query + - kusto_get_shots + - kusto_known_services + note: "Provided by the Agency CLI (uvx microsoft-fabric-rti-mcp). After registering, RESTART Scout. Add clusters under kusto_clusters." + + - id: teams + name: "Microsoft Teams MCP server (Agency — group chats + rich messaging)" + scout_key: teams + provider: "cmd /c \"%APPDATA%\\agency\\CurrentVersion\\agency.exe --version\"" + command: "%APPDATA%\\agency\\CurrentVersion\\agency.exe" + args: ["mcp", "teams"] + required_by: [Teams notification gaps, group-chat creation, rich Teams messaging] + # CRITICAL: command-based servers read this allowlist STATICALLY — an empty list + # makes Scout silently DROP the server. These are the 36 Agency Teams tools + # (namespaced microsoft_teams-* once loaded). Keep this populated. + tools: + - ListTeams + - ListChats + - ListChannels + - GetTeam + - ListTeamMembers + - GetChat + - CreateChat + - DeleteChat + - UpdateChat + - GetChannel + - CreateChannel + - UpdateChannel + - ListChatMembers + - ListChannelMembers + - AddChatMember + - AddChannelMember + - UpdateChannelMember + - SendFileToChat + - SendFileToUser + - SendFileToChannel + - ListChannelFiles + - SendMessageToUser + - ListChatMessages + - ListChannelMessages + - SendMessageToChat + - SendMessageToSelf + - SendMessageToChannel + - GetChatMessage + - UpdateChatMessage + - DeleteChatMessage + - ReplyToChannelMessage + - ListChannelMessageReplies + - GetRichMessageFormats + - GetUserPresence + - SearchTeamsMessages + - SearchTeamMessagesQueryParameters + note: "Provided by the Agency CLI (agency mcp teams). Fills Teams gaps WorkIQ lacks (group-chat creation, add members, channel posts). After registering, RESTART Scout. NOT auto-approved — Teams writes prompt for confirmation; see docs/teams-mcp-setup.md." + +# Kusto/ADX clusters the tool queries (data — infra wires these into the kusto MCP +# via --known-services). Add an entry per cluster we use. +kusto_clusters: + - service_uri: "https://idsharedeus2.eastus2.kusto.windows.net" + default_database: "d496be22d62a46b0a3cf67ea2e736fd8" + description: "ID shared EastUS2 (idsharedeus2) — Android release telemetry" + + diff --git a/release-agent/config/scenarios/at_rc_gate.yaml b/release-agent/config/scenarios/at_rc_gate.yaml new file mode 100644 index 00000000..34585316 --- /dev/null +++ b/release-agent/config/scenarios/at_rc_gate.yaml @@ -0,0 +1,42 @@ +# Fast-forward THROUGH Phase 2 (Build & RC Verification): the four verification steps +# resolve from deterministic OFFLINE mocks (no az) and stash the pipeline ids, then the +# rc_report step auto-advances (go_test was removed — rc_report's 90% UI gate is the +# decision). Lands positioned at the entry of Phase 3 (bug bash). Use this to exercise +# the RC verification chain, the pipeline-id stash, and the status render without touching +# the network. +# +# release-agent sim run --scenario at_rc_gate +# +name: at_rc_gate +release_id: 2026-08 +ccd: 2026-08-26 +ccd_source: confirmed +as_of: CCD+2 +target: { phase: build_verify, at: done } +data: mock +mocks: + build_verify.checker_fired: + triggering: { run: { id: 1678599, queueTime: "2026-08-13T06:00:00Z" }, result: succeeded } + build_verify.orchestrator_health: + run: + id: 1678611 + tags: [ "AuthenticatorBranch=release-2026-08-13", "NextCommonVersion=24.6.0", + "NextMsalVersion=8.4.2", "NextBrokerVersion=16.5.0", + "RC-ECS=1678863", "RC-Local=1678864" ] + stages: + - { name: "Validate Branch and Versions availability", state: completed, result: succeeded } + - { name: "Create Release Branches", state: completed, result: succeeded } + - { name: "Trigger RC Testing", state: completed, result: succeeded } + - { name: "Remove RC Tags", state: pending, result: null } + build_verify.mrwp_ecs: + mrwp_id: "1678863" + stages: + - { name: "Build", state: completed, result: succeeded } + - { name: "UI Automation", state: completed, result: failed } + tests: { total: 5871, passed: 5767, failed: 104 } + build_verify.mrwp_local: + mrwp_id: "1678864" + stages: + - { name: "Build", state: completed, result: succeeded } + - { name: "UI Automation", state: completed, result: failed } + tests: { total: 5856, passed: 5756, failed: 100 } diff --git a/release-agent/config/scenarios/build_verify_live.yaml b/release-agent/config/scenarios/build_verify_live.yaml new file mode 100644 index 00000000..6d8b83f8 --- /dev/null +++ b/release-agent/config/scenarios/build_verify_live.yaml @@ -0,0 +1,15 @@ +# Test Phase 2 END-TO-END against the REAL pipelines: fast-forwards Phases 0-1, +# then runs the four verification steps LIVE (real `az` reads of the 2026-08 +# checker → orchestrator → ECS/Local MRWP runs), auto-advances rc_report, and lands +# positioned at the bug-bash entry (go_test was removed — rc_report's 90% UI gate is the +# decision). Seeds the real release, so afterwards use the normal skill: status / next. +# +# release-agent sim run --scenario build_verify_live +# +name: build_verify_live +release_id: 2026-08 +ccd: 2026-08-26 +ccd_source: confirmed +as_of: CCD+2 +target: { phase: build_verify, at: done } +data: live # the 4 build_verify steps hit real az; earlier phases are mocked diff --git a/release-agent/config/scenarios/mid_build_verify_open.yaml b/release-agent/config/scenarios/mid_build_verify_open.yaml new file mode 100644 index 00000000..9ae4f158 --- /dev/null +++ b/release-agent/config/scenarios/mid_build_verify_open.yaml @@ -0,0 +1,13 @@ +# Drop in at the START of Phase 2 (build_verify) with Phases 0-1 done, entry gate +# signed, clock one day past CCD. The four verification steps are NOT run — you +# validate them yourself LIVE against the real 2026-08 pipeline runs from here. +# +# release-agent sim run --scenario mid_build_verify_open +# +name: mid_build_verify_open +release_id: 2026-08 +ccd: 2026-08-26 +ccd_source: confirmed +as_of: CCD+1 +target: { phase: build_verify, at: open } +data: live # target phase reads real az; earlier phases are fast-forwarded diff --git a/release-agent/config/schedule.yaml b/release-agent/config/schedule.yaml new file mode 100644 index 00000000..c397b9e0 --- /dev/null +++ b/release-agent/config/schedule.yaml @@ -0,0 +1,26 @@ +# Where the Code Complete Date (CCD) comes from — the system of record. +# +# The orchestrator SEEDS CCD from this pipeline at `init`, and when you change +# CCD (`set-ccd`) or skip the release (`skip-release`) it WRITES BACK to these +# same variables — gated (explicit --confirm) and audited (reason + event log). +# +# CCD rule (mirrors the pipeline's own YAML): +# * override_variable, if set AND its month == the release month -> use it +# * otherwise -> 2nd Wednesday of the month +# +# Per-phase anchors (e.g. Phase 0 opens CCD-7) live next to each phase in +# phases.yaml as `anchor: "CCD-7"`. This file only says WHERE CCD comes from. + +# The release owner's timezone. Phase due-ness and every step's `fire_at_local` are +# evaluated on THIS wall clock — not the host's — so a UTC automation host doesn't +# roll the date (and open phases / fire timed comms) hours early. IANA name; needs the +# `tzdata` package on Windows. Defaults to America/Los_Angeles if omitted/unloadable. +timezone: America/Los_Angeles + +ccd_source: + pipeline_id: 3038 + org: "https://identitydivision.visualstudio.com" + project: "Engineering" + name: "Code Complete Calendar Checker" + override_variable: "overrideCodeCompleteDate" # YYYY-MM-DD, month-scoped + skip_variable: "skipRelease" # any non-empty value suppresses the release diff --git a/release-agent/mocks.local.example.yaml b/release-agent/mocks.local.example.yaml new file mode 100644 index 00000000..0c195e0e --- /dev/null +++ b/release-agent/mocks.local.example.yaml @@ -0,0 +1,135 @@ +# mocks.local.example.yaml — copy to `mocks.local.yaml` to use it. +# +# `mocks.local.yaml` is GITIGNORED: your personal testing file, never pushed. +# When present, the engine REPLACES the listed steps on every `next` / `step-action`. +# Everything NOT listed runs for real, so you keep driving the release through Scout +# normally — the mocked steps just resolve themselves. +# +# TWO layers of control (see `python -m orchestrator.cli mock-spec` for the live list): +# +# 1) Engine-level — works for EVERY non-gate step in EVERY phase: +# outcome: done # mark complete, skip its real work +# outcome: blocked # hold for the owner (rehearse the block path) +# note: "..." / reason: "..." +# +# 2) Per-step payload overrides — only where a step DECLARES them (its MOCKABLE +# spec). These keep the action REAL but redirect it. Currently: +# preflight.notice send_to: ["me@microsoft.com"] # real email, only to me +# preflight.flight_reminder send_to: me # real Teams post, my own chat +# Unknown keys are reported back as `unknown_overrides` so typos show. +# +# Gate steps (🚦 below) are NOT mockable — a gate needs a real human decision. + +# ============================================================================ +# ACTIVE example (delete/edit as you like): real run, skip the CG query. +# ============================================================================ +preflight.cg: + outcome: done + note: "mocked: CG clean for local testing" + + +# ============================================================================ +# FULL SAFE-TEST PROFILE — uncomment ALL of this to run the whole flow with ZERO +# real side-effects (no emails, no Teams posts, no ADO writes, no live az/MCP). +# Clears the entry gate offline and mocks every Phase-0 side-effect. +# Then just drive the release through Scout normally ("start a test release for +# ", "continue", …) — nothing special to type. Still sign the attest items +# (oncall_window, saw_ame, yubikey, play_console_access, confirm_reminders, vitals) +# when Scout asks — those are human confirmations with no side-effects. +# ============================================================================ +# # -- entry gate: force the real ADO/config/MCP checks to pass -- +# readiness.build_access: { outcome: pass } +# readiness.mcp_servers: { outcome: pass } +# readiness.silent_perms: { outcome: pass } +# readiness.adx_access: { outcome: pass } +# readiness.oncall_now: { outcome: pass } +# # -- Phase 0: no real sends / writes / network -- +# preflight.notice: { outcome: done } # or { send_to: ["you@microsoft.com"] } to really receive it +# preflight.flight_reminder: { outcome: done } # or { send_to: me } to really get it in your chat +# preflight.lockdown: { outcome: done } +# preflight.breaking: { changelog: "vNext\n----\n- [MINOR] x (#1)\nVersion 1.0.0\n" } +# preflight.cg: { alerts: [] } +# preflight.cron: { run: { queueTime: "2026-08-14T12:00:00Z", result: succeeded } } +# preflight.wiki: { outcome: done } + + +# ============================================================================ +# FULL CATALOG — uncomment any line and add a behavior. `{ outcome: done }` shown; +# swap for `{ outcome: blocked, reason: "..." }` to rehearse a hold. +# ============================================================================ + +# ---- Phase 0 · preflight ---- +# preflight.notice: { send_to: ["me@microsoft.com"] } # scout — or { outcome: done } +# preflight.flight_reminder: { send_to: me } # scout — or { outcome: done } +# preflight.confirm_reminders: { outcome: done } # attest +# preflight.lockdown: { outcome: done } # scout +# preflight.breaking: { outcome: done } # agent +# preflight.cg: { outcome: done } # agent (active above) +# preflight.vitals: { outcome: done } # attest +# preflight.cron: { outcome: done } # agent +# preflight.wiki: { outcome: done } # agent + +# ---- Phase 1 · ccd ---- +# ccd.final_reminder: { outcome: done } · send_to: # scout (email) +# ccd.pr_reminder: { outcome: done } · send_to: me # scout (Teams) +# ccd.localization: { outcome: done } # scout (pipeline trigger) + +# ---- Phase 2 · build_verify ---- +# build_verify.stages_ok: { outcome: done } # agent +# build_verify.retain: { outcome: done } # agent +# build_verify.health: { outcome: done } # agent +# build_verify.ui_auto: { outcome: done } # agent +# build_verify.payload: { outcome: done } # agent +# build_verify.mrwp_rc: { outcome: done } # agent +# build_verify.rc_report: { outcome: done } # scout (emails RC report + applies the 90% UI gate) + +# ---- Phase 3 · bug_bash ---- +# bug_bash.clone_plans_broker: { outcome: done } # agent (copy Broker master plan) +# bug_bash.clone_plans_auth: { outcome: done } # agent (create Authenticator query-suite) +# bug_bash.coordinate: { outcome: done } # agent +# bug_bash.ui_failures: { outcome: done } # reminder (human) +# bug_bash.signoffs: { outcome: done } # agent +# bug_bash.bash_done: 🚦 gate — NOT mockable + +# ---- Phase 4 · finalize ---- +# finalize.gate_watch: 🚦 gate — NOT mockable +# finalize.integ_prs: { outcome: done } # agent +# finalize.verify_pub: { outcome: done } # agent +# finalize.final_comms: { outcome: done } # agent +# finalize.nonrc_pin: { outcome: done } # agent +# finalize.tag: { outcome: done } # agent +# finalize.backmerge: { outcome: done } # agent + +# ---- Phase 5 · rollout_start ---- +# rollout_start.notice: { outcome: done } # agent +# rollout_start.artifact: { outcome: done } # agent +# rollout_start.signoff_start: 🚦 gate — NOT mockable + +# ---- Phase 6 · monitor ---- +# monitor.health_report: { outcome: done } # agent +# monitor.adoption: { outcome: done } # agent +# monitor.guards: { outcome: done } # agent +# monitor.advance: 🚦 gate — NOT mockable +# monitor.progress_email: { outcome: done } # agent +# monitor.dashboard: { outcome: done } # agent +# monitor.crash_annot: { outcome: done } # reminder (human) +# monitor.hotfix_eval: { outcome: done } # reminder (human) + +# ---- Phase 7 · partner ---- +# partner.china: { outcome: done } # reminder (human) +# partner.samsung: { outcome: done } # reminder (human) +# partner.ngms: { outcome: done } # agent +# partner.teams_dev: { outcome: done } # agent +# partner.approvals: { outcome: done } # agent +# partner.review_times: { outcome: done } # agent + +# ---- Phase (conditional) · hotfix ---- +# hotfix.cherry: { outcome: done } # agent +# hotfix.rebuild: { outcome: done } # agent +# hotfix.smoke: 🚦 gate — NOT mockable +# hotfix.expedite: { outcome: done } # agent +# hotfix.alpha_orgs: { outcome: done } # agent + +# ---- Phase 9 · close ---- +# close.closeout: { outcome: done } # agent +# close.done: 🚦 gate — NOT mockable diff --git a/release-agent/orchestrator/__init__.py b/release-agent/orchestrator/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/release-agent/orchestrator/automations.py b/release-agent/orchestrator/automations.py new file mode 100644 index 00000000..72622438 --- /dev/null +++ b/release-agent/orchestrator/automations.py @@ -0,0 +1,226 @@ +"""Per-release automation planning + validation (traceability layer). + +`config/automations.yaml` declares WHICH Scout automations a release provisions and +WHICH STEPS each drives. This module turns that data into: + + * plan(release, ccd) — concrete specs the skill uses to create + register each + automation (name, schedule, prompt, steps, fire time). Timing is DERIVED from + each step module's `fire_at_local`, so the step module is the single source. + * validate() — the self-enforcing guardrail: every step that declares a + `fire_at_local` is owned by EXACTLY ONE automation; every automation's steps + exist and share ONE fire time. A test runs this so the mapping can't drift. + +The engine never calls Scout's automation API — the skill does. This is pure data + +computation (no IO beyond reading the two yaml files). +""" +from __future__ import annotations + +import os +from datetime import datetime + +import yaml + +import steps as steps_pkg +from orchestrator import schedule + + +def automations_path(config_path: str) -> str: + """config/automations.yaml sits next to phases.yaml (config_path).""" + return os.path.join(os.path.dirname(config_path), "automations.yaml") + + +def load_defs(config_path: str) -> list: + p = automations_path(config_path) + if not os.path.exists(p): + return [] + with open(p, "r", encoding="utf-8") as fh: + doc = yaml.safe_load(fh) or {} + return doc.get("automations", []) or [] + + +def _step_fire_at(step_key: str): + """The `fire_at_local` a step module declares (or None). step_key = '.'.""" + phase, _, sid = step_key.partition(".") + mod = steps_pkg.get_step(phase, sid) + if mod is None: + return None + return (getattr(mod, "CONFIG", {}) or {}).get("fire_at_local") + + +def fire_at(phase_id: str, step_id: str): + """Public: the `fire_at_local` (HH:MM) a step declares, or None. Used by the engine + to gate a timed step until its wall-clock time.""" + return _step_fire_at(f"{phase_id}.{step_id}") + + +def _all_scheduled_steps(config_path: str) -> dict: + """{ '.': fire_at_local } for every discovered module that declares + a fire_at_local — i.e. every step that a timed automation must own.""" + out = {} + for key, mod in steps_pkg.discover().items(): + fire = (getattr(mod, "CONFIG", {}) or {}).get("fire_at_local") + if fire: + out[key] = fire + return out + + +def validate(config_path: str) -> list: + """Return a list of human-readable problems (empty = healthy). Enforces: + 1. every automation step exists (has a discovered module), + 2. every TIME-OF-DAY automation's steps share ONE fire_at_local (its fire time) + and each declares fire_at_local, + 3. no step is owned by two TIME-OF-DAY automations, + 4. every scheduled step (declares fire_at_local) is owned by SOME time-of-day + automation. + INTERVAL automations (those with `every:`, e.g. a poller) are exempt from the + fire_at_local accounting — they may share a step with a time-of-day automation + and their steps need not declare fire_at_local — but their steps must still exist. + """ + problems = [] + defs = load_defs(config_path) + owned = {} # step_key -> slug (time-of-day only) + for d in defs: + slug = d.get("slug", "?") + s_steps = d.get("steps", []) or [] + if not s_steps: + problems.append(f"automation '{slug}' has no steps") + continue + interval = bool(d.get("every")) + fires = set() + for sk in s_steps: + mod = steps_pkg.get_step(*sk.split(".", 1)) if "." in sk else None + if mod is None: + problems.append(f"automation '{slug}' references unknown step '{sk}'") + continue + if interval: + continue # pollers are exempt from fire-time accounting + if sk in owned: + problems.append(f"step '{sk}' is owned by two time-of-day automations " + f"('{owned[sk]}' and '{slug}')") + owned[sk] = slug + fire = (getattr(mod, "CONFIG", {}) or {}).get("fire_at_local") + if not fire: + problems.append(f"step '{sk}' (in '{slug}') declares no fire_at_local") + else: + fires.add(fire) + if not interval and len(fires) > 1: + problems.append(f"automation '{slug}' groups steps with different fire " + f"times {sorted(fires)} — split them") + + for sk in _all_scheduled_steps(config_path): + if sk not in owned: + problems.append(f"scheduled step '{sk}' (has fire_at_local) is not owned " + f"by any time-of-day automation in automations.yaml") + return problems + + +def _ccd_cron(ccd_date, hhmm: str): + """A cron schedule pinned to the EXACT Code Complete Date + fire time — NOT a + recurring weekday. `every ` fires on the NEXT matching weekday, which for + a CCD more than a week out (these are provisioned at release start) is the wrong + date — it fired the CCD-day comms a week early. Cron `M H D Mo *` targets the CCD's + day-of-month + month exactly, so a one-shot fires ON the CCD. Returns the NL Scout + accepts (e.g. 'cron: 0 9 26 8 *') or None if inputs are missing/invalid. + + TIMEZONE: emit the LOCAL wall-clock time directly — do NOT convert to UTC. + Scout's scheduler interprets cron in host-local time (empirically verified + 2026-08-20: a cron '37 9' fired at 09:37 PDT / 16:37 UTC, not 09:37 UTC). So a + `hhmm` of '09:00' correctly fires at 09:00 local on the CCD. Adding a UTC + conversion here would shift every CCD-day comm by the host's UTC offset.""" + if not ccd_date or not hhmm: + return None + try: + t = datetime.strptime(hhmm, "%H:%M") + except ValueError: + return None + return f"cron: {t.minute} {t.hour} {ccd_date.day} {ccd_date.month} *" + + +def _prompt_for(spec: dict, release: str) -> str: + """A concrete instruction the automation runs. Scout resolves each step via + step-action, executes the send/trigger, records it, and journals it. + + A step MAY OWN a bespoke prompt by declaring `automation_prompt(release, spec)` on its + module (the single source of truth, like `fire_at_local`) — used for genuinely bespoke + flows such as the localization trigger + poller. This keeps the planner generic: it + never special-cases a step id. Steps without one get the default send + record-step + prompt below.""" + steps = spec.get("steps") or [] + step_list = ", ".join(steps) + + # Single-step automation whose step owns a bespoke prompt → delegate to the module. + if len(steps) == 1: + phase, _, sid = steps[0].partition(".") + mod = steps_pkg.get_step(phase, sid) + fn = getattr(mod, "automation_prompt", None) + if callable(fn): + prompt = fn(release, spec) + if prompt: + return prompt + + # Default: send/trigger + record-step done (reminders). + return ( + f"Release {release} — {spec['name'].format(release=release)}.\n" + f"It is Code Complete Day. For EACH of these steps in order: {step_list} —\n" + f"1. run `step-action --release {release} --phase {spec['phase']} --step `;\n" + f"2. execute the returned needs_skill action (send the email / post the Teams " + f"message) with the given payload;\n" + f"3. `record-step --release {release} --phase {spec['phase']} --step " + f"--status pass` (or blocked with a reason);\n" + f"4. silently journal it: `journal --release {release} --source scout " + f"--kind automation --text \" ran \"`.\n" + f"Respect the mocks.local.yaml redirects if present. Report a one-line summary." + ) + + +def plan(config_path: str, release: str, ccd: str) -> dict: + """Concrete provisioning specs for a release. Returns + {release, ccd, problems, automations:[...]}. Each automation spec has: + slug, name, phase, steps, purpose, fire_at, schedule (NL for m_create_automation), + ccd_date, weekday, prompt, registration (the `automation register` args).""" + problems = validate(config_path) + defs = load_defs(config_path) + ccd_date = schedule.parse_date(ccd) if ccd else None + weekday = ccd_date.strftime("%A") if ccd_date else None + + out = [] + for d in defs: + slug = d.get("slug", "?") + s_steps = d.get("steps", []) or [] + interval = d.get("every") + name = d.get("name", slug).format(release=release) + if interval: + fire_at, sched, one_shot = None, f"every {interval}", False + else: + fire_at = _step_fire_at(s_steps[0]) if s_steps else None + # Pin to the EXACT CCD date via cron — never 'every ' (which fires + # the next matching weekday, a week early for a CCD provisioned in advance). + sched = _ccd_cron(ccd_date, fire_at) + one_shot = True + spec = { + "slug": slug, + "name": name, + "phase": d.get("phase"), + "steps": s_steps, + "kind": "step-driving", # everything in automations.yaml drives steps + "purpose": d.get("purpose", ""), + "fire_at": fire_at, + "ccd_date": ccd_date.isoformat() if ccd_date else None, + "weekday": weekday, + "schedule": sched, # one-shot on the CCD date, or an interval poller + "one_shot": one_shot, + "interval": interval or None, + # ON-DEMAND automations (e.g. the RC poller) are NOT provisioned at release + # start — the skill creates them only when their trigger condition arises + # (an in-flight re-triggered RC) and tears them down when it clears. + "on_demand": bool(d.get("on_demand")), + } + spec["prompt"] = _prompt_for(spec, release) + # Exactly what to record after creating it, so linkage + schedule are captured + # (schedule lets `automation sync` detect CCD drift and re-pin the cron). + spec["registration"] = { + "name": name, "release": release, "purpose": d.get("purpose", ""), + "steps": s_steps, "kind": "step-driving", "schedule": sched, "slug": slug, + } + out.append(spec) + return {"release": release, "ccd": ccd, "problems": problems, "automations": out} diff --git a/release-agent/orchestrator/cli.py b/release-agent/orchestrator/cli.py new file mode 100644 index 00000000..2e2e8211 --- /dev/null +++ b/release-agent/orchestrator/cli.py @@ -0,0 +1,58 @@ +"""Release Orchestrator — CLI entry point (thin assembler). + +The interface the /release-agent skill calls. This file only wires the parser and +dispatches; the command handlers live in `orchestrator/commands/` (one module per +domain) and shared plumbing in `orchestrator/cli_common.py`. + + python -m orchestrator.cli [options] + +State lives in //release-state.json (gitignored). +Config is release-agent/config/*.yaml. +""" +from __future__ import annotations +import argparse +import os +import sys + +# Force UTF-8 stdout so status glyphs don't crash under Windows cp1252 when piped. +try: + sys.stdout.reconfigure(encoding="utf-8") + sys.stderr.reconfigure(encoding="utf-8") +except (AttributeError, ValueError): # non-reconfigurable stream / unsupported encoding + pass + +HERE = os.path.dirname(os.path.abspath(__file__)) +ROOT = os.path.dirname(HERE) # release-agent/ +sys.path.insert(0, ROOT) + +from orchestrator import cli_common as C +from orchestrator.commands import REGISTRARS + + +def build_parser(): + p = argparse.ArgumentParser(prog="release-agent", + description="Release Orchestrator backbone (X4+X5).") + p.add_argument("--config", default=C.DEFAULT_CONFIG) + p.add_argument("--runs-root", default=C.DEFAULT_RUNS_ROOT) + # NOTE: --as-of is defined per-command (status/next/approve/deny/done/resume/notify), + # where it must appear AFTER the subcommand. It is intentionally NOT a global flag: + # argparse lets a subparser's own --as-of silently clobber a global one, which is a + # footgun. Commands that don't take a simulated clock simply omit it. + sub = p.add_subparsers(dest="cmd", required=True) + for register in REGISTRARS: + register(sub) + return p + + +def main(argv=None): + args = build_parser().parse_args(argv) + # Serialize state read-modify-write per release so parallel CLI invocations + # (e.g. the skill firing record-step calls at once) can't clobber each other. + runs_root = getattr(args, "runs_root", None) + release = C.effective_release(runs_root, getattr(args, "release", None)) + with C.state_lock(runs_root, release): + return args.func(args) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/release-agent/orchestrator/cli_common.py b/release-agent/orchestrator/cli_common.py new file mode 100644 index 00000000..a97cf1f1 --- /dev/null +++ b/release-agent/orchestrator/cli_common.py @@ -0,0 +1,259 @@ +"""Shared CLI plumbing for the Release Orchestrator command modules. + +The CLI is split into a thin assembler (`cli.py`) plus one module per command +domain under `commands/`. This module holds the pieces those command modules +share: path resolution, state/orchestrator loading, the event log, user-facing +emit (print + auto-log), and the small render helpers used when advancing. + +Everything here takes explicit parameters (runs_root / release / config) rather +than the argparse namespace, so the helpers are decoupled from the parser and +easy to reuse and test. +""" +from __future__ import annotations + +import os +import time +from contextlib import contextmanager + +from orchestrator.state import ReleaseState +from orchestrator.engine import Orchestrator +from orchestrator import discovery, render, schedule +from orchestrator.eventlog import EventLog +from tools import checks +import yaml as _yaml + +HERE = os.path.dirname(os.path.abspath(__file__)) +ROOT = os.path.dirname(HERE) # release-agent/ +DEFAULT_CONFIG = os.path.join(ROOT, "config", "phases.yaml") +SCHEDULE_CONFIG = os.path.join(ROOT, "config", "schedule.yaml") +REQUIREMENTS_CONFIG = os.path.join(ROOT, "config", "requirements.yaml") +# runs live OUTSIDE release-agent/, in android-complete/.release-runs (gitignored) +DEFAULT_RUNS_ROOT = os.path.join(os.path.dirname(ROOT), ".release-runs") + +# ---- inter-process state lock ---- +_LOCK_TIMEOUT = 30.0 # max seconds to wait for another CLI process to release +_LOCK_STALE = 120.0 # a lock older than this is treated as abandoned (crashed proc) + + +@contextmanager +def state_lock(runs_root: str, release): + """Serialize a release's state read-modify-write ACROSS CLI processes. + + Every mutating command loads state, mutates, then saves. Two running at once + (e.g. the skill firing `record-step` calls in parallel, or an hourly `tick` + overlapping an interactive command) would clobber each other — a last-writer- + wins lost update. This exclusive per-release lock makes each CLI invocation + atomic: a second process blocks until the first has saved and released. + Read-only commands hold it only for their brief duration. + + No release (e.g. `list`, `infra`) → no lock: nothing release-scoped to guard. + A lock older than _LOCK_STALE is stolen (its owner crashed). + """ + if not release: + yield + return + lock_dir = os.path.join(runs_root, release) + os.makedirs(lock_dir, exist_ok=True) + lock_path = os.path.join(lock_dir, ".state.lock") + deadline = time.monotonic() + _LOCK_TIMEOUT + fd = None + while True: + try: + fd = os.open(lock_path, os.O_CREAT | os.O_EXCL | os.O_WRONLY) + os.write(fd, str(os.getpid()).encode()) + break + except FileExistsError: + try: + if time.time() - os.path.getmtime(lock_path) > _LOCK_STALE: + os.remove(lock_path) # abandoned by a crashed process + continue + except OSError: + pass + if time.monotonic() > deadline: + raise TimeoutError( + f"could not acquire state lock for release {release} within " + f"{_LOCK_TIMEOUT:.0f}s — another CLI process is holding it") + time.sleep(0.05) + try: + yield + finally: + try: + os.close(fd) + except OSError: + pass + try: + os.remove(lock_path) + except OSError: + pass + + +def effective_release(runs_root, release): + """The release id to lock on. The explicit `--release` when given; otherwise, + for discovery-mode mutating commands (e.g. the hourly `tick`), the single + active release so it's still serialized against interactive commands. Returns + None when ambiguous / none exist (nothing to serialize on).""" + if release: + return release + if not runs_root: + return None + try: + res = discovery.resolve(runs_root, None) + if res.get("resolution") == "one" and res.get("release"): + return res["release"].get("release_id") + except Exception: + pass + return None + + +# ---- paths / state ---- +def state_path(runs_root: str, release: str) -> str: + return os.path.join(runs_root, release, "release-state.json") + + +def load_state(runs_root: str, release: str) -> ReleaseState: + return ReleaseState.load(state_path(runs_root, release)) + + +def save_state(st: ReleaseState, runs_root: str, release: str) -> None: + st.save(state_path(runs_root, release)) + + +def parse_as_of(args): + """The simulated clock from --as-of (None ⇒ engine uses today).""" + s = getattr(args, "as_of", None) + return schedule.parse_date(s) if s else None + + +def load_orch(runs_root: str, release: str, config: str, as_of=None): + """Load state + build an Orchestrator wired to the --as-of clock.""" + st = load_state(runs_root, release) + return st, Orchestrator(config, st, as_of=as_of) + + +# ---- config ---- +def ccd_source() -> dict: + """Where CCD comes from (pipeline coords) — from config/schedule.yaml.""" + try: + with open(SCHEDULE_CONFIG, "r", encoding="utf-8") as fh: + return (_yaml.safe_load(fh) or {}).get("ccd_source", {}) or {} + except OSError: + return {} + + +# ---- event log / emit ---- +def elog(runs_root: str, release: str) -> EventLog: + return EventLog(runs_root, release) + + +def emit(runs_root: str, release: str, text: str, kind: str = "message", options=None, + log_text: str = None): + """Print a user-facing block AND auto-log it as scout output, so the log + always captures 'what was shown' without relying on the skill/LLM to journal. + + `log_text` lets the caller journal a COMPACT form while still printing the full + block to the console — used for the `advance`/`status` renders whose full status + table is pure presentation (the structured step events already capture the state), + so we don't bloat events.jsonl with repeated multi-KB table snapshots.""" + print(text) + try: + elog(runs_root, release).scout_said( + log_text if log_text is not None else text, kind=kind, options=options) + except Exception: + pass + + +# ---- advancing the loop (shared by next / approve / done) ---- +TAGS = {"ran": "[ok]", "gate": "[gate]", "reminder": "[action]", "scheduled": "[scheduled]", + "complete": "[done]", "idle": "[--]", "readiness": "[entry-gate]", + "blocked": "[BLOCKED]", "halted": "[HALTED]"} + + +def log_actions(el: EventLog, actions, state=None): + """Record engine actions as events (step_ran / step_blocked / gate_hold / …). + + When `state` is provided, phase/step events are enriched with the step's recorded + OUTCOME — `status` (done/blocked/skipped) and `note` (the actual result or block + reason) — so the event log is self-contained and queryable without scraping the + rendered `scout_output` blobs. A step that ran but recorded a block is logged as + `step_blocked` (not `step_ran`) so failures are directly filterable.""" + events = { + "ran": "step_ran", "gate": "gate_hold", "reminder": "reminder_hold", + "scheduled": "scheduled_hold", "readiness": "readiness_hold", + "blocked": "blocked_hold", "halted": "halted_hold", "complete": "release_complete", + } + for a in actions: + name = events.get(a.kind) + if not name: + continue + if a.kind in ("ran", "gate", "reminder", "scheduled"): + fields = {"phase": a.phase, "step": a.step, "name": a.name} + # Enrich with the step's recorded outcome (the finding / block reason), + # not just the fact that it ran. + if state is not None and a.phase and a.step: + stp = state.get_step(a.phase, a.step) + if stp is not None: + if getattr(stp, "status", None): + fields["status"] = stp.status + if getattr(stp, "note", None): + fields["note"] = stp.note + # A ran step that recorded a block is a failure → log it distinctly. + if a.kind == "ran" and fields.get("status") == "blocked": + name = "step_blocked" + el.log(name, **fields) + else: + el.log(name) + + +def advance_block(actions, orch, lead=None) -> str: + """The canonical 'what happened + new status' block for advance commands.""" + out = list(lead or []) + for a in actions: + out.append(f" {TAGS.get(a.kind, '-')} {a.message}") + out.append("\n" + render.status_view(orch.status_report())) + return "\n".join(out) + + +def advance_log_summary(actions, lead=None) -> str: + """Compact journal form of an advance — the per-action outcome lines only, WITHOUT + the appended full status table (presentation; the structured step_ran/step_blocked + events already record the state). Keeps events.jsonl lean and self-describing.""" + out = [l.strip() for l in (lead or [])] + for a in actions: + out.append(f"{TAGS.get(a.kind, '-')} {a.message}") + return "\n".join(l for l in out if l).strip() + + +# ---- CCD / pipeline helpers ---- +def refresh_conflict(st: ReleaseState) -> bool: + """Best-effort: re-read the pipeline override and refresh st.ccd_conflict + (a pipeline date that differs from our stored CCD). Returns True if the state + changed (caller should save). Silent on any read failure — never blocks.""" + if not st.ccd: + return False + src = ccd_source() + if not src.get("pipeline_id"): + return False + ok, val, _ = checks.read_pipeline_variable( + src["org"], src["project"], src["pipeline_id"], src["override_variable"]) + if not ok: + return False + conflict = schedule.pipeline_conflict(st.release_id, val, st.ccd) + new = conflict.isoformat() if conflict else None + if new != st.ccd_conflict: + st.ccd_conflict = new + return True + return False + + +def write_ccd_var(src: dict, value: str): + """Write the CCD override variable on the pipeline. Returns CheckResult.""" + return checks.set_pipeline_variable( + src["org"], src["project"], src["pipeline_id"], src["override_variable"], value) + + +def resolve_release_id(runs_root: str, release): + """Return an explicit release id or discover the active one (or None).""" + if release: + return release + rel = discovery.resolve(runs_root, None).get("release") + return rel["release_id"] if rel else None diff --git a/release-agent/orchestrator/commands/__init__.py b/release-agent/orchestrator/commands/__init__.py new file mode 100644 index 00000000..1ce78330 --- /dev/null +++ b/release-agent/orchestrator/commands/__init__.py @@ -0,0 +1,27 @@ +"""Command modules for the Release Orchestrator CLI. + +Each module in this package owns one domain of commands. A module exposes a +`register(subparsers)` function that adds its subparser(s) and wires each to its +handler via `set_defaults(func=...)`. `cli.py` imports REGISTRARS and calls each +one, so adding a command is a localized change (new/edited module only). +""" +from . import (release, readiness, pipeline, notify, infra_cmd, automation, + logs, lockdown, notice, step_action, localization, rc_report, rc_poll, sim) + +# Order controls how subcommands appear in --help. +REGISTRARS = [ + release.register, + readiness.register, + pipeline.register, + notify.register, + lockdown.register, + step_action.register, + notice.register, + localization.register, + rc_report.register, + rc_poll.register, + sim.register, + logs.register, + automation.register, + infra_cmd.register, +] diff --git a/release-agent/orchestrator/commands/automation.py b/release-agent/orchestrator/commands/automation.py new file mode 100644 index 00000000..58697e5e --- /dev/null +++ b/release-agent/orchestrator/commands/automation.py @@ -0,0 +1,172 @@ +"""Automation registry command: register / list / deregister / plan provisioned +Scout automations so they can be created with step linkage and torn down cleanly at +release close.""" +from __future__ import annotations +import json as _json + +from orchestrator.registry import AutomationRegistry, kind_of +from orchestrator import automations as auto_plan +from orchestrator import cli_common as C + + +def cmd_automation(args): + """Track Scout automations the orchestrator provisions, so they can be torn + down at release close. This only records ids + step linkage — the skill does the + actual Scout create/delete via m_create_automation / m_delete_automation.""" + reg = AutomationRegistry(args.runs_root, getattr(args, "release", None)) + if args.action == "plan": + return _cmd_plan(args) + if args.action == "sync": + return _cmd_sync(args) + if args.action == "register": + if not (args.id and args.name): + print("register needs --id and --name.") + return 1 + try: + e = reg.register(args.id, args.name, release=args.release, + shared=args.shared, purpose=args.purpose or "", + steps=getattr(args, "step", None) or [], + kind=getattr(args, "kind", None) or None, + schedule=getattr(args, "schedule", None) or None, + slug=getattr(args, "slug", None) or None) + except ValueError as ex: + print(f"register error: {ex}") + return 1 + where = "shared" if e["scope"] == "shared" else f"release {e['release']}" + drives = f" — drives {', '.join(e['steps'])}" if e.get("steps") else " — owns no steps" + print(f"Registered automation {e['id']} [{e['kind']}] ({where}): {e['name']}{drives}") + return 0 + if args.action == "deregister": + if not args.id: + print("deregister needs --id.") + return 1 + print("Deregistered." if reg.deregister(args.id) else "No such automation id in registry.") + return 0 + # list + items = reg.list(release=args.release, scope=(args.scope or None), + step=(getattr(args, "step_filter", None) or None), + kind=(getattr(args, "kind", None) or None)) + if args.json: + print(_json.dumps(items, indent=2)) + return 0 + if not items: + print("No automations registered." if args.release is None + else f"No automations registered for release {args.release}.") + return 0 + for e in items: + where = "shared" if e.get("scope") == "shared" else (e.get("release") or "?") + k = kind_of(e) + drives = (f" drives: {', '.join(e.get('steps') or [])}" if e.get("steps") + else " (release-level — no steps)") + print(f" {e['id']} [{k}] [{where}] {e['name']} — {e.get('purpose','')}{drives}") + return 0 + + +def _cmd_plan(args): + """Emit the concrete per-release automations to provision (from + config/automations.yaml + the release CCD), each with the exact steps it drives. + The skill creates each via m_create_automation, then `automation register`s it + with the same --step ids. Fails loudly if the config/step mapping has drifted.""" + config_path = getattr(args, "config", None) or C.DEFAULT_CONFIG + st = C.load_state(args.runs_root, args.release) + result = auto_plan.plan(config_path, args.release, getattr(st, "ccd", None)) + if args.json: + print(_json.dumps(result, indent=2)) + return 0 + if result["problems"]: + print("⚠ automation mapping problems (fix config/automations.yaml or step fire_at_local):") + for p in result["problems"]: + print(f" - {p}") + print() + if not result["ccd"]: + print(f"Release {args.release} has no CCD yet — set it before provisioning CCD automations.") + for a in result["automations"]: + sched = a.get("schedule") or "(no CCD → schedule unknown)" + print(f"• {a['name']}") + print(f" slug: {a['slug']}") + print(f" schedule: {sched} (one-shot; fires {a.get('fire_at')} on CCD {a.get('ccd_date')})") + print(f" drives: {', '.join(a['steps'])}") + print(f" purpose: {a['purpose']}") + print(f" register: automation register --id --name \"{a['name']}\" " + f"--release {args.release} --purpose \"{a['purpose']}\" " + f"--slug \"{a['slug']}\" --schedule \"{sched}\" " + + " ".join(f"--step {s}" for s in a["steps"])) + return 0 if not result["problems"] else 1 + + +def _cmd_sync(args): + """Detect step-driving automations whose SCHEDULE is stale vs the current CCD, and + emit what to change so the skill can re-apply it via m_update_automation. The CCD-day + automations are cron-pinned to the CCD; if the CCD moves (set-ccd) the live schedule + must move with it. Matches each REGISTERED automation to its desired schedule by the + set of steps it drives. Emits {release, ccd, updates:[{id, name, steps, + current_schedule, desired_schedule, changed}], problems}. + + The skill: for every entry with changed=true → m_update_automation(id, + schedule=desired_schedule), then `automation register` it again WITH --schedule + so the registry records the newly-applied schedule.""" + config_path = getattr(args, "config", None) or C.DEFAULT_CONFIG + st = C.load_state(args.runs_root, args.release) + ccd = getattr(st, "ccd", None) + reg = AutomationRegistry(args.runs_root, getattr(args, "release", None)) + registered = reg.list(release=args.release, kind="step-driving") + plan = auto_plan.plan(config_path, args.release, ccd) + desired_by_slug = {a["slug"]: a for a in plan["automations"]} + + updates = [] + for e in registered: + spec = desired_by_slug.get(e.get("slug")) # matched by slug (stable, unambiguous) + if spec is None: + continue # no matching desired spec — skip + desired = spec.get("schedule") + current = e.get("schedule") + updates.append({ + "id": e["id"], "name": e["name"], "slug": e.get("slug") or spec["slug"], + "steps": e.get("steps") or [], + "current_schedule": current, "desired_schedule": desired, + "changed": bool(desired) and desired != current, + }) + result = {"release": args.release, "ccd": ccd, "problems": plan["problems"], + "updates": updates} + + if args.json: + print(_json.dumps(result, indent=2)) + return 0 + if not ccd: + print(f"Release {args.release} has no CCD — set it before syncing automations.") + return 1 + changed = [u for u in updates if u["changed"]] + if not registered: + print(f"No step-driving automations registered for release {args.release}.") + elif not changed: + print(f"All {len(updates)} CCD automation(s) already in sync with CCD {ccd}.") + else: + print(f"{len(changed)} automation(s) need a schedule update for CCD {ccd}:") + for u in changed: + print(f" • {u['name']} ({u['id']}): {u['current_schedule']} → {u['desired_schedule']}") + print(f" m_update_automation(id={u['id']}, schedule=\"{u['desired_schedule']}\"), " + f"then re-register with --schedule \"{u['desired_schedule']}\"") + return 0 + + +def register(sub): + au = sub.add_parser("automation", help="Track provisioned automations (plan/register/list/deregister/sync) for teardown + CCD re-pin") + au.add_argument("action", choices=["plan", "register", "list", "deregister", "sync"]) + au.add_argument("--id", default=None, help="Scout automation id") + au.add_argument("--name", default="", help="Automation name (for register)") + au.add_argument("--release", default=None, help="Release scope (omit + --shared for machine-wide)") + au.add_argument("--shared", action="store_true", help="Mark as shared/persistent (not torn down per release)") + au.add_argument("--scope", default=None, choices=["shared", "release"], help="Filter list by scope") + au.add_argument("--kind", default=None, choices=["release-level", "step-driving"], + help="For register: override the auto-derived kind. For list: filter by kind.") + au.add_argument("--step", action="append", default=[], + help="For register: a '.' id this automation drives (repeatable)") + au.add_argument("--step-filter", default=None, dest="step_filter", + help="For list: show only automations that drive this '.' id") + au.add_argument("--purpose", default="", help="Short description") + au.add_argument("--schedule", default=None, + help="For register: the Scout schedule the automation was created with (stored so `sync` can detect CCD drift)") + au.add_argument("--slug", default=None, + help="For register: the stable slug from automations.yaml (sync's unambiguous match key)") + au.add_argument("--json", action="store_true") + au.set_defaults(func=cmd_automation) diff --git a/release-agent/orchestrator/commands/infra_cmd.py b/release-agent/orchestrator/commands/infra_cmd.py new file mode 100644 index 00000000..38df422e --- /dev/null +++ b/release-agent/orchestrator/commands/infra_cmd.py @@ -0,0 +1,53 @@ +"""Infrastructure preflight command: check CLIs + register/verify MCP servers in +Scout. Named infra_cmd to avoid clashing with the orchestrator.infra module.""" +from __future__ import annotations +import json as _json + +from orchestrator import infra +from orchestrator import cli_common as C + + +def cmd_infra(args): + """Infrastructure preflight: check CLI/host deps and register + verify the + MCP servers the skill needs in Scout. Run before the tool-level requirements. + Registers missing MCP servers into Scout's config (backup first) unless + --no-register; --json for machine output.""" + report = infra.run(C.REQUIREMENTS_CONFIG, register=not getattr(args, "no_register", False)) + if getattr(args, "json", False): + print(_json.dumps(report, indent=2)) + return 0 if report["ok"] else 1 + print("Infrastructure preflight") + if not report.get("scout_present", True): + print(" ⛔ Microsoft Scout not detected (~/.scout missing).") + scout_req = next((r for r in report["requirements"] if r["id"] == "scout"), None) + if scout_req and scout_req.get("install"): + print(f" install: {scout_req['install']}") + print(" Install Scout FIRST, then re-run — MCP servers can't be registered without it.") + print(" CLIs / host:") + for r in report["requirements"]: + mark = "OK" if r["ok"] else "MISSING" + print(f" [{mark}] {r['name']}") + if not r["ok"] and r["install"]: + print(f" install: {r['install']}") + print(" MCP servers (Scout config):") + if not report["mcp_servers"]: + print(" (none required)") + for m in report["mcp_servers"]: + label = {"present": "OK", "registered": "REGISTERED", "would_register": "MISSING", + "provider_missing": "PROVIDER MISSING", "launcher_missing": "LAUNCHER MISSING", + "scout_missing": "SCOUT NOT INSTALLED"}.get(m["status"], m["status"].upper()) + print(f" [{label}] {m['name']} — {m['detail']}") + if report["restart_needed"]: + print("\n ⚠ RESTART Scout to load newly-registered MCP server(s).") + if not report["ok"]: + print("\n Some infrastructure is missing — resolve the items above, then re-run.") + else: + print("\n Infrastructure OK.") + return 0 if report["ok"] else 1 + + +def register(sub): + inf = sub.add_parser("infra", help="Infrastructure preflight: check CLIs + register/verify MCP servers in Scout") + inf.add_argument("--no-register", action="store_true", help="Only report; don't register missing MCP servers") + inf.add_argument("--json", action="store_true") + inf.set_defaults(func=cmd_infra) diff --git a/release-agent/orchestrator/commands/localization.py b/release-agent/orchestrator/commands/localization.py new file mode 100644 index 00000000..bd5bb4d4 --- /dev/null +++ b/release-agent/orchestrator/commands/localization.py @@ -0,0 +1,145 @@ +"""Localization poll commands — the recorder/decider seam for the Phase-1 +`localization` step (P1-2). + +The step's logic lives in `steps/ccd/localization.py` (trigger + pure `decide`). +These commands are the thin CLI seam the skill/poller calls: + + * record-localization-run — after the pipeline is triggered, store the queued + build id + start time on the step (leaves it IN-FLIGHT, not done). + * check-localization — one poll: given the run's completion state (and the + OneLocBuild@3 log when complete), apply `decide()` and either wait, escalate + (email the engineer) + hold, or finish (post the PR to Code reviews + mark done, + or mark done with no strings). Prints the decision JSON so the poller can perform + the email/chat side-effect described in it. +""" +from __future__ import annotations +import json as _json +from datetime import datetime, timezone + +from orchestrator import cli_common as C +from orchestrator import mocks as mocks_mod +from steps.lib.context import SELF_CHAT_ID +from steps.ccd import localization as L + + +def _now_iso(): + return datetime.now(timezone.utc).isoformat() + + +def cmd_record_localization_run(args): + """Store the triggered build id + start time on the localization step. Leaves the + step in-flight (pending) so the poller can drive it to completion.""" + st = C.load_state(args.runs_root, args.release) + step = st.get_step("ccd", "localization") + step.data["build_id"] = args.build_id + step.data["started_at"] = args.started_at or _now_iso() + if args.run_url: + step.data["run_url"] = args.run_url + st.set_step("ccd", "localization", step) + C.save_state(st, args.runs_root, args.release) + C.emit(args.runs_root, args.release, + f"[localization] pipeline triggered — build {args.build_id}; polling every " + f"{L.CONFIG['poll_interval_min']}m (timeout {L.CONFIG['timeout_hours']}h).", + kind="localization") + return 0 + + +def _truthy(v) -> bool: + return str(v).strip().lower() in ("1", "true", "yes", "y", "complete", "completed", "succeeded") + + +def cmd_check_localization(args): + """One poll of the localization run. Reads the stored start time, applies the + deterministic decision, records terminal state, and prints the decision JSON.""" + now = None + if args.now: + try: + now = datetime.fromisoformat(args.now.replace("Z", "+00:00")) + except ValueError: + print(_json.dumps({"error": f"bad --now: {args.now!r}"})) + return 1 + + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + step = st.get_step("ccd", "localization") + + # Guard: nothing to poll if it wasn't triggered, or it's already terminal. + if not step.data.get("started_at"): + print(_json.dumps({"decision": "not_started", + "note": "localization has not been triggered yet"})) + return 0 + if step.status in ("done", "skipped", "blocked"): + print(_json.dumps({"decision": "already_final", "status": step.status})) + return 0 + + logs = args.logs + if logs is None and args.logs_file: + try: + with open(args.logs_file, "r", encoding="utf-8") as fh: + logs = fh.read() + except OSError as e: + print(_json.dumps({"error": f"could not read --logs-file: {e}"})) + return 1 + + decision = L.decide(st, is_complete=_truthy(args.complete), logs=logs, now=now) + d = decision["decision"] + + # mocks.local.yaml send_to → redirect the completion PR post to your own chat. + if d == "complete_pr" and decision.get("chat"): + spec = mocks_mod.load_mocks().get("ccd.localization") or {} + if "send_to" in spec: + val = spec["send_to"] + val = {"me": SELF_CHAT_ID, "self": SELF_CHAT_ID}.get(val, val) + decision["chat"]["chatId"] = val + decision["test_redirect"] = {"send_to": val} + + if d == "wait": + # Not terminal — keep in-flight, just record progress on the step. + step.data["last_checked"] = now.isoformat() if now else _now_iso() + step.note = decision["note"] + st.set_step("ccd", "localization", step) + C.save_state(st, args.runs_root, args.release) + C.emit(args.runs_root, args.release, f"[localization] {decision['note']}", kind="localization") + elif d == "timeout": + # Hold the step for the engineer; the poller sends decision['email']. + orch.record_scout_step("ccd", "localization", "attention", decision["note"]) + C.save_state(orch.state, args.runs_root, args.release) + C.emit(args.runs_root, args.release, f"[attention] localization: {decision['note']}", + kind="localization") + else: # complete_pr | complete_none → done + orch.record_scout_step("ccd", "localization", "pass", decision["note"]) + done = orch.state.get_step("ccd", "localization") + done.by = "scout" + done.data = step.data # preserve build id / start time + if decision.get("links"): + done.links = decision["links"] # the PR link + orch.state.set_step("ccd", "localization", done) + C.save_state(orch.state, args.runs_root, args.release) + C.emit(args.runs_root, args.release, f"[ok] localization: {decision['note']}", + kind="localization") + + print(_json.dumps(decision)) + return 0 + + +def register(sub): + rr = sub.add_parser("record-localization-run", + help="Record the triggered localization build id + start time (leaves it in-flight)") + rr.add_argument("--release", required=True) + rr.add_argument("--build-id", required=True, dest="build_id") + rr.add_argument("--run-url", default=None, dest="run_url") + rr.add_argument("--started-at", default=None, dest="started_at", + help="ISO-8601 start time; defaults to now") + rr.set_defaults(func=cmd_record_localization_run) + + cl = sub.add_parser("check-localization", + help="One poll of the localization run: wait / escalate (email) / finish (post PR)") + cl.add_argument("--release", required=True) + cl.add_argument("--complete", default="false", + help="Whether the pipeline run has finished (true/false/succeeded)") + cl.add_argument("--logs", default=None, + help="OneLocBuild@3 task log text (when complete) to scan for the PR id") + cl.add_argument("--logs-file", default=None, dest="logs_file", + help="Path to the OneLocBuild@3 log instead of --logs") + cl.add_argument("--now", default=None, help="Override 'now' (ISO-8601) for elapsed/timeout math") + cl.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + cl.set_defaults(func=cmd_check_localization) diff --git a/release-agent/orchestrator/commands/lockdown.py b/release-agent/orchestrator/commands/lockdown.py new file mode 100644 index 00000000..1ca4ede9 --- /dev/null +++ b/release-agent/orchestrator/commands/lockdown.py @@ -0,0 +1,48 @@ +"""Lockdown / CCOA overlap check — the recorder seam (Phase-0 step `lockdown`, S1). + +The step's logic now lives in `steps/preflight/lockdown.py` (one home): the gather +description (`build`), the deterministic overlap rule (`overlapping_periods`), and +the decision (`decide`). This module is the thin CLI recorder: the skill scrapes +the AAD-gated CCOA page and passes the periods here as JSON; `check-lockdown` +decides overlap DETERMINISTICALLY (not the LLM) and records the step result: + * no Production-env CCOA overlaps the release window -> step passes. + * one or more overlap -> step holds for the owner (who shifts CCD via set-ccd). + +`overlapping_periods` is re-exported so existing imports keep working. +""" +from __future__ import annotations +import json as _json + +from orchestrator import cli_common as C +from steps.preflight.lockdown import overlapping_periods, decide # noqa: F401 (re-export) + + +def cmd_check_lockdown(args): + st = C.load_state(args.runs_root, args.release) + if not st.ccd: + print("No CCD set for this release — cannot compute the release window.") + return 1 + try: + raw = _json.loads(args.periods_json or "[]") + except ValueError: + print("Could not parse --periods-json (expected a JSON array).") + return 1 + + status, detail = decide(st, raw) + _, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + orch.record_scout_step("preflight", "lockdown", status, detail) + C.save_state(orch.state, args.runs_root, args.release) + tag = "ok" if status == "pass" else "attention" + lead = "Lockdown check" if status == "pass" else "Lockdown overlap" + C.emit(args.runs_root, args.release, f"[{tag}] {lead}: {detail}", kind="lockdown") + return 0 + + +def register(sub): + cl = sub.add_parser("check-lockdown", + help="Decide CCOA lockdown overlap from scraped periods and record the step") + cl.add_argument("--release", required=True) + cl.add_argument("--periods-json", required=True, + help='JSON array of {name, environment, start, end} (dates YYYY-MM-DD, UTC)') + cl.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + cl.set_defaults(func=cmd_check_lockdown) diff --git a/release-agent/orchestrator/commands/logs.py b/release-agent/orchestrator/commands/logs.py new file mode 100644 index 00000000..a74f61ac --- /dev/null +++ b/release-agent/orchestrator/commands/logs.py @@ -0,0 +1,77 @@ +"""Event-log commands: log (show/analyze) and journal (record interaction).""" +from __future__ import annotations +import json as _json + +from orchestrator.eventlog import EventLog, summarize +from orchestrator import cli_common as C + + +def cmd_log(args): + """Show or analyze this release's event log (per-release only).""" + el = EventLog(args.runs_root, args.release) + events = el.read(args.limit) + if args.analyze: + print(_json.dumps(summarize(events), indent=2)) + return 0 + if args.json: + print(_json.dumps(events, indent=2)) + return 0 + if not events: + print("No events logged yet.") + return 0 + for e in events: + src = e.get("source", "engine") + loc = f" {e['phase']}/{e.get('step','')}" if e.get("phase") else "" + extra = "" + if e.get("event") == "step_qa": + q = (e.get("question") or "").replace("\n", " ") + a = (e.get("answer") or "").replace("\n", " ") + extra += f" Q=\"{q[:60]}{'…' if len(q) > 60 else ''}\" A=\"{a[:60]}{'…' if len(a) > 60 else ''}\"" + if e.get("driver"): + extra += f" driver=\"{e['driver']}\"" + if e.get("text"): + t = e["text"].replace("\n", " ") + extra += f" \"{t[:80]}{'…' if len(t) > 80 else ''}\"" + if e.get("choice"): + extra += f" choice={e['choice']}" + print(f" {e['ts']} {src:<6} {e.get('actor','?'):<10} {e['event']}{loc}{extra}") + return 0 + + +def cmd_journal(args): + """Record an INTERACTION event (what Scout showed / what the user chose). + The skill calls this so the per-release log captures the real conversation + for debugging. Best-effort; never affects the flow.""" + el = C.elog(args.runs_root, args.release) + if args.kind == "qa": + el.qa(args.question or args.text or "", args.answer or "", + phase=args.phase or None, step=args.step or None) + return 0 + if args.source == "scout": + el.scout_said(args.text or "", kind=args.kind or "message", options=args.option or None) + else: + el.user_said(args.text or "", kind=args.kind or "input", choice=args.choice or None) + return 0 + + +def register(sub): + lg = sub.add_parser("log", help="Show or analyze this release's event log") + lg.add_argument("--release", required=True) + lg.add_argument("--analyze", action="store_true", help="Print a rolled-up summary") + lg.add_argument("--limit", type=int, default=None) + lg.add_argument("--json", action="store_true") + lg.set_defaults(func=cmd_log) + + jn = sub.add_parser("journal", help="Record an interaction event (scout output / user input / step Q&A)") + jn.add_argument("--release", required=True) + jn.add_argument("--source", choices=["scout", "user"], default="user", + help="Who produced it (ignored for --kind qa, which is two-sided)") + jn.add_argument("--text", default="", help="What was shown / said") + jn.add_argument("--kind", default="", help="e.g. prompt, checklist, message, choice, input, qa") + jn.add_argument("--choice", default="", help="For user: the option id/label chosen") + jn.add_argument("--option", action="append", help="For scout: an option presented (repeatable)") + jn.add_argument("--question", default="", help="For --kind qa: the question the user asked") + jn.add_argument("--answer", default="", help="For --kind qa: the answer Scout gave") + jn.add_argument("--phase", default="", help="For --kind qa: the phase the question was about") + jn.add_argument("--step", default="", help="For --kind qa: the step the question was about") + jn.set_defaults(func=cmd_journal) diff --git a/release-agent/orchestrator/commands/notice.py b/release-agent/orchestrator/commands/notice.py new file mode 100644 index 00000000..66ea1b68 --- /dev/null +++ b/release-agent/orchestrator/commands/notice.py @@ -0,0 +1,33 @@ +"""`record-step` — generic recorder for a scout-assisted phase step. + +The old `prepare-notice` / `prepare-flight-reminder` commands are gone: scout +steps are now resolved by the generic `step-action` command (which returns the +step's uniform NeedsSkill payload) and the skill records the result here. +""" +from __future__ import annotations + +from orchestrator import cli_common as C + + +def cmd_record_step(args): + """Record a scout-assisted phase step result (skill calls this after doing the + out-of-engine work, e.g. sending the notice email).""" + _, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + act = orch.record_scout_step(args.phase, args.step, args.status, args.detail or "") + C.save_state(orch.state, args.runs_root, args.release) + C.emit(args.runs_root, args.release, + f"[{'ok' if args.status == 'pass' else 'attention'}] {args.step}: {act.message}", + kind="step") + return 0 + + +def register(sub): + rs = sub.add_parser("record-step", + help="Record a scout-assisted phase step result (pass|attention)") + rs.add_argument("--release", required=True) + rs.add_argument("--phase", default="preflight") + rs.add_argument("--step", required=True) + rs.add_argument("--status", required=True, choices=["pass", "attention"]) + rs.add_argument("--detail", default="") + rs.add_argument("--as-of", default=None) + rs.set_defaults(func=cmd_record_step) diff --git a/release-agent/orchestrator/commands/notify.py b/release-agent/orchestrator/commands/notify.py new file mode 100644 index 00000000..3fd261f4 --- /dev/null +++ b/release-agent/orchestrator/commands/notify.py @@ -0,0 +1,140 @@ +"""Notification + owner commands: notify (daily phase digest) and set-owner.""" +from __future__ import annotations +import json as _json +import os + +from orchestrator.state import ReleaseState +from orchestrator.engine import Orchestrator +from orchestrator import render, schedule +from orchestrator import notifications as notif +from orchestrator import cli_common as C +from tools import checks + + +def _empty_payload(rid, config_path=None): + """The 'nothing to send' payload — still reports which channels are configured + so callers see a stable shape.""" + ch = notif.channels(notif.load_config(config_path)) if config_path else {"email": True, "teams": False} + return {"message": "", "html": "", "subject": "", "owner_email": None, + "owner_name": None, "release": rid, "channels": ch, "teams": None} + + +def cmd_set_owner(args): + """Set/change the release owner (who reminders are emailed to).""" + st = C.load_state(args.runs_root, args.release) + st.owner_email = (getattr(args, "owner_email", None) or checks.current_az_user()) + if getattr(args, "owner_name", None): + st.owner_name = args.owner_name + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log("owner_set", owner=st.owner_email) + if not st.owner_email: + print("Couldn't resolve an owner (no --owner-email and az user unavailable).") + return 1 + who = f"{st.owner_name + ' ' if st.owner_name else ''}{st.owner_email}" + print(f"Release {args.release} owner set to {who}.") + return 0 + + +def cmd_notify(args): + """Emit the daily phase digest IF the active phase is open with outstanding + work, else nothing. Read-only (does NOT advance the flow — use `tick` for that). + De-duped to one per calendar day; --force bypasses; --json prints the mailer + payload {message,subject,owner_email,owner_name,release}.""" + rid = C.resolve_release_id(args.runs_root, args.release) + want_json = getattr(args, "json", False) + if not rid: + if want_json: + print(_json.dumps(_empty_payload(None, getattr(args, "config", None)))) + return 0 + payload = _notify_payload(args, rid, advance=False) + if want_json: + print(_json.dumps(payload)) + elif payload["message"]: + print(payload["message"]) + return 0 + + +def _notify_payload(args, rid, advance): + """Shared by `notify` and `tick`. Optionally ADVANCE the flow first + (run_until_gate), then read the state machine and build the once-per-day + digest payload. Returns {message, subject, owner_email, owner_name, release}. + `message` is "" unless a digest is due AND not already sent today (or --force).""" + sp = C.state_path(args.runs_root, rid) + if not os.path.exists(sp): + return _empty_payload(rid, getattr(args, "config", None)) + as_of = C.parse_as_of(args) + if advance: + # Auto-advance: run every agent step that can run, holding at the first + # gate / action-needed. Idempotent — a no-op once holding or not due. + st, orch = C.load_orch(args.runs_root, rid, args.config, as_of) + actions = orch.run_until_gate() + C.save_state(st, args.runs_root, rid) + C.log_actions(C.elog(args.runs_root, rid), actions, state=st) + else: + st = ReleaseState.load(sp) + orch = Orchestrator(args.config, st, as_of=as_of) + report = orch.status_report() + msg = render.notification(report) + html = render.notification_html(report) + md = render.notification_markdown(report) + subject = render.notification_subject(report) + today = (as_of or schedule.today()).isoformat() + fresh = bool(msg) and (getattr(args, "force", False) or st.last_notified_date != today) + if fresh: + st.last_notified_date = today + st.save(sp) + try: + C.elog(args.runs_root, rid).log("notified", text=msg, owner=st.owner_email) + except Exception: + pass + # Fan-out channels (config/notifications.yaml). Email is the existing path; when + # Teams is on and a digest is actually due, attach a delivery descriptor (Scout + # bot by default, or an explicit chat). + ncfg = notif.load_config(getattr(args, "config", None)) + ch = notif.channels(ncfg) + teams = notif.teams_delivery(ncfg, html, msg, md) if (fresh and msg and ch.get("teams")) else None + return {"message": msg if fresh else "", "html": html if fresh else "", + "subject": subject, "owner_email": st.owner_email, + "owner_name": st.owner_name, "release": rid, + "channels": ch, "teams": teams} + + +def cmd_tick(args): + """One automation heartbeat: discover the active release, ADVANCE it (run the + agent steps that can run, holding at gates/actions), then emit the daily digest + payload for the mailer. Safe to run often — advancing is idempotent and the + digest is de-duped to once per calendar day. This is what the hourly Scout + automation runs so an open phase makes progress even if the 9am tick was missed + (machine off) — the next tick after the machine is on picks it up.""" + rid = C.resolve_release_id(args.runs_root, args.release) + if not rid: + print(_json.dumps(_empty_payload(None, getattr(args, "config", None)))) + return 0 + payload = _notify_payload(args, rid, advance=True) + if getattr(args, "json", False): + print(_json.dumps(payload)) + elif payload["message"]: + print(payload["message"]) + return 0 + + +def register(sub): + so = sub.add_parser("set-owner", help="Set/change the release owner (who reminders are emailed to)") + so.add_argument("--release", required=True) + so.add_argument("--owner-email", default=None, help="Owner email (default: signed-in az user)") + so.add_argument("--owner-name", default=None, help="Owner display name (optional)") + so.set_defaults(func=cmd_set_owner) + + nt = sub.add_parser("notify", help="Emit a push line if something needs the user now (else nothing)") + nt.add_argument("--release", default=None, help="Target release; if omitted, discover the active one") + nt.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD) — debug override; default today") + nt.add_argument("--force", action="store_true", help="Bypass de-dup (always emit if actionable)") + nt.add_argument("--json", action="store_true", help="Emit {message,subject,owner_email,owner_name,release} for the mailer") + nt.set_defaults(func=cmd_notify) + + tk = sub.add_parser("tick", help="Automation heartbeat: ADVANCE the active release, then emit the digest payload") + tk.add_argument("--release", default=None, help="Target release; if omitted, discover the active one") + tk.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD) — debug override; default today") + tk.add_argument("--force", action="store_true", help="Bypass the once-per-day digest de-dup") + tk.add_argument("--json", action="store_true", help="Emit {message,subject,owner_email,owner_name,release} for the mailer") + tk.set_defaults(func=cmd_tick) diff --git a/release-agent/orchestrator/commands/pipeline.py b/release-agent/orchestrator/commands/pipeline.py new file mode 100644 index 00000000..d963ca8b --- /dev/null +++ b/release-agent/orchestrator/commands/pipeline.py @@ -0,0 +1,208 @@ +"""Pipeline-write commands (real production changes to ADO pipeline 3038): +set-ccd and skip-release. Both are gated (preview → --confirm) and audited. +Also `check-ccd` (read-only) — reconcile the release CCD with the pipeline override +for the entry gate.""" +from __future__ import annotations +import json as _json + +from orchestrator import schedule +from orchestrator import cli_common as C +from tools import checks + + +def cmd_check_ccd(args): + """Read-only CCD validation for the entry gate. Answers the complete + "is this CCD good?" question — temporally viable AND reconciled with the + pipeline. Classifies (priority order): + + unset — no CCD on the release (release id isn't YYYY-MM) + past — CCD is before today: INVALID, can't code-complete in the past (BLOCK) + conflict — pipeline override is an in-month date that DIFFERS from the CCD (BLOCK) + unreadable — couldn't read the pipeline (transient / auth) — attest fallback + match — CCD is future-dated and agrees with the pipeline + + Regardless of status (once a CCD is set) it also emits the calendar picture — + days_to_ccd, phase0_open, runway_days, compressed — so the gate can WARN when + Phase 0's normal CCD-7 window is squeezed (compressed) without blocking. + + Prints JSON. The skill records the `ccd_confirmed` gate item from this: + match→pass (warn if compressed); past/conflict→resolve via set-ccd then re-check; + unreadable→attest the CCD manually.""" + st = C.load_state(args.runs_root, args.release) + out = {"release": st.release_id, "ccd": st.ccd, "ccd_source": st.ccd_source, + "override": None, "ccd_conflict": None, "status": "unset"} + if not st.ccd: + if getattr(args, "json", False): + print(_json.dumps(out)) + else: + print("No CCD set for this release (release id isn't YYYY-MM). Use set-ccd.") + return 0 + + # Reconciliation with the pipeline (which date) — populate override/conflict first + # so the resolver has the pipeline value on hand even when temporal wins below. + recon = "match" + src = C.ccd_source() + if src.get("pipeline_id"): + ok, val, detail = checks.read_pipeline_variable( + src["org"], src["project"], src["pipeline_id"], src["override_variable"]) + if not ok: + recon = "unreadable" + out["detail"] = detail + else: + out["override"] = val or None + conflict = schedule.pipeline_conflict(st.release_id, val, st.ccd) + if conflict: + recon = "conflict" + out["ccd_conflict"] = conflict.isoformat() + st.ccd_conflict = conflict.isoformat() + else: + st.ccd_conflict = None + C.save_state(st, args.runs_root, args.release) + + # Temporal viability (whether that date is even runnable) — layered on top. + as_of = C.parse_as_of(args) or schedule.today() + ccd_d = schedule.parse_date(st.ccd) + via = schedule.ccd_viability(ccd_d, as_of) if ccd_d else {} + out.update({"today": as_of.isoformat(), **{k: via.get(k) for k in + ("days_to_ccd", "phase0_open", "runway_days", "compressed")}}) + # `past` is the worst problem — it overrides the reconciliation status (a past CCD + # must be rescheduled regardless of whether it matches the pipeline). override/ + # ccd_conflict stay populated so the resolver can offer a future override to adopt. + out["status"] = "past" if via.get("past") else recon + + if getattr(args, "json", False): + print(_json.dumps(out)) + return 0 + when = schedule.humanize_delta(via.get("days_to_ccd", 0)) if via else "" + warn = (f" ⚠ Phase 0 is compressed: only {via.get('runway_days')} of the normal " + f"{via.get('normal_window', 7)} prep days remain." if via.get("compressed") else "") + msg = { + "match": f"CCD {st.ccd} ({when}) is future-dated and reconciled with the pipeline.{warn}", + "compressed": "", # folded into match/others via `warn` + "past": f"⛔ CCD {st.ccd} is in the PAST ({when}) — reschedule to a future date " + f"with set-ccd before proceeding.", + "conflict": f"⚠ Pipeline override {out['ccd_conflict']} DIFFERS from CCD {st.ccd} " + f"({when}) — reconcile with set-ccd before proceeding.{warn}", + "unreadable": f"Couldn't read the pipeline override to validate CCD {st.ccd} " + f"({out.get('detail','')}). Confirm the CCD manually.{warn}", + "unset": "No CCD set.", + }[out["status"]] + print(msg) + return 0 + + +def cmd_set_ccd(args): + """Change the Code Complete Date. Writes the pipeline override (real change) — + requires --confirm and a --reason. Without --confirm, previews the write.""" + st = C.load_state(args.runs_root, args.release) + src = C.ccd_source() + if not src.get("pipeline_id"): + print("No CCD source configured (config/schedule.yaml).") + return 1 + if not (args.reason and args.reason.strip()): + print("A --reason is required (audited).") + return 1 + + if args.default: + new_ccd, source, value = schedule.default_ccd(args.release), "default", "" + what = f"clear the override → default {new_ccd.isoformat()} (2nd Wednesday)" + else: + d = schedule.parse_date(args.date) + if not d: + print(f"Bad --date '{args.date}' (expected YYYY-MM-DD).") + return 1 + ry, rm = schedule.parse_release_month(args.release) + if (d.year, d.month) != (ry, rm): + print(f"CCD {d.isoformat()} is not in release month {args.release}. The pipeline " + f"override is month-scoped, so a different month wouldn't apply. " + f"Use the release id for that month instead.") + return 1 + new_ccd, source, value = d, "manual", d.isoformat() + what = f"set CCD override → {value}" + + if not args.confirm: + print(f"[preview] Would {what} on pipeline {src['pipeline_id']} " + f"({src['override_variable']}).\n Re-run with --confirm to write it. Reason: {args.reason.strip()}") + return 0 + + res = C.write_ccd_var(src, value) + if not res.ok: + print(f"Failed to write pipeline variable: {res.detail}") + return 1 + st.ccd = new_ccd.isoformat() + st.ccd_source = source + st.ccd_conflict = None # the date is now settled — clear any conflict + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log( + "ccd_changed", value=value or "(default)", source=source, driver=args.reason.strip()) + opens = schedule.anchor_date(new_ccd, "CCD-7").isoformat() + # The CCD-day automations are cron-pinned to the CCD — a moved CCD makes them stale. + # Tell the skill to re-sync so the live schedules follow the new date. + sync_note = "" + try: + from orchestrator.registry import AutomationRegistry + sd = [e for e in AutomationRegistry(args.runs_root).list(release=args.release) + if e.get("kind") == "step-driving"] + if sd: + sync_note = ("\n ⚠ CCD-day automations are pinned to the CCD — run " + f"`automation sync --release {args.release} --json` and apply any " + "schedule updates (m_update_automation) so they fire on the new date.") + except Exception: + pass + C.emit(args.runs_root, args.release, + f"✅ CCD set to **{st.ccd}** ({source}); pipeline updated. " + f"Phase 0 opens {opens} (CCD-7).\n {res.detail}{sync_note}", kind="ccd") + return 0 + + +def cmd_skip_release(args): + """Suppress the release by setting the pipeline 'skipRelease' switch (real change).""" + st = C.load_state(args.runs_root, args.release) + src = C.ccd_source() + if not (args.reason and args.reason.strip()): + print("A --reason is required (audited).") + return 1 + clearing = bool(getattr(args, "clear", False)) + value = "" if clearing else "skipped" + verb = "clear" if clearing else "set" + if not args.confirm: + print(f"[preview] Would {verb} '{src.get('skip_variable')}' on pipeline " + f"{src.get('pipeline_id')}.\n Re-run with --confirm. Reason: {args.reason.strip()}") + return 0 + res = checks.set_pipeline_variable( + src["org"], src["project"], src["pipeline_id"], src["skip_variable"], value) + if not res.ok: + print(f"Failed to write pipeline variable: {res.detail}") + return 1 + st.skip_release = not clearing + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log( + "release_skip_cleared" if clearing else "release_skip_set", driver=args.reason.strip()) + msg = ("✅ Release un-skipped — pipeline will trigger normally." + if clearing else + "🛑 Release marked SKIP in the pipeline — the monthly trigger is suppressed until cleared.") + C.emit(args.runs_root, args.release, msg + f"\n {res.detail}", kind="skip_release") + return 0 + + +def register(sub): + cc = sub.add_parser("check-ccd", help="Validate the release CCD — past/reconciled/compressed (read-only; for the entry gate)") + cc.add_argument("--release", required=True) + cc.add_argument("--as-of", dest="as_of", default="", help="Simulated clock YYYY-MM-DD (default: today)") + cc.add_argument("--json", action="store_true", help="Emit {ccd,override,ccd_conflict,status,days_to_ccd,runway_days,compressed,...}") + cc.set_defaults(func=cmd_check_ccd) + + sc = sub.add_parser("set-ccd", help="Change the Code Complete Date (writes pipeline override; --confirm)") + sc.add_argument("--release", required=True) + sc.add_argument("--date", default="", help="New CCD (YYYY-MM-DD), must be in the release month") + sc.add_argument("--default", action="store_true", help="Clear the override → 2nd-Wednesday default") + sc.add_argument("--reason", default="", help="Why (audited — required)") + sc.add_argument("--confirm", action="store_true", help="Actually write to the pipeline (else preview)") + sc.set_defaults(func=cmd_set_ccd) + + sr = sub.add_parser("skip-release", help="Suppress/cancel the release via the pipeline switch (--confirm)") + sr.add_argument("--release", required=True) + sr.add_argument("--clear", action="store_true", help="Clear the skip (re-enable the release)") + sr.add_argument("--reason", default="", help="Why (audited — required)") + sr.add_argument("--confirm", action="store_true", help="Actually write to the pipeline (else preview)") + sr.set_defaults(func=cmd_skip_release) diff --git a/release-agent/orchestrator/commands/rc_poll.py b/release-agent/orchestrator/commands/rc_poll.py new file mode 100644 index 00000000..124ef6d1 --- /dev/null +++ b/release-agent/orchestrator/commands/rc_poll.py @@ -0,0 +1,129 @@ +"""`poll-rc` — one poll of an in-flight Phase-2 RC verification (the 30-min RC poller). + +After a re-triggered RC (see `rc-retriggered`), the Build & RC Verification phase holds +on an IN-FLIGHT MRWP run (status-aware verify — see steps/build_verify/_common). This +command is the poller seam the `build-verify-rc-poller` automation calls every 30 min: + + 1. advance the drain (`run_until_gate`) so the in-flight verify step re-checks the run's + live status — still running → stays in-flight; completed → the normal stage rule + + UI gate apply and the phase moves on. + 2. emit a deterministic decision the skill acts on: + waiting — still running; nothing to send. + nudge — running past the 6h courtesy threshold; send the owner a heads-up (once). + resolved — the new RC completed and PASSED the gate; Phase 2 advanced (deregister + the poller). + blocked — the new RC completed but re-blocked the gate (still failing). + idle — nothing in-flight (not in Phase 2, or nothing was re-triggered). + +Decisions are pure functions of state; the 6h nudge stamps `nudged_at` on the step so it +is sent at most once. `--now` overrides the clock for the elapsed/nudge math (tests).""" +from __future__ import annotations +import json as _json +from datetime import datetime, timezone + +from orchestrator import cli_common as C + +# The poll cadence + courtesy-nudge threshold. A re-triggered RC that runs longer than +# NUDGE_AFTER_HOURS gets ONE heads-up to the owner (it is not a failure — Scout keeps +# polling), per the agreed Phase-2 blocked-state handling. +POLL_INTERVAL_MIN = 30 +NUDGE_AFTER_HOURS = 6 + +# The verify steps whose run can be in-flight (checker/orchestrator resolve instantly). +_RC_VERIFY_STEPS = ("mrwp_ecs", "mrwp_local") + + +def _parse_iso(s): + try: + return datetime.fromisoformat(str(s).replace("Z", "+00:00")) + except (ValueError, TypeError): + return None + + +def _elapsed_hours(since_iso, now): + since = _parse_iso(since_iso) + if not since: + return 0.0 + if since.tzinfo is None: + since = since.replace(tzinfo=timezone.utc) + return max(0.0, (now - since).total_seconds() / 3600.0) + + +def _nudge_payload(st, sid, hrs: int) -> dict: + """A SHORT courtesy heads-up (not the full RC report) — the re-triggered RC is taking + a while but Scout is still polling; no action needed yet.""" + label = {"mrwp_ecs": "MRWP (ECS)", "mrwp_local": "MRWP (Local)"}.get(sid, sid) + subject = f"[Release {st.release_id}] Re-triggered RC still running after ~{hrs}h" + body = ( + f"Heads-up: the re-triggered {label} run for release {st.release_id} has been " + f"running for about {hrs} hours. This is NOT a failure — Scout is still polling " + f"every {POLL_INTERVAL_MIN} minutes and will re-apply the RC gate the moment the " + f"run completes, with no action needed from you. If a {hrs}h RC run is unexpected, " + f"open the run in ADO to check for a stuck stage.") + teams = (f"⏳ Release {st.release_id}: the re-triggered {label} RC has been running " + f"~{hrs}h. Scout is still polling every {POLL_INTERVAL_MIN}m and re-applies " + f"the gate on completion — no action needed yet.") + return { + "email": {"to": [st.owner_email] if st.owner_email else [], + "subject": subject, "body": body}, + "teams": {"text": teams}, + } + + +def cmd_poll_rc(args): + now = _parse_iso(args.now) if getattr(args, "now", None) else datetime.now(timezone.utc) + if now is None: + print(_json.dumps({"error": f"bad --now: {args.now!r}"})) + return 1 + + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + # Advance: the in-flight verify step re-checks the run's LIVE status. Still running → + # stays in-flight; completed → the stage rule + UI gate run and the phase moves on. + orch.run_until_gate() + st = orch.state + C.save_state(st, args.runs_root, args.release) + + inflight = None + for sid in _RC_VERIFY_STEPS: + s = st.get_step("build_verify", sid) + if s.status == "in_flight": + inflight = (sid, s) + break + + if inflight: + sid, s = inflight + elapsed = _elapsed_hours(s.data.get("in_flight_since"), now) + decision = {"decision": "waiting", "step": sid, + "elapsed_hours": round(elapsed, 2), "poll_in_min": POLL_INTERVAL_MIN} + if elapsed >= NUDGE_AFTER_HOURS and not s.data.get("nudged_at"): + s.data["nudged_at"] = now.isoformat() + st.set_step("build_verify", sid, s) + C.save_state(st, args.runs_root, args.release) + decision["decision"] = "nudge" + decision["nudge"] = _nudge_payload(st, sid, int(elapsed)) + C.emit(args.runs_root, args.release, + f"[rc-poller] {sid} in-flight ~{int(elapsed)}h — 6h courtesy nudge sent " + f"to the owner.", kind="build_verify") + else: + rc = st.get_step("build_verify", "rc_report") + if rc.status == "done": + decision = {"decision": "resolved", "status": "passed", "note": rc.note} + elif rc.status == "blocked": + decision = {"decision": "blocked", "note": rc.note} + else: + decision = {"decision": "idle", + "note": "no in-flight RC in Build & RC Verification"} + + print(_json.dumps(decision)) + return 0 + + +def register(sub): + p = sub.add_parser("poll-rc", + help="One poll of an in-flight Phase-2 RC: advance + emit a " + "waiting/nudge/resolved/blocked/idle decision (30-min poller)") + p.add_argument("--release", required=True) + p.add_argument("--now", default=None, + help="Override 'now' (ISO-8601) for the elapsed / 6h-nudge math") + p.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + p.set_defaults(func=cmd_poll_rc) diff --git a/release-agent/orchestrator/commands/rc_report.py b/release-agent/orchestrator/commands/rc_report.py new file mode 100644 index 00000000..54a7163b --- /dev/null +++ b/release-agent/orchestrator/commands/rc_report.py @@ -0,0 +1,202 @@ +"""`rc-report` — the Phase 2 RC-pipeline + test status report (read-only). + +Assembles the release chain (checker → orchestrator → the two MRWP RC-testing runs) and +their Test-tab results into one view, on demand. Distinct from the build_verify steps +(which gate): this only REPORTS — it never blocks or changes state. `--json` emits the +raw model for the skill; otherwise a formatted text report is printed. +""" +from __future__ import annotations +import json as _json + +from orchestrator import cli_common as C +from steps.build_verify import _common as K + + +def cmd_rc_report(args): + from tools import pipelines as P + st = C.load_state(args.runs_root, args.release) + month = getattr(st, "release_id", None) or args.release + model = P.release_report(K.ORG, K.PROJECT, month, + checker_def=K.CHECKER_DEF, orch_def=K.ORCHESTRATOR_DEF) + _persist(st, model, args) + if getattr(args, "json", False): + print(_json.dumps(model, indent=2)) + return 0 + print(_format(model)) + return 1 if model.get("problems") else 0 + + +def _persist(st, model, args): + """Record the resolved runs (+ snapshots) on state so status/digest/rc_report read + them without a live call. Best-effort — a report must never fail because the state + write did. (This is the LIVE `rc-report` diagnostic refreshing the record; the verify + steps are the primary writers.)""" + if st is None: + return + try: + ch = model.get("checker") or {} + if ch.get("run_id"): + K.stash_checker(st, ch["run_id"], ch.get("when")) + o = model.get("orchestrator") or {} + if o.get("run_id"): + K.stash_orchestrator(st, o["run_id"], + versions={k: v for k, v in (o.get("versions") or {}).items() if v}, + parked=o.get("parked")) + mr = model.get("mrwp") or {} + for slot in ("ECS", "Local"): + m = mr.get(slot) or {} + if m.get("run_id"): + K.stash_mrwp(st, slot, {k: m.get(k) for k in + ("run_id", "complete", "ran", "total", "failed_stages", + "yellow_stages", "never_ran", "tests", "failed_suites")}) + C.save_state(st, args.runs_root, args.release) + except Exception: + pass + + +def _u(build_id): + return K.build_url(build_id) if build_id else "" + + +def cmd_record_rc_report(args): + """Record the rc_report step's outcome AFTER the skill has emailed the RC report. + + Re-reads the live model, applies the three-tier UI-automation gate (K.rc_ui_gate), + records `pass` (>=90% UI pass — clean/warn → step done, release auto-advances into bug + bash) or `attention` (<90% → step BLOCKS for owner investigation), and stashes the + evaluated pipeline-run links on the step so its Details point at every artifact behind + the verdict. + + This is the follow-up the rc_report NeedsSkill names (`payload.followup_command`), so + the skill runs it instead of a blind `record-step --status pass`.""" + _, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + try: + model = K.rc_report_model(orch.state) + except Exception as e: # pragma: no cover - defensive + print(_json.dumps({"error": f"could not build the RC model ({e})."})) + return 1 + + gate = K.rc_ui_gate(model) + links = K.rc_run_links(model) + status = "attention" if gate["blocking"] else "pass" + orch.record_scout_step("build_verify", "rc_report", status, gate["detail"]) + + # record_scout_step doesn't carry links — attach the evaluated-run refs (and stamp + # the recorder as scout) on the resulting step, preserving its status/note. + step = orch.state.get_step("build_verify", "rc_report") + step.links = links + step.by = "scout" + orch.state.set_step("build_verify", "rc_report", step) + C.save_state(orch.state, args.runs_root, args.release) + + C.emit(args.runs_root, args.release, + f"[{'ok' if status == 'pass' else 'attention'}] rc_report: " + f"{gate['detail'].splitlines()[0]}", kind="step") + print(_json.dumps({"verdict": gate["verdict"], "status": status, + "pass_pct": gate["pass_pct"], "ui_total": gate["ui_total"], + "ui_failed": gate["ui_failed"], "threshold": gate["threshold"], + "detail": gate["detail"], "links": links})) + return 0 if status == "pass" else 2 + + +def _format(m) -> str: + L = [f"## RC Pipeline Status — Release {m['release']}", ""] + + ch = m.get("checker") or {} + if ch.get("fired"): + L.append(f"✅ **Code Complete Checker** fired the release — run {ch['run_id']} ({ch['when']}).") + elif "error" in ch: + L.append(f"⚠ **Code Complete Checker** — couldn't read ({ch['error']}).") + else: + L.append("⏳ **Code Complete Checker** — no triggering run yet (before Code Complete Day, or not fired).") + + o = m.get("orchestrator") or {} + if not o.get("found"): + err = f" ({o['error']})" if "error" in o else "" + L.append(f"⛔ **Release Orchestrator** — no run found{err}.") + else: + vstr = K.format_versions(o.get("versions"), fallback="versions n/a") + if o.get("healthy"): + park = "parked at 'Remove RC Tags' (awaiting owner approval)" if o.get("parked") \ + else f"'{o.get('park_stage')}' already cleared" + L.append(f"✅ **Release Orchestrator** run {o['run_id']} healthy — pre-gate stages green, {park}. {vstr}.") + else: + L.append(f"⛔ **Release Orchestrator** run {o['run_id']} — stage(s) not green: " + f"{', '.join(o.get('failed_stages') or [])}. {vstr}.") + L.append(f" {_u(o.get('run_id'))}") + + for provider in ("ECS", "Local"): + r = (m.get("mrwp") or {}).get(provider) + if not r: + continue + if "error" in r: + L.append(f"⚠ **MRWP {provider}** run {r.get('run_id')} — couldn't read stages ({r['error']}).") + continue + icon = "✅" if r.get("complete") else "⛔" + verdict = "ran to completion" if r.get("complete") else "did NOT run to completion" + extras = [] + if r.get("failed_stages"): + extras.append(f"{len(r['failed_stages'])} red") + if r.get("yellow_stages"): + extras.append(f"{len(r['yellow_stages'])} yellow") + ex = f" ({', '.join(extras)})" if extras else "" + L.append(f"{icon} **MRWP {provider}** run {r['run_id']} — {verdict}: {r.get('ran')}/{r.get('total')} stages{ex}.") + if not r.get("complete") and r.get("never_ran"): + L.append(f" never ran: {', '.join(n for n in r['never_ran'] if n)}") + t = r.get("tests") or {} + cats = t.get("categories") or {} + _lbl = {"unit": "Unit", "instrumented": "Instrumented", "ui": "UI automation"} + for cat in ("unit", "instrumented", "ui"): + c = cats.get(cat) or {} + if not c.get("total"): + continue + fr = round((c.get("failed", 0)) * 100.0 / c["total"], 1) + gate = " ← RC gate" if cat == "ui" else "" + L.append(f" {_lbl[cat]:13} {c.get('passed')}/{c.get('total')} passed · " + f"{c.get('failed')} failed · {fr}%{gate}") + # Failing tests, grouped by suite (UI first), each tagged by category. + suites = r.get("failed_suites") + if suites: + for s in K.sort_failed_suites(suites): + cat = _lbl.get(s.get("category", "ui"), "UI automation") + fr = round(s["failed"] * 100.0 / s["total"], 1) if s["total"] else 0.0 + L.append(f" • [{cat}] {s['name']}: {s['failed']}/{s['total']} failed ({fr}%)") + for tname in s.get("tests", []): + L.append(f" - {tname}") + shown = len(s.get("tests", [])) + if shown < s["failed"]: + L.append(f" … and {s['failed'] - shown} more (see the run)") + elif t and t.get("failed"): + L.append(f" (failing test names unavailable — open the run)") + L.append(f" {_u(r.get('run_id'))}") + + probs = m.get("problems") or [] + if probs: + L += ["", "**Issues:**"] + for p in probs: + L.append(f" - {p}") + # Unit retry warning — failed-then-passed on retry (counted as passed). + recovered = K.recovered_unit_tests(m) + if recovered: + L += ["", f"⚠ **Retry warning** — {len(recovered)} unit test(s) failed then passed " + f"on retry (counted as passed):"] + for t in recovered[:20]: + L.append(f" - {t}") + if len(recovered) > 20: + L.append(f" … and {len(recovered) - 20} more") + return "\n".join(L) + + +def register(sub): + rp = sub.add_parser("rc-report", help="Phase 2 RC-pipeline + test status report (read-only)") + rp.add_argument("--release", required=True) + rp.add_argument("--json", action="store_true", help="Emit the raw report model") + rp.set_defaults(func=cmd_rc_report) + + rr = sub.add_parser( + "record-rc-report", + help="Record the rc_report step after emailing: apply the 90%% UI gate " + "(pass|attention/block) + stash the evaluated run links") + rr.add_argument("--release", required=True) + rr.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + rr.set_defaults(func=cmd_record_rc_report) diff --git a/release-agent/orchestrator/commands/readiness.py b/release-agent/orchestrator/commands/readiness.py new file mode 100644 index 00000000..ec6bc994 --- /dev/null +++ b/release-agent/orchestrator/commands/readiness.py @@ -0,0 +1,154 @@ +"""Readiness entry-gate commands: checklist, verify, sign, decline.""" +from __future__ import annotations +import json as _json + +from orchestrator import render +from orchestrator import cli_common as C + + +def cmd_checklist(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + if getattr(args, "verify", False): + orch.gate.verify() + C.save_state(st, args.runs_root, args.release) + chk = orch.gate.checklist() + attest = getattr(args, "attest_prompt", False) + if getattr(args, "json", False): + # `--attest-prompt --json` → the DETERMINISTIC m_ask_user payload (question + + # answer cards) the skill passes straight to m_ask_user, so the always-rendered + # Scout card is the source of truth regardless of whether the table rendered. + if attest: + print(_json.dumps(render.attest_prompt_payload(chk, args.release), indent=2)) + else: + print(_json.dumps(chk, indent=2)) + return 0 + if attest: + # Human-readable: ONLY the '✋ Your confirmation needed' block (no table). + C.emit(args.runs_root, args.release, render.attest_prompt(chk), kind="readiness_attest_prompt") + return 0 + # canonical, consistent display block (the template) — same for every engineer. + # auto-logged as scout output so the log always records what was shown. + C.emit(args.runs_root, args.release, render.readiness_table(chk, args.release), kind="readiness_checklist") + return 0 + + +def cmd_verify(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + chk = orch.gate.verify() + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log( + "readiness_verified", + results=[{"id": it["id"], "status": it["status"]} for it in chk["auto_items"]]) + for it in chk["auto_items"]: + mark = "OK" if it["status"] == "pass" else "FAIL" + print(f" [{mark}] {it['id']}: {it['status']} — {it.get('message','')}") + return 0 + + +def cmd_sign(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + ids = args.item or [] + if not ids: + # No blanket sign: the engineer must name the item(s) they confirmed. + # This closes the integrity hole where `sign --all` attested every human + # item in one blind call with no evidence and no per-item confirmation. + print("Refusing to sign: name the item(s) you confirmed with --item " + "(repeatable). Attest only what the engineer explicitly confirmed.") + return 2 + # Validate the ids are real attest items before recording anything. + chk_before = orch.gate.checklist() + attest_ids = {i["id"] for i in chk_before["attest_items"]} + unknown = [i for i in ids if i not in attest_ids] + if unknown: + print(f"Not attestable (unknown or not an attest item): {', '.join(unknown)}") + return 2 + chk = orch.gate.sign(ids, note=args.note or None) + C.save_state(st, args.runs_root, args.release) + el = C.elog(args.runs_root, args.release) + # Log EACH attestation individually with its evidence note, so the trail shows + # exactly what was confirmed (not one opaque items:"all"). + for iid in ids: + el.log("readiness_attested", item=iid, driver=args.note or None) + el.log("readiness_signed" if chk["signed"] else "readiness_partial", + items=ids, signed=chk["signed"]) + if chk["signed"]: + print(f"Readiness signed at {chk['signed_at']}. Entry gate cleared — you can now start Phase 0.") + else: + pending = [i["id"] for i in chk["items"] if not i["satisfied"]] + print(f"Recorded {', '.join(ids)}. Still pending: {', '.join(pending)}") + return 0 + + +def cmd_decline(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + chk = orch.gate.decline(args.item or []) + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log( + "readiness_declined", items=args.item or [], blocked=chk["blocked"], driver=args.reason or None) + if chk["blocked"]: + labels = [next((i["label"] for i in chk["items"] if i["id"] == b), b) + for b in chk["blocked_items"]] + msg = ("⛔ BLOCKED — cannot start: " + ", ".join(labels) + ".\n" + " " + chk.get("blocked_message", "").strip()) + else: + msg = f"Recorded as unable: {', '.join(args.item or [])}" + C.emit(args.runs_root, args.release, msg, kind="decline_result") + return 0 + + +def cmd_record_check(args): + """Record the result of a scout-assisted auto readiness check (source: scout), + e.g. the ICM on-call lookup. The skill runs the check via its MCP tools and + calls this to store the pass/fail result in the engine.""" + st, orch = C.load_orch(args.runs_root, args.release, args.config) + res = orch.gate.record_check(args.item, args.status, args.detail or "") + if "error" in res: + print(res["error"]) + return 1 + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log( + "readiness_check_recorded", item=args.item, status=args.status, driver=args.detail or None) + item = next((i for i in res["items"] if i["id"] == args.item), None) + mark = {"pass": "OK", "degraded": "WARN"}.get(args.status, "FAIL") + tail = " — entry gate cleared." if res.get("signed") else "" + C.emit(args.runs_root, args.release, + f"[{mark}] {(item or {}).get('label', args.item)}: {args.status}" + f"{(' — ' + args.detail) if args.detail else ''}{tail}", kind="record_check") + return 0 + + +def register(sub): + c = sub.add_parser("checklist", help="Show the readiness entry-gate checklist") + c.add_argument("--release", required=True) + c.add_argument("--verify", action="store_true", help="Run auto verifiers before showing") + c.add_argument("--attest-prompt", dest="attest_prompt", action="store_true", + help="Emit ONLY the '✋ Your confirmation needed' block (no table) — the second readiness render") + c.add_argument("--json", action="store_true") + c.set_defaults(func=cmd_checklist) + + v = sub.add_parser("verify", help="Run the auto readiness verifiers") + v.add_argument("--release", required=True) + v.set_defaults(func=cmd_verify) + + rc = sub.add_parser("record-check", + help="Record a scout-assisted auto check result (e.g. ICM on-call)") + rc.add_argument("--release", required=True) + rc.add_argument("--item", required=True, help="Readiness item id (must be a source:scout auto item)") + rc.add_argument("--status", required=True, choices=["pass", "fail", "degraded"], + help="pass | fail (or 'degraded' for opt-out items the user proceeds without)") + rc.add_argument("--detail", default="", help="Short evidence/summary (e.g. 'not in roster')") + rc.set_defaults(func=cmd_record_check) + + sg = sub.add_parser("sign", help="Attest human readiness items (also runs auto verify)") + sg.add_argument("--release", required=True) + sg.add_argument("--item", action="append", + help="Attest a specific item id the engineer confirmed (repeatable, required)") + sg.add_argument("--note", default="", + help="Evidence: what the engineer confirmed (recorded per item)") + sg.set_defaults(func=cmd_sign) + + dc = sub.add_parser("decline", help="Declare you CANNOT satisfy an item (may block ownership)") + dc.add_argument("--release", required=True) + dc.add_argument("--item", action="append", required=True, help="Item id you cannot satisfy (repeatable)") + dc.add_argument("--reason", default="", help="Why (recorded in the event log)") + dc.set_defaults(func=cmd_decline) diff --git a/release-agent/orchestrator/commands/release.py b/release-agent/orchestrator/commands/release.py new file mode 100644 index 00000000..478fea2c --- /dev/null +++ b/release-agent/orchestrator/commands/release.py @@ -0,0 +1,361 @@ +"""Release lifecycle + manual overrides: init, list, status, next, approve, deny, +done, activate, skip, reopen, halt, resume.""" +from __future__ import annotations +import json as _json +import os + +from orchestrator.state import ReleaseState +from orchestrator.engine import Orchestrator +from orchestrator import discovery, render, schedule +from orchestrator import cli_common as C +from tools import checks + + +def cmd_init(args): + sp = C.state_path(args.runs_root, args.release) + if os.path.exists(sp) and not args.force: + print(f"Release {args.release} already exists at {sp} (use --force to recreate).") + return 1 + st = ReleaseState(release_id=args.release, status="not_started") + + # Release owner (the engineer running this release) — release metadata. + # Priority: explicit --owner-email (skill can pass the richer profile) then + # the signed-in az user. Never hardcoded. + st.owner_email = (getattr(args, "owner_email", None) or checks.current_az_user()) + st.owner_name = getattr(args, "owner_name", None) or None + owner_note = "" if st.owner_email else " (couldn't resolve owner — set with set-owner)" + + # Capture the owner's timezone NOW (at init, on their interactive machine) and + # persist it — so later headless automation runs, which may execute in a UTC process + # context, still evaluate due-ness + fire_at_local on the OWNER's clock. Priority: + # explicit --timezone, else auto-detect the local IANA zone; falls back to the + # config/schedule.yaml default when neither is available. + st.timezone = (getattr(args, "timezone", None) or schedule.detect_local_tz()) + tz_note = "" if st.timezone else " (couldn't detect — using default)" + + # CCD is canonically the 2nd Wednesday. We still READ the pipeline override, + # but we do NOT silently adopt it — if it differs, we flag a conflict for the + # user to resolve (2nd-Wed default vs the pipeline date). + src = C.ccd_source() + override, note = None, "" + if src.get("pipeline_id"): + ok, val, detail = checks.read_pipeline_variable( + src["org"], src["project"], src["pipeline_id"], src["override_variable"]) + if ok: + override = val + else: + note = f" (couldn't read pipeline override: {detail})" + try: + default = schedule.default_ccd(args.release) + st.ccd = default.isoformat() + st.ccd_source = "default" + conflict = schedule.pipeline_conflict(args.release, override, st.ccd) + st.ccd_conflict = conflict.isoformat() if conflict else None + except (ValueError, IndexError): + default, conflict = None, None + note = " (release id isn't YYYY-MM — CCD not set; use set-ccd)" + st.save(sp) + + C.elog(args.runs_root, args.release).log( + "release_started", forced=bool(args.force), + ccd=st.ccd, ccd_source=st.ccd_source, ccd_conflict=st.ccd_conflict, + owner=st.owner_email, timezone=st.timezone) + owner_line = f" Owner: {st.owner_name + ' ' if st.owner_name else ''}{st.owner_email or '(unresolved)'}{owner_note}" + tz_line = f" Timezone: {st.timezone or schedule.DEFAULT_TZ}{tz_note}" + if st.ccd: + opens = schedule.anchor_date(default, "CCD-7").isoformat() + lines = [f"Initialized release {args.release}.", f" state: {sp}", owner_line, tz_line, + f" Code Complete Date: {st.ccd} (2nd Wednesday){note}", + f" Phase 0 (Pre-flight) opens {opens} (CCD-7). Until then nothing fires."] + if conflict: + lines.append(f" ⚠ Pipeline override is {conflict.isoformat()}, which differs from the " + f"2nd-Wednesday default. Confirm which is the real CCD before proceeding.") + print("\n".join(lines)) + else: + print(f"Initialized release {args.release}.\n state: {sp}\n{owner_line}\n{tz_line}\n{note}") + return 0 + + +def cmd_list(args): + """List discovered releases (none/one/many). --json for the skill.""" + res = discovery.resolve(args.runs_root, getattr(args, "release", None)) + if args.json: + print(_json.dumps(res, indent=2)) + return 0 + all_ = res["all"] + if res["resolution"] == "none": + if getattr(args, "release", None): + print(f"No release '{args.release}' found. Start one with: init --release {args.release}") + else: + print("No active release on this machine. Start one with: init --release ") + return 0 + print(f"Found {len(all_)} release(s):") + for r in all_: + mark = "->" if r is res["release"] else " " + print(f" {mark} {r['release_id']} [{r['status']}] updated {r['updated_at']}") + if res["resolution"] == "ambiguous": + print(f"\nMultiple releases found — assuming most recent: {res['release']['release_id']} " + f"(confirm before acting).") + return 0 + + +def cmd_status(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + if not getattr(args, "no_pipeline_check", False) and C.refresh_conflict(st): + C.save_state(st, args.runs_root, args.release) + if getattr(args, "json", False): + print(_json.dumps(orch.status_report(), indent=2)) + return 0 + rep = orch.status_report() + C.emit(args.runs_root, args.release, render.status_view(rep), kind="status", + log_text=(f"status viewed — {rep['release_id']} {rep['done']}/{rep['total']} " + f"({rep['percent']}%), phase: {rep.get('current_phase_name','')}")) + return 0 + + +def cmd_next(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + actions = orch.run_until_gate() + C.save_state(st, args.runs_root, args.release) # persist BEFORE any display + C.log_actions(C.elog(args.runs_root, args.release), actions, state=st) + C.emit(args.runs_root, args.release, C.advance_block(actions, orch), kind="advance", + log_text=C.advance_log_summary(actions)) + return 0 + + +def cmd_approve(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + gate_phase, gate_step = st.current_phase, st.current_step + act = orch.approve_gate(args.comment or "") + el = C.elog(args.runs_root, args.release) + if act.kind != "idle": + el.log("gate_approved", phase=gate_phase, step=gate_step, driver=args.comment or None) + actions = orch.run_until_gate() + C.save_state(st, args.runs_root, args.release) # persist BEFORE any display + C.log_actions(el, actions, state=st) + C.emit(args.runs_root, args.release, + C.advance_block(actions, orch, lead=[f" {act.message}"]), kind="advance", + log_text=C.advance_log_summary(actions, lead=[act.message])) + return 0 + + +def cmd_deny(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + gate_phase, gate_step = st.current_phase, st.current_step + act = orch.deny_gate(args.comment or "") + if act.kind != "idle": + C.elog(args.runs_root, args.release).log( + "gate_denied", phase=gate_phase, step=gate_step, driver=args.comment or None) + C.save_state(st, args.runs_root, args.release) + C.emit(args.runs_root, args.release, + f" {act.message}\n\n" + render.status_view(orch.status_report()), kind="deny", + log_text=act.message) + return 0 + + +def cmd_done(args): + """Mark a reminder (human, non-gate) step done, then advance to the next hold.""" + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + act = orch.complete_step(getattr(args, "phase", None), getattr(args, "step", None), args.note or "") + if act.kind == "idle": + print(act.message) + return 1 + el = C.elog(args.runs_root, args.release) + el.log("reminder_done", phase=act.phase, step=act.step, driver=args.note or None) + actions = orch.run_until_gate() + C.save_state(st, args.runs_root, args.release) + C.log_actions(el, actions, state=st) + C.emit(args.runs_root, args.release, + C.advance_block(actions, orch, lead=[f" {act.message}"]), kind="advance", + log_text=C.advance_log_summary(actions, lead=[act.message])) + return 0 + + +def cmd_skip(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + act = orch.skip_step(args.phase, args.step, args.reason or "") + if act.kind == "idle": # rejected (no reason / bad step) — nothing changed + print(act.message) + return 1 + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log("step_skipped", phase=args.phase, step=args.step, driver=args.reason) + C.emit(args.runs_root, args.release, act.message, kind="override") + return 0 + + +def cmd_reopen(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + act = orch.reopen_step(args.phase, args.step, args.reason or "") + if act.kind == "idle": + print(act.message) + return 1 + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log("step_reopened", phase=args.phase, step=args.step, driver=args.reason or None) + C.emit(args.runs_root, args.release, act.message, kind="override") + return 0 + + +# The Phase-2 RC-testing steps a re-triggered RC invalidates: the two MRWP verifications +# and the terminal RC report/gate. checker_fired / orchestrator_health are NOT reopened — +# a re-triggered RC re-runs MRWP against the same orchestrator run. +_RC_RETRIGGER_STEPS = ("mrwp_ecs", "mrwp_local", "rc_report") + + +def cmd_rc_retriggered(args): + """The human explicitly signals that a NEW RC has been triggered (a flaky-run + re-trigger, or the orchestrator re-running RC testing after a broker cherry-pick). + + Reopens the Phase-2 RC-testing steps so the engine re-resolves the NEWEST MRWP run + (mrwp_run_ids already picks the highest id) and re-applies the gate. Scout's poller / + next then holds while the new run is in-flight and re-evaluates on completion — so an + early poll can't mark an in-progress RC as a false failure.""" + st, orch = C.load_orch(args.runs_root, args.release, args.config) + reason = (args.reason or "RC re-triggered").strip() + reopened = [] + for sid in _RC_RETRIGGER_STEPS: + act = orch.reopen_step("build_verify", sid, reason) + if act.kind != "idle": + reopened.append(sid) + # a reopened step is no longer an owner action / block + key = f"build_verify.{sid}" + st.pending_human = [p for p in st.pending_human if p != key] + if not reopened: + print("No Phase-2 RC steps found to reopen (is this release in Build & RC " + "Verification?).") + return 1 + if st.status in ("awaiting_action", "holding_gate", "complete", "halted"): + st.status = "running" + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log( + "rc_retriggered", driver=reason, steps=",".join(reopened)) + msg = (f"RC re-trigger acknowledged — reopened {', '.join(reopened)}. Scout will " + f"re-resolve the newest RC and re-apply the gate; it holds (no action needed) " + f"while the run is still in-flight and polls every 30 min. Reason: {reason}") + C.emit(args.runs_root, args.release, msg, kind="override") + print(msg) + return 0 + + +def cmd_halt(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + act = orch.halt(args.reason or "") + if act.kind == "idle": + print(act.message) + return 1 + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log("release_halted", driver=args.reason) + C.emit(args.runs_root, args.release, act.message, kind="override") + return 0 + + +def cmd_resume(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config, C.parse_as_of(args)) + act = orch.resume(args.reason or "") + if not getattr(args, "no_pipeline_check", False): + C.refresh_conflict(st) + C.save_state(st, args.runs_root, args.release) + C.elog(args.runs_root, args.release).log("release_resumed", driver=args.reason or None) + tail = "" + if st.ccd_conflict: + tail = (f"\n ⚠ Pipeline override {st.ccd_conflict} differs from CCD {st.ccd} — " + f"confirm which is correct (see status).") + C.emit(args.runs_root, args.release, (act.message + tail), kind="override") + return 0 + + +def cmd_activate(args): + st, orch = C.load_orch(args.runs_root, args.release, args.config) + orch.activate_conditional(args.phase) + C.save_state(st, args.runs_root, args.release) + print(f"Activated conditional phase: {args.phase}") + return 0 + + +def register(sub): + i = sub.add_parser("init", help="Start a new release run") + i.add_argument("--release", required=True) + i.add_argument("--force", action="store_true") + i.add_argument("--owner-email", default=None, help="Release owner email (default: signed-in az user)") + i.add_argument("--owner-name", default=None, help="Release owner display name (optional)") + i.add_argument("--timezone", default=None, + help="Owner IANA timezone (default: auto-detected from this machine, e.g. America/Los_Angeles)") + i.set_defaults(func=cmd_init) + + l = sub.add_parser("list", help="Discover releases (none/one/many)") + l.add_argument("--release", required=False, default=None) + l.add_argument("--json", action="store_true") + l.set_defaults(func=cmd_list) + + s = sub.add_parser("status", help="Show the run-state brief") + s.add_argument("--release", required=True) + s.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + s.add_argument("--json", action="store_true") + s.add_argument("--no-pipeline-check", action="store_true", + help="Skip re-reading the pipeline to detect CCD drift (faster/offline).") + s.set_defaults(func=cmd_status) + + n = sub.add_parser("next", help="Advance until the next gate / completion") + n.add_argument("--release", required=True) + n.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + n.set_defaults(func=cmd_next) + + a = sub.add_parser("approve", help="Approve the current holding gate, continue") + a.add_argument("--release", required=True) + a.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + a.add_argument("--comment", default="") + a.set_defaults(func=cmd_approve) + + d = sub.add_parser("deny", help="Deny the current holding gate") + d.add_argument("--release", required=True) + d.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + d.add_argument("--comment", default="") + d.set_defaults(func=cmd_deny) + + dn = sub.add_parser("done", help="Mark a reminder (human, non-gate) step done, then advance") + dn.add_argument("--release", required=True) + dn.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + dn.add_argument("--phase", default=None, help="Defaults to the current holding step") + dn.add_argument("--step", default=None) + dn.add_argument("--note", default="", help="Optional note (audited)") + dn.set_defaults(func=cmd_done) + + # ---- manual overrides ---- + sk = sub.add_parser("skip", help="Skip a step without running it (reason REQUIRED)") + sk.add_argument("--release", required=True) + sk.add_argument("--phase", required=True) + sk.add_argument("--step", required=True) + sk.add_argument("--reason", required=True, help="Why (audit — required)") + sk.set_defaults(func=cmd_skip) + + ro = sub.add_parser("reopen", help="Reopen a done/skipped step so it runs again") + ro.add_argument("--release", required=True) + ro.add_argument("--phase", required=True) + ro.add_argument("--step", required=True) + ro.add_argument("--reason", default="", help="Why (optional)") + ro.set_defaults(func=cmd_reopen) + + rt = sub.add_parser("rc-retriggered", + help="Signal a NEW RC was triggered — reopens the Phase-2 RC steps " + "so Scout re-evaluates the newest RC (holds while in-flight)") + rt.add_argument("--release", required=True) + rt.add_argument("--reason", default="", + help="Why it was re-triggered (e.g. 'flaky broker suite re-run' or " + "'broker cherry-pick #123') — recorded for audit") + rt.set_defaults(func=cmd_rc_retriggered) + + ht = sub.add_parser("halt", help="Emergency hold — nothing advances until resume (reason REQUIRED)") + ht.add_argument("--release", required=True) + ht.add_argument("--reason", required=True, help="Why (audit — required)") + ht.set_defaults(func=cmd_halt) + + rs = sub.add_parser("resume", help="Clear an emergency halt") + rs.add_argument("--release", required=True) + rs.add_argument("--as-of", default=None, help="Simulated clock (YYYY-MM-DD); default today") + rs.add_argument("--no-pipeline-check", action="store_true", help="Skip CCD-drift check") + rs.add_argument("--reason", default="", help="Why (optional)") + rs.set_defaults(func=cmd_resume) + + ac = sub.add_parser("activate", help="Turn on a conditional phase (e.g. hotfix)") + ac.add_argument("--release", required=True) + ac.add_argument("--phase", required=True) + ac.set_defaults(func=cmd_activate) diff --git a/release-agent/orchestrator/commands/sim.py b/release-agent/orchestrator/commands/sim.py new file mode 100644 index 00000000..d3ff3b06 --- /dev/null +++ b/release-agent/orchestrator/commands/sim.py @@ -0,0 +1,75 @@ +"""`sim` — seed the release to a mid-release point so you can test a phase. + + release-agent sim list # list available scenarios + release-agent sim run --scenario # seed the REAL release to the target + +A scenario fast-forwards the real engine (earlier phases + entry gate completed from +mocks) and writes the produced state as the real release — any existing state at that +id is backed up first. After it runs you just talk to the skill normally: `status`, +`rc-report`, `next`, `approve`. Pass `--runs-root` to target a throwaway sandbox instead. +""" +from __future__ import annotations +import json as _json + +from orchestrator import sim as S +from orchestrator import render + + +def cmd_sim_list(args): + names = S.list_scenarios() + if getattr(args, "json", False): + print(_json.dumps(names, indent=2)); return 0 + if not names: + print("No scenarios in config/scenarios/."); return 0 + print("Scenarios:") + for n in names: + print(f" - {n}") + return 0 + + +def cmd_sim_run(args): + res = S.run_scenario(args.scenario, runs_root=getattr(args, "runs_root", None), + config_path=getattr(args, "config", None), + freeze=getattr(args, "freeze", False)) + if getattr(args, "json", False): + d = {k: v for k, v in res.__dict__.items() if k != "state"} + print(_json.dumps(d, indent=2)); return 0 if res.reached else 1 + + tick = "✅" if res.reached else "⚠" + print(f"{tick} Seeded release {res.release_id} → {res.target['phase']} · {res.target['at']} " + f"(scenario '{res.scenario}')") + print(f" CCD {res.ccd} · as_of {res.as_of} · " + f"forwarded {res.steps_forwarded} step(s), approved {len(res.gates_approved)} gate(s)") + print(f" stop: [{res.stop_kind}] {res.stop_message}") + if res.problems: + print(" problems:") + for p in res.problems: + print(f" - {p}") + if res.backed_up_to: + print(f" ↩ previous state backed up → {res.backed_up_to}") + if res.frozen_to: + print(f" ❄ frozen → {res.frozen_to}") + print(" → now just talk to the release skill: status · rc-report · next · approve") + print() + from orchestrator.engine import Orchestrator + orch = Orchestrator(getattr(args, "config", None) or S.C.DEFAULT_CONFIG, res.state, + as_of=S.schedule.parse_date(res.as_of)) + print(render.status_view(orch.status_report())) + return 0 if res.reached else 1 + + +def register(sub): + sp = sub.add_parser("sim", help="Seed the release to a mid-release point (testing)") + ssub = sp.add_subparsers(dest="sim_cmd", required=True) + + lp = ssub.add_parser("list", help="List available scenarios") + lp.add_argument("--json", action="store_true") + lp.set_defaults(func=cmd_sim_list) + + rp = ssub.add_parser("run", help="Seed the real release to a scenario's target point") + rp.add_argument("--scenario", required=True, help="Scenario name (config/scenarios/.yaml)") + rp.add_argument("--runs-root", default=S.DEFAULT_SEED_RUNS, + help="Runs-root to seed (default: the REAL runs-root; pass a path for a sandbox)") + rp.add_argument("--freeze", action="store_true", help="Also snapshot the produced state to tests/fixtures/.json") + rp.add_argument("--json", action="store_true", help="Emit the raw SimResult") + rp.set_defaults(func=cmd_sim_run) diff --git a/release-agent/orchestrator/commands/step_action.py b/release-agent/orchestrator/commands/step_action.py new file mode 100644 index 00000000..115e474b --- /dev/null +++ b/release-agent/orchestrator/commands/step_action.py @@ -0,0 +1,306 @@ +"""Generic dispatcher for co-located step modules (`step-action`). + +This is the ONE command the skill calls to resolve any migrated step into its +uniform outcome. It replaces the old per-step `prepare-X` commands: instead of a +bespoke `prepare-notice`, `prepare-flight-reminder`, … the skill runs + + python -m orchestrator.cli step-action --release --phase preflight --step notice + +The command looks up the step module (`steps.get_step`), calls its `build(state)` +(passing through any `--param k=v` the module's signature accepts, e.g. variant), +and prints the outcome as JSON. The skill reads `kind` and reacts uniformly: + + done → already complete, nothing to run. + blocked → surface `reason` to the owner. + needs_human → show `prompt` (attestation or reminder). + needs_skill → run `tool` with `payload`, then `record-step --step `. + +Adding a scout step is now: write ONE module under steps// (auto-discovered) +— no CLI command, no registry, no skill-reference edits. +""" +from __future__ import annotations +import inspect +import json as _json + +from orchestrator import cli_common as C +from orchestrator import mocks as mocks_mod +from orchestrator import knowledge as kb +from orchestrator.outcomes import as_dict +from steps.lib.context import SELF_CHAT_ID +from steps.lib import mockctx +import steps + + +def _apply_overrides(out: dict, mockable: dict, spec: dict) -> None: + """Apply local-test payload overrides a step DECLARES via its MOCKABLE spec. + + Each MOCKABLE entry maps a mock-file key → a payload rewrite: + sets payload field to overwrite + as: "list" coerce a scalar to [scalar] + aliases value shortcuts (e.g. {"me": SELF_CHAT_ID}) + tag_subject prefix payload.subject with "[TEST → me]" + Keys in `spec` that aren't declared (and aren't the engine-level reserved + keys) are surfaced as `unknown_overrides` so typos are visible.""" + pl = out.get("payload") or {} + reserved = {"outcome", "note", "reason"} # handled by the engine, not here + applied = {} + for key, rule in (mockable or {}).items(): + if rule.get("kind") != "payload": # input → build(); post → check-localization + continue + if key not in spec: + continue + val = spec[key] + if "aliases" in rule and not isinstance(val, list) and val in rule["aliases"]: + val = rule["aliases"][val] + if rule.get("as") == "list" and not isinstance(val, list): + val = [val] + pl[rule["sets"]] = val + if rule.get("tag_subject") and pl.get("subject") and not pl["subject"].startswith("[TEST"): + pl["subject"] = f"[TEST → me] {pl['subject']}" + applied[key] = val + if applied: + out["test_redirect"] = applied + out["note"] = f"[test-redirect] {out.get('note', '')}".rstrip() + unknown = [k for k in spec if k not in (mockable or {}) and k not in reserved] + if unknown: + out["unknown_overrides"] = unknown + + +def _parse_params(pairs) -> dict: + """Turn ['variant=update', 'x=y'] into {'variant': 'update', 'x': 'y'}.""" + out = {} + for item in pairs or []: + if "=" not in item: + raise ValueError(f"--param must be KEY=VALUE, got: {item!r}") + k, v = item.split("=", 1) + out[k.strip()] = v + return out + + +def _accepted_kwargs(build, params: dict) -> dict: + """Filter params to only the keyword args `build` actually declares, so an + unrelated --param never crashes a step that doesn't take it.""" + try: + sig = inspect.signature(build) + except (TypeError, ValueError): + return dict(params) + if any(p.kind == inspect.Parameter.VAR_KEYWORD for p in sig.parameters.values()): + return dict(params) # build(**kwargs) takes anything + names = {name for name in sig.parameters if name != "state"} + return {k: v for k, v in params.items() if k in names} + + +def cmd_step_action(args): + mod = steps.get_step(args.phase, args.step) + if mod is None: + print(_json.dumps({ + "error": f"step '{args.phase}.{args.step}' is not migrated to the " + f"uniform contract; no co-located handler in steps/.", + "phase": args.phase, "step": args.step, + })) + return 1 + if not hasattr(mod, "build"): + print(_json.dumps({ + "error": f"step module '{args.phase}.{args.step}' has no build()", + })) + return 1 + + # Agent steps run IN-PROCESS inside the engine's `next` (they perform the real + # deterministic action). Executing their build() here would run that action a + # second time, out of band — refuse and point to `next`. + if getattr(mod, "KIND", None) == "agent": + print(_json.dumps({ + "error": f"step '{args.phase}.{args.step}' is an agent step — the engine " + f"runs it in-process via `next`; don't dispatch it with step-action.", + "phase": args.phase, "step": args.step, "kind": "agent", + })) + return 1 + + try: + params = _parse_params(getattr(args, "param", None)) + except ValueError as e: + print(_json.dumps({"error": str(e)})) + return 1 + + st = C.load_state(args.runs_root, args.release) + kwargs = _accepted_kwargs(mod.build, params) + spec = mocks_mod.load_mocks().get(f"{args.phase}.{getattr(mod, 'ID', args.step)}") or {} + with mockctx.active(spec): # expose `input` knobs to build() + outcome = mod.build(st, **kwargs) + + out = as_dict(outcome) + out["phase"] = args.phase + out["step"] = getattr(mod, "ID", args.step) + out["release"] = args.release + + # Local-test payload overrides: a mocks.local.yaml entry may set knobs the step + # DECLARES via its MOCKABLE spec (e.g. `send_to` on notice) — keeps the send + # real but redirects it. See `mock-spec` for what each step exposes. + if out.get("kind") == "needs_skill" and spec: + _apply_overrides(out, getattr(mod, "MOCKABLE", {}), spec) + + print(_json.dumps(out)) + return 0 + + +def _classify(step: dict) -> str: + if step.get("gate"): + return "gate" + if step.get("source") == "scout": + return "scout" + if step.get("attest"): + return "attest" + if step.get("owner") == "human": + return "reminder" + return "agent" + + +def _catalog(config_path: str) -> dict: + """Every step across every phase, with its mock-ability. Engine-level + (outcome: done|blocked) applies to any non-gate step; payload overrides only + exist where a migrated step declares MOCKABLE.""" + import yaml + with open(config_path, "r", encoding="utf-8") as fh: + cfg = yaml.safe_load(fh) + out = {} + for phase in cfg.get("phases", []): + pid = phase["id"] + for s in phase.get("steps", []): + kind = _classify(s) + key = f"{pid}.{s['id']}" + entry = {"phase": pid, "name": s.get("name", s["id"]), "kind": kind, + "outcome_mockable": kind != "gate", "overrides": {}} + mod = steps.get_step(pid, s["id"]) # migrated? + if mod is not None: + entry["overrides"] = getattr(mod, "MOCKABLE", {}) or {} + out[key] = entry + return out + + +def _readiness_auto_items(config_path: str): + """Readiness AUTO items (build_access, mcp_servers, oncall_now, …) — mockable + via `readiness.: {outcome: pass|fail}` to clear/fail the entry gate offline.""" + import os + import yaml + rp = os.path.join(os.path.dirname(config_path), "readiness.yaml") + if not os.path.exists(rp): + return [] + with open(rp, "r", encoding="utf-8") as fh: + cfg = yaml.safe_load(fh) or {} + out = [] + for it in cfg.get("items", []): + if it.get("verify") == "auto": + out.append({"id": it["id"], "source": it.get("source", "python")}) + return out + + +def cmd_mock_spec(args): + """List what each step (across ALL phases) exposes to mocks.local.yaml.""" + config_path = getattr(args, "config", None) or C.DEFAULT_CONFIG + catalog = _catalog(config_path) + readiness = _readiness_auto_items(config_path) + if getattr(args, "json", False): + print(_json.dumps({"steps": catalog, "readiness": readiness})) + return 0 + + print("mocks.local.yaml — what you can put under each \".\":\n") + print(" Engine-level (works for EVERY non-gate step, any phase):") + print(" outcome: done # mark complete, skip its real work") + print(" outcome: blocked # hold for the owner (+ reason: \"...\")\n") + print(" Per-step properties (declared by the step; input = feeds real logic):") + any_ov = False + for key, e in catalog.items(): + for name, rule in (e.get("overrides") or {}).items(): + any_ov = True + kind = rule.get("kind", "payload") + print(f" {key}: {{ {name}: }} [{kind}] — {rule.get('desc', '')}") + if not any_ov: + print(" (none declared yet)") + + if readiness: + print("\n Readiness entry-gate AUTO checks (real ADO/config/MCP — mock to clear offline):") + for it in readiness: + print(f" readiness.{it['id']}: {{ outcome: pass|fail }} [{it['source']}]") + + print("\n Every step — exactly what you can mock (🚦 gate = NOT mockable):") + cur = None + for key, e in catalog.items(): + if e["phase"] != cur: + cur = e["phase"] + print(f" [{cur}]") + if e["kind"] == "gate": + knobs = "🚦 not mockable (gate needs a real human decision)" + else: + parts = ["outcome: done|blocked"] + list((e.get("overrides") or {}).keys()) + knobs = " · ".join(parts) + print(f" {key:30} [{e['kind']:6}] → {knobs}") + return 0 + + +def cmd_step_info(args): + """Answer a user's question about a step from the knowledge base — what it does, + where to look, how to resolve it, links, FAQs. Consult this before answering + step questions so the info is accurate (not guessed).""" + k = kb.get_knowledge(args.phase, args.step) + if getattr(args, "json", False): + print(_json.dumps({"phase": args.phase, "step": args.step, "knowledge": k})) + return 0 + if not k: + print(f"No knowledge entry yet for {args.phase}.{args.step}. " + f"Add one to config/knowledge.yaml.") + return 0 + print(kb.render_knowledge(args.phase, args.step, k)) + return 0 + + +def cmd_gate_info(args): + """Answer a user's question about an ENTRY-GATE readiness item from the knowledge + base — what it verifies, who resolves it, where to look, how to satisfy/clear it, + links, FAQs. Gate items live under the `readiness.` key. Consult this before + answering gate questions so the info is accurate (not guessed).""" + k = kb.get_knowledge("readiness", args.item) + if getattr(args, "json", False): + print(_json.dumps({"item": args.item, "knowledge": k})) + return 0 + if not k: + print(f"No knowledge entry yet for readiness.{args.item}. " + f"Add one to config/knowledge.yaml under 'readiness.{args.item}'.") + return 0 + print(kb.render_knowledge("readiness", args.item, k)) + return 0 + + +def register(sub): + sp = sub.add_parser( + "step-action", + help="Resolve a migrated step into its uniform outcome JSON " + "(done|blocked|needs_human|needs_skill)") + sp.add_argument("--release", required=True) + sp.add_argument("--phase", default="preflight") + sp.add_argument("--step", required=True) + sp.add_argument("--param", action="append", default=[], + help="Optional KEY=VALUE passed to the step's build() " + "(e.g. --param variant=update). Repeatable.") + sp.set_defaults(func=cmd_step_action) + + ms = sub.add_parser( + "mock-spec", + help="List what every step (all phases) exposes to mocks.local.yaml") + ms.add_argument("--json", action="store_true", help="Emit the catalog as JSON") + ms.set_defaults(func=cmd_mock_spec) + + si = sub.add_parser( + "step-info", + help="Show a step's knowledge (what it does, where to look, how to resolve, links, FAQs)") + si.add_argument("--phase", default="preflight") + si.add_argument("--step", required=True) + si.add_argument("--json", action="store_true", help="Emit the knowledge as JSON") + si.set_defaults(func=cmd_step_info) + + gi = sub.add_parser( + "gate-info", + help="Show an entry-gate readiness item's knowledge (what it verifies, who resolves it, how to clear it, links, FAQs)") + gi.add_argument("--item", required=True, help="Readiness item id, e.g. build_access, oncall_now, yubikey") + gi.add_argument("--json", action="store_true", help="Emit the knowledge as JSON") + gi.set_defaults(func=cmd_gate_info) + diff --git a/release-agent/orchestrator/discovery.py b/release-agent/orchestrator/discovery.py new file mode 100644 index 00000000..6e8cd056 --- /dev/null +++ b/release-agent/orchestrator/discovery.py @@ -0,0 +1,71 @@ +"""Release discovery — the none / one / many logic. + +Scans the runs root for release folders and reports what's there so the +/release-agent skill can: + * 0 releases -> tell the user none is active, offer to start one + * 1 release -> use it + * many -> present the assumed one (most recently updated) + ask to confirm + +Deterministic; the skill only presents what this returns. +""" +from __future__ import annotations +import os +import json +from typing import Optional + + +def _summarize(state_file: str) -> Optional[dict]: + try: + with open(state_file, "r", encoding="utf-8") as fh: + data = json.load(fh) + except (OSError, ValueError): # missing/unreadable file or bad JSON → skip it + return None + return { + "release_id": data.get("release_id"), + "status": data.get("status"), + "current_phase": data.get("current_phase"), + "current_step": data.get("current_step"), + "updated_at": data.get("updated_at"), + "state_file": state_file, + } + + +def list_releases(runs_root: str) -> list: + """Return summaries of all releases found, newest-updated first.""" + out = [] + if not os.path.isdir(runs_root): + return out + for name in os.listdir(runs_root): + sf = os.path.join(runs_root, name, "release-state.json") + if os.path.isfile(sf): + s = _summarize(sf) + if s: + out.append(s) + out.sort(key=lambda s: s.get("updated_at") or "", reverse=True) + return out + + +def resolve(runs_root: str, requested: Optional[str] = None) -> dict: + """Decide which release to act on. + + Returns a dict: + { "resolution": "none" | "one" | "explicit" | "ambiguous", + "release": , # the chosen/assumed release + "all": [] } # everything found + + - none : no releases exist -> caller should offer to start one + - one : exactly one exists -> use it + - explicit : caller named one and it exists -> use it + - ambiguous : several exist and none named -> 'release' is the assumed + (most recently updated); caller should confirm. + """ + all_ = list_releases(runs_root) + if requested: + match = next((r for r in all_ if r["release_id"] == requested), None) + return {"resolution": "explicit" if match else "none", + "release": match, "all": all_} + if not all_: + return {"resolution": "none", "release": None, "all": all_} + if len(all_) == 1: + return {"resolution": "one", "release": all_[0], "all": all_} + return {"resolution": "ambiguous", "release": all_[0], "all": all_} diff --git a/release-agent/orchestrator/engine.py b/release-agent/orchestrator/engine.py new file mode 100644 index 00000000..fa86c654 --- /dev/null +++ b/release-agent/orchestrator/engine.py @@ -0,0 +1,620 @@ +"""Release Orchestrator — the conductor (deterministic engine, X4). + +Responsibilities (per §7.1): + 1. Load the release state machine from config/phases.yaml. + 2. Own the dispatch loop: find next step -> run its (stub) agent -> + record result -> advance, or HOLD at a gate for human approval. + 3. Persist run-state via ReleaseState (X5). + +The engine is the BRAIN: it decides what's next. The skill is only the mouth/ears. +No LLM logic here — this is fully unit-testable and replayable. +""" +from __future__ import annotations +import os +from dataclasses import dataclass +from datetime import date, datetime, time +from typing import Optional + +import yaml + +from .state import ReleaseState, StepState, GateDecision, _now +from .readiness import ReadinessGate +from . import schedule +from . import mocks as mocks_mod +from .status_views import StatusViewMixin +from steps.lib import mockctx +import steps +from phases import stub_runner + + +@dataclass +class NextAction: + """What the conductor decided on this invocation — the engine's output.""" + kind: str # 'ran' | 'gate' | 'reminder' | 'scheduled' | 'complete' | 'idle' | 'readiness' | 'blocked' | 'halted' + phase: Optional[str] = None + step: Optional[str] = None + name: Optional[str] = None + message: str = "" + + +class Orchestrator(StatusViewMixin): + """The conductor: owns the state machine, dispatch loop, gates, and structured + status. The readiness entry gate is delegated to ReadinessGate (self.gate); + presentation lives in render.py. This class holds no formatting logic.""" + + def __init__(self, config_path: str, state: ReleaseState, readiness_path: str = None, + as_of: date = None, mocks: dict = None, tz=None, now: datetime = None): + with open(config_path, "r", encoding="utf-8") as fh: + self.config = yaml.safe_load(fh) + readiness_cfg = None + if readiness_path is None: + readiness_path = os.path.join(os.path.dirname(config_path), "readiness.yaml") + if os.path.exists(readiness_path): + with open(readiness_path, "r", encoding="utf-8") as fh: + readiness_cfg = yaml.safe_load(fh) + self.state = state + self.gate = ReadinessGate(readiness_cfg, state) + # Local step mocks (personal, gitignored mocks.local.yaml). Absent → {}. + # Pass mocks={} in tests for isolation from any developer's local file. + self.mocks = mocks if mocks is not None else mocks_mod.load_mocks() + self.gate.mocks = self.mocks # readiness. mocks for the entry gate + # The simulated clock, in the OWNER's timezone (not the host's — a UTC host must + # not roll the date early). Precedence: an explicit tz arg → the tz captured on + # the release at init (state.timezone) → config/schedule.yaml → DEFAULT_TZ. + # `self.as_of` is the date used for phase due-ness; `self.now_local` is the + # wall-clock time used to gate a step's fire_at_local. + tz_name = getattr(state, "timezone", None) or self._config_timezone(config_path) + self.tz = tz if tz is not None else schedule.get_tz(tz_name) + if now is not None: # explicit datetime (precise tests / callers) + self.now_local = now + self.as_of = (as_of.date() if isinstance(as_of, datetime) + else as_of) or now.date() + elif isinstance(as_of, datetime): # a datetime passed as as_of + self.now_local = as_of + self.as_of = as_of.date() + elif as_of is not None: # a bare DATE (debug clock / most tests): + # keep it as the due-date, and treat the wall clock as end-of-day so a + # date-only test still sees fire_at_local steps as past their fire time. + self.as_of = as_of + self.now_local = datetime.combine(as_of, time(23, 59, 59), self.tz) + else: # live: real now in the owner's zone + self.now_local = schedule.now_local(self.tz) + self.as_of = self.now_local.date() + + @staticmethod + def _config_timezone(config_path: str) -> Optional[str]: + """Read the release timezone from config/schedule.yaml (`timezone:`), or None + (⇒ schedule.DEFAULT_TZ). Best-effort; never fails the engine.""" + try: + p = os.path.join(os.path.dirname(config_path), "schedule.yaml") + if os.path.exists(p): + with open(p, "r", encoding="utf-8") as fh: + return (yaml.safe_load(fh) or {}).get("timezone") + except (OSError, yaml.YAMLError): # missing/unreadable/invalid schedule.yaml + pass + return None + + # ---- time anchoring (CCD-relative phase windows) ---- + def _ccd(self) -> Optional[date]: + return schedule.parse_date(self.state.ccd) + + def _phase_anchor_date(self, phase: dict) -> Optional[date]: + """The date a phase opens, or None if it has no anchor / CCD is unknown.""" + spec = phase.get("anchor") + ccd = self._ccd() + if not spec or ccd is None: + return None + return schedule.anchor_date(ccd, spec) + + def _phase_due(self, phase: dict) -> bool: + """A phase is due once the clock reaches its anchor. No anchor ⇒ always due.""" + ad = self._phase_anchor_date(phase) + return ad is None or self.as_of >= ad + + def _step_time_ready(self, phase: dict, step: dict) -> bool: + """A step that declares a `fire_at_local` (e.g. the 09:00 CCD comms) is NOT + runnable by the engine's automatic paths until that wall-clock time arrives, in + the owner's timezone, on its fire day. This stops the every-hour worker from + draining a timed step the instant its phase goes due — the step is left for its + dedicated cron-pinned automation (which calls step-action directly and so isn't + gated). Non-timed steps are always ready.""" + from orchestrator import automations + fire = automations.fire_at(phase["id"], step["id"]) + if not fire: + return True + try: + hh, mm = (int(x) for x in str(fire).split(":")[:2]) + except (ValueError, TypeError): + return True # malformed fire_at_local ⇒ don't gate + anchor = self._phase_anchor_date(phase) + if anchor is not None: + if self.as_of > anchor: # past the fire day ⇒ run ASAP (catch-up) + return True + if self.as_of < anchor: # before it (phase not due) ⇒ not ready + return False + return self.now_local.time() >= time(hh, mm) + + @staticmethod + def _is_reminder(step: dict) -> bool: + """A human, non-gate step is a reminder: the engine can't do it, so it + holds and tells the person to do it, then waits for them to mark it done.""" + return step.get("owner") == "human" and not step.get("gate") + + # ---- state-machine traversal ---- + def _iter_steps(self): + """Yield (phase_dict, step_dict) in definition order, skipping conditional + phases unless explicitly activated on the state.""" + for phase in self.config["phases"]: + if phase.get("conditional") and phase["id"] not in self._activated_conditionals(): + continue + for step in phase["steps"]: + yield phase, step + + def _activated_conditionals(self) -> set: + # A conditional phase (e.g. hotfix) is activated by an explicit note flag. + return {n.split("activate:")[1].strip() + for n in self.state.notes if isinstance(n, str) and n.startswith("activate:")} + + def activate_conditional(self, phase_id: str) -> None: + self.state.notes.append(f"activate:{phase_id}") + + def _first_incomplete(self): + for phase, step in self._iter_steps(): + if not self.state.is_done(phase["id"], step["id"]): + return phase, step + return None, None + + # ---- dispatch ---- + def _current_phase(self): + """The first included phase that still has incomplete steps (definition + order). Conditional phases are skipped unless activated.""" + for phase in self.config["phases"]: + if not self._phase_included(phase): + continue + if all(self.state.is_done(phase["id"], s["id"]) for s in phase["steps"]): + continue + return phase + return None + + def current_phase_id(self) -> Optional[str]: + """Public: id of the first included phase with incomplete steps, or None when + the release is complete. The engine's authoritative 'where are we' — derived + from config order + the done-map, not the cached cursor. Used by the simulator + to know when a fast-forward has reached a target phase.""" + p = self._current_phase() + return p["id"] if p else None + + @staticmethod + def _step_kind(step: dict) -> str: + """Classify a step: gate | scout | attest | reminder | auto.""" + if step.get("gate"): + return "gate" + if step.get("source") == "scout": + return "scout" + if step.get("attest"): + return "attest" + if step.get("owner") == "human": + return "reminder" + return "auto" + + def _is_mocked(self, pid: str, step: dict) -> bool: + """True if a local mock replaces this step. Gate steps are never mockable + (a gate needs a real human decision).""" + return ((not step.get("gate")) + and mocks_mod.stepresult_for(self.mocks, pid, step["id"]) is not None) + + + def _deps_met(self, pid: str, step: dict) -> bool: + """True when every step this one depends_on is done (deps are within-phase).""" + for dep in step.get("depends_on", []) or []: + if not self.state.is_done(pid, dep): + return False + return True + + def step_once(self, attempted=None) -> NextAction: + """Advance exactly one step (or hold). For a sequential phase this is the + classic first-incomplete-step logic. For a parallel phase it runs one ready + step whose dependencies are met, letting independent steps progress even + when a sibling is holding. `attempted` (a set, managed by run_until_gate) + prevents re-running an auto step twice within one drain.""" + if self.state.status == "complete": + return NextAction(kind="complete", message="Release already complete.") + + # HALTED: emergency hold set by a human. Nothing advances until resume(). + if self.state.halted: + self.state.status = "halted" + return NextAction( + kind="halted", + message="Release is HALTED" + + (f": {self.state.halt_reason}" if self.state.halt_reason else "") + + ". Run resume to continue.", + ) + + # BLOCKED: an entry-gate item was declared unsatisfiable. + if self.state.blocked: + self.state.status = "blocked" + labels = self.gate.blocked_labels() + msg = (self.gate.config or {}).get("blocked_message", "").strip() + return NextAction( + kind="blocked", + message="Entry gate blocked — cannot start: " + ", ".join(labels) + ". " + msg, + ) + + # ENTRY GATE: nothing runs until the readiness checklist is signed. + if not self.state.readiness_signed: + self.state.status = "readiness_gate" + return NextAction( + kind="readiness", + message="HOLDING at the readiness entry gate. Sign the checklist before Phase 0 can start.", + ) + + phase = self._current_phase() + if phase is None: + self.state.status = "complete" + self.state.current_phase = None + self.state.current_step = None + return NextAction(kind="complete", message="All steps done — release complete.") + + self.state.current_phase = phase["id"] + + # TIME GATE: if this phase hasn't reached its anchor date yet, hold as scheduled. + if not self._phase_due(phase): + opens = self._phase_anchor_date(phase) + self.state.status = "scheduled" + days = (opens - self.as_of).days + first = next((s for s in phase["steps"] + if not self.state.is_done(phase["id"], s["id"])), None) + return NextAction( + kind="scheduled", phase=phase["id"], + step=first["id"] if first else None, + name=first["name"] if first else None, + message=f"{phase['name']} opens {opens.isoformat()} " + f"({schedule.humanize_delta(days)}). Nothing to do yet.", + ) + + if phase.get("execution") == "parallel": + return self._step_parallel(phase, attempted) + return self._step_sequential(phase) + + # ---- sequential dispatch (classic: one step at a time, stop at first hold) ---- + def _step_sequential(self, phase: dict) -> NextAction: + step = next(s for s in phase["steps"] + if not self.state.is_done(phase["id"], s["id"])) + self.state.current_step = step["id"] + + # A locally-mocked step is resolved right here (skips its real scout/attest/ + # agent handling) so the flow advances naturally under Scout. + if self._is_mocked(phase["id"], step): + return self._run_auto_step(phase, step, block_holds=True) + + # TIME GATE (within the day): a step with a fire_at_local isn't runnable until + # its wall-clock time — hold as scheduled so the every-hour worker doesn't fire + # it early; its dedicated cron automation runs it at the pinned time. + if not self._step_time_ready(phase, step): + from orchestrator import automations + fire = automations.fire_at(phase["id"], step["id"]) + self.state.status = "scheduled" + return NextAction( + kind="scheduled", phase=phase["id"], step=step["id"], name=step["name"], + message=f"{phase['name']} → {step['name']} is scheduled for {fire} " + f"(fires via its timed automation). Nothing to do yet.") + + if step.get("gate") and not self._gate_approved(phase["id"], step["id"]): + self.state.status = "holding_gate" + return NextAction( + kind="gate", phase=phase["id"], step=step["id"], name=step["name"], + message=f"HOLDING at gate: {phase['name']} → {step['name']}. Awaiting human decision.", + ) + + if self._is_reminder(step): + self.state.status = "awaiting_action" + key = f"{phase['id']}.{step['id']}" + if key not in self.state.pending_human: + self.state.pending_human.append(key) + if step.get("attest"): + msg = (f"CONFIRM — attest that this is done to proceed: {step['name']}. " + f"Mark it done once you've verified it.") + else: + msg = f"ACTION NEEDED — you need to: {step['name']}. Mark it done when complete." + return NextAction(kind="reminder", phase=phase["id"], step=step["id"], + name=step["name"], message=msg) + + if step.get("source") == "scout": + self.state.status = "awaiting_action" + key = f"{phase['id']}.{step['id']}" + if key not in self.state.pending_human: + self.state.pending_human.append(key) + return NextAction( + kind="reminder", phase=phase["id"], step=step["id"], name=step["name"], + message=f"Scout-assisted check pending — {step['name']}. " + f"Scout runs this automatically when you open it.", + ) + + return self._run_auto_step(phase, step, block_holds=True) + + # ---- parallel dispatch (dependency-aware; independent steps don't block) ---- + def _step_parallel(self, phase: dict, attempted) -> NextAction: + pid = phase["id"] + + def ready(s): + return ((not self.state.is_done(pid, s["id"])) and self._deps_met(pid, s) + and self._step_time_ready(phase, s)) + + # 1) Run ONE ready, not-yet-attempted runnable step (auto agent, or an + # already-approved gate). Independent steps progress even if a sibling holds. + for s in phase["steps"]: + if not ready(s): + continue + kind = self._step_kind(s) + runnable = (kind == "auto" + or (kind == "gate" and self._gate_approved(pid, s["id"])) + or self._is_mocked(pid, s)) # mocked steps run here too + if not runnable: + continue + key = f"{pid}.{s['id']}" + if attempted is not None and key in attempted: + continue + if attempted is not None: + attempted.add(key) + return self._run_auto_step(phase, s, block_holds=False) + + # 2) No more auto progress — surface the holds (all at once). + holds = [] + for s in phase["steps"]: + if not ready(s): + continue + kind = self._step_kind(s) + blocked_auto = kind == "auto" and self.state.get_step(pid, s["id"]).status == "blocked" + unapproved_gate = kind == "gate" and not self._gate_approved(pid, s["id"]) + if kind in ("scout", "attest", "reminder") or blocked_auto or unapproved_gate: + holds.append(s) + + gates = [s for s in holds if self._step_kind(s) == "gate"] + if gates: + g = gates[0] + self.state.status = "holding_gate" + self.state.current_step = g["id"] + return NextAction(kind="gate", phase=pid, step=g["id"], name=g["name"], + message=f"HOLDING at gate: {phase['name']} → {g['name']}. Awaiting human decision.") + + non_gate = [s for s in holds if self._step_kind(s) != "gate"] + for s in non_gate: + key = f"{pid}.{s['id']}" + if key not in self.state.pending_human: + self.state.pending_human.append(key) + if non_gate: + self.state.status = "awaiting_action" + # Scout steps are the SKILL's automated work (it runs them via step-action), + # NOT a user hold — so the current-step / action cue should point at a + # genuine USER hold (attest / blocked / reminder) when one exists, and only + # fall back to a scout step when scout work is all that's left. + user_holds = [s for s in non_gate if self._step_kind(s) != "scout"] + focus = (user_holds or non_gate)[0] + self.state.current_step = focus["id"] + names = "; ".join(s["name"] for s in non_gate) + return NextAction(kind="reminder", phase=pid, step=focus["id"], + name=focus["name"], + message=f"{len(non_gate)} item(s) need attention: {names}") + + # Not complete, but nothing is ready — remaining steps wait on unmet deps. + self.state.status = "awaiting_action" + return NextAction(kind="reminder", phase=pid, + message="Waiting on prerequisite steps to complete.") + + def _run_auto_step(self, phase: dict, step: dict, block_holds: bool) -> NextAction: + """Run an agent step. On success → done. On failure: in sequential mode + (block_holds=True) HOLD as action-needed and break; in parallel mode + (block_holds=False) mark it blocked + register it, but return 'ran' so the + drain continues with independent steps.""" + pid = phase["id"] + # A local mock short-circuits the real runner (agent call), returning the + # declared StepResult (done → complete; blocked → hold). + result = mocks_mod.stepresult_for(self.mocks, pid, step["id"]) + if result is None: + # Resolve the runner from the co-located step module (KIND == 'agent'); + # steps without a module fall back to the stub. No agent registry. + mod = steps.get_step(pid, step["id"]) + if mod is not None and getattr(mod, "KIND", None) == "agent" and hasattr(mod, "run"): + runner = mod.run + else: + runner = stub_runner.run_stub + # Expose any declared `input` knobs (e.g. cg `alerts`) to the step's + # build() so its REAL logic runs on the injected data. + with mockctx.active(self.mocks.get(f"{pid}.{step['id']}", {})): + result = runner(pid, step, self.state) + key = f"{pid}.{step['id']}" + # IN-FLIGHT: the step's underlying pipeline run is still executing — NOT a failure. + # Hold the phase as 'waiting on the pipeline' (no user action) and let the poller / + # tick re-run the step until the run completes. Stamp when we first saw it in-flight + # so a poller can send the 6h courtesy nudge. + if getattr(result, "in_flight", False): + prev = self.state.get_step(pid, step["id"]) + data = dict(getattr(prev, "data", {}) or {}) + data.setdefault("in_flight_since", _now()) + data["poll_in_min"] = getattr(result, "poll_in_min", 30) + self.state.set_step(pid, step["id"], + StepState(status="in_flight", note=result.action, by="agent", + links=list(getattr(result, "links", None) or []), + data=data)) + self.state.pending_human = [p for p in self.state.pending_human if p != key] + self.state.status = "running" + return NextAction(kind="waiting", phase=pid, step=step["id"], name=step["name"], + message=f"WAITING — {step['name']}: {result.action}") + if not result.ok: + self.state.set_step(pid, step["id"], + StepState(status="blocked", note=result.action, by=result.by, + links=list(getattr(result, "links", None) or []))) + if key not in self.state.pending_human: + self.state.pending_human.append(key) + if block_holds: + self.state.status = "awaiting_action" + return NextAction(kind="reminder", phase=pid, step=step["id"], + name=step["name"], + message=f"ACTION NEEDED — {step['name']}: {result.action}") + return NextAction(kind="ran", phase=pid, step=step["id"], name=step["name"], + message=f"BLOCKED — {step['name']}: {result.action}") + self.state.set_step(pid, step["id"], + StepState(status="done", completed_at=_now(), + note=result.action, by=result.by, + links=list(getattr(result, "links", None) or []))) + if result.by == "human": + self.state.pending_human = [p for p in self.state.pending_human if p != key] + self.state.status = "running" + return NextAction(kind="ran", phase=pid, step=step["id"], + name=step["name"], message=result.action) + + def run_until_gate(self, max_steps: int = 500) -> list: + """Drive the loop until a gate hold, completion, or step cap. + Returns the list of NextAction taken. `attempted` prevents re-running an + auto step twice within this drain (so a re-blocking step can't loop).""" + actions = [] + attempted = set() + for _ in range(max_steps): + act = self.step_once(attempted) + actions.append(act) + if act.kind in ("gate", "reminder", "scheduled", "complete", "readiness", "blocked", "halted", "waiting"): + break + return actions + + # ---- manual overrides (human-driven transitions, §7.1 constraint #5) ---- + def _find_step(self, phase_id: str, step_id: str): + phase = next((p for p in self.config["phases"] if p["id"] == phase_id), None) + if phase and any(s["id"] == step_id for s in phase["steps"]): + return phase + return None + + def skip_step(self, phase_id: str, step_id: str, reason: str) -> NextAction: + """Mark a step skipped (counts as done for progression) without running it. + A reason is REQUIRED (audit). For 'doesn't apply' or 'done manually outside the tool'.""" + if not (reason and reason.strip()): + return NextAction(kind="idle", message="A reason is required to skip a step.") + if not self._find_step(phase_id, step_id): + return NextAction(kind="idle", message=f"No such step: {phase_id}/{step_id}") + self.state.set_step(phase_id, step_id, + StepState(status="skipped", completed_at=_now(), + note=f"Skipped: {reason.strip()}", by="human")) + if self.state.status == "holding_gate" and self.state.current_step == step_id: + self.state.status = "running" + return NextAction(kind="ran", phase=phase_id, step=step_id, + message=f"Skipped {phase_id}/{step_id} — {reason.strip()}") + + def complete_step(self, phase_id: str = None, step_id: str = None, note: str = "") -> NextAction: + """Mark a reminder (human, non-gate) step done. Defaults to the step the + conductor is currently holding on. This is how a person clears an + 'ACTION NEEDED' hold once they've actually done the task.""" + phase_id = phase_id or self.state.current_phase + step_id = step_id or self.state.current_step + if not (phase_id and step_id) or not self._find_step(phase_id, step_id): + return NextAction(kind="idle", message=f"No such step: {phase_id}/{step_id}") + self.state.set_step(phase_id, step_id, + StepState(status="done", completed_at=_now(), + note=(note.strip() or "Marked done"), by="human")) + key = f"{phase_id}.{step_id}" + self.state.pending_human = [p for p in self.state.pending_human + if p != key and not p.startswith(key + " ")] + if self.state.status == "awaiting_action": + self.state.status = "running" + tail = f" — {note.strip()}" if note and note.strip() else "" + return NextAction(kind="ran", phase=phase_id, step=step_id, + message=f"Done: {phase_id}/{step_id}{tail}") + + def record_scout_step(self, phase_id: str, step_id: str, status: str, + detail: str = "") -> NextAction: + """Record the outcome of a scout-assisted step (one the skill ran via MCP/ + browser, e.g. the CCOA lockdown check). + * status == "pass" -> mark the step done and let the flow continue. + * status == "attention" -> keep it held (needs the owner) with the detail + (e.g. a Production CCOA lockdown overlaps — the owner must shift CCD).""" + if not self._find_step(phase_id, step_id): + return NextAction(kind="idle", message=f"No such step: {phase_id}/{step_id}") + if status == "pass": + return self.complete_step(phase_id, step_id, detail) + # attention: leave the step outstanding, flagged for the owner. + self.state.set_step(phase_id, step_id, + StepState(status="blocked", note=detail, by="scout")) + self.state.status = "awaiting_action" + key = f"{phase_id}.{step_id}" + if key not in self.state.pending_human: + self.state.pending_human.append(key) + return NextAction(kind="reminder", phase=phase_id, step=step_id, + message=f"Needs your attention — {detail}") + + def reopen_step(self, phase_id: str, step_id: str, reason: str = "") -> NextAction: + """Undo a done/skipped step so the conductor runs it again. Reason optional.""" + if not self._find_step(phase_id, step_id): + return NextAction(kind="idle", message=f"No such step: {phase_id}/{step_id}") + self.state.steps.pop(self.state.key(phase_id, step_id), None) # remove -> pending + # drop any prior gate approval for this step so a gate re-holds + self.state.gate_decisions = [g for g in self.state.gate_decisions + if g.get("step") != f"{phase_id}.{step_id}"] + if self.state.status == "complete": + self.state.status = "running" + note = f" — {reason.strip()}" if reason and reason.strip() else "" + return NextAction(kind="ran", phase=phase_id, step=step_id, + message=f"Reopened {phase_id}/{step_id}{note}") + + def halt(self, reason: str) -> NextAction: + """Emergency hold — nothing advances until resume(). Reason REQUIRED (audit).""" + if not (reason and reason.strip()): + return NextAction(kind="idle", message="A reason is required to halt the release.") + self.state.halted = True + self.state.halt_reason = reason.strip() + self.state.status = "halted" + return NextAction(kind="halted", message=f"Release HALTED — {reason.strip()}") + + def resume(self, reason: str = "") -> NextAction: + """Clear an emergency halt. Reason optional.""" + if not self.state.halted: + return NextAction(kind="idle", message="Release is not halted.") + self.state.halted = False + self.state.halt_reason = None + self.state.status = "running" + note = f" — {reason.strip()}" if reason and reason.strip() else "" + return NextAction(kind="idle", message=f"Release resumed{note}.") + + # ---- gates ---- + def _gate_approved(self, phase: str, step: str) -> bool: + for gd in self.state.gate_decisions: + if gd.get("step") == f"{phase}.{step}" and gd.get("decision") == "approved": + return True + return False + + def approve_gate(self, comment: str = "") -> NextAction: + """Record approval for the current holding gate and continue.""" + phase = self.state.current_phase + step = self.state.current_step + if self.state.status != "holding_gate" or not phase: + return NextAction(kind="idle", message="No gate is currently holding.") + self.state.gate_decisions.append( + asdict_gate(GateDecision(step=f"{phase}.{step}", decision="approved", + at=_now(), comment=comment))) + # mark the gate step done and advance + self.state.set_step(phase, step, + StepState(status="done", completed_at=_now(), + note=f"Gate approved. {comment}".strip(), by="human")) + self.state.status = "running" + return NextAction(kind="ran", phase=phase, step=step, + message=f"Gate approved: {phase} → {step}. {comment}".strip()) + + def deny_gate(self, comment: str = "") -> NextAction: + phase = self.state.current_phase + step = self.state.current_step + if self.state.status != "holding_gate" or not phase: + return NextAction(kind="idle", message="No gate is currently holding.") + self.state.gate_decisions.append( + asdict_gate(GateDecision(step=f"{phase}.{step}", decision="denied", + at=_now(), comment=comment))) + self.state.pending_human.append(f"{phase}.{step} (denied: {comment})") + self.state.status = "blocked" + return NextAction(kind="gate", phase=phase, step=step, + message=f"Gate DENIED: {phase} → {step}. Release blocked. {comment}".strip()) + + # `_phase_included` is a shared helper (used by both the state machine and the status + # views mixin). The status view-model builders live in orchestrator/status_views.py. + def _phase_included(self, phase: dict) -> bool: + return (not phase.get("conditional")) or phase["id"] in self._activated_conditionals() + + +def asdict_gate(gd: GateDecision) -> dict: + return {"step": gd.step, "decision": gd.decision, "at": gd.at, + "by": gd.by, "comment": gd.comment} diff --git a/release-agent/orchestrator/eventlog.py b/release-agent/orchestrator/eventlog.py new file mode 100644 index 00000000..60f4271d --- /dev/null +++ b/release-agent/orchestrator/eventlog.py @@ -0,0 +1,117 @@ +"""Event log — the per-release record we analyze to debug and improve. + +Scope: PER-RELEASE ONLY. Each release keeps its own complete log at + //events.jsonl +There is no machine-wide aggregate — a release is a self-contained unit and its +log travels with it. + +What it must capture to be useful for debugging: not just the engine commands, +but the actual INTERACTION — + * what Scout presented to the engineer (prompts, the rendered checklist, options) + * what the engineer chose / typed (their input) + * the engine's own events (steps, gate holds, decisions with drivers) + +Each line is one JSON object: + { ts, release_id, actor, source, event, ... } + source: "engine" (deterministic engine action) | "scout" (agent output) | "user" (engineer input) + +Logging is best-effort and MUST never break or alter the interaction. +""" +from __future__ import annotations +import json +import os +import getpass +from datetime import datetime, timezone + + +def _now() -> str: + return datetime.now(timezone.utc).isoformat() + + +def _actor() -> str: + try: + return getpass.getuser() + except Exception: + return os.getenv("USERNAME") or os.getenv("USER") or "unknown" + + +class EventLog: + def __init__(self, runs_root: str, release_id: str): + self.runs_root = runs_root + self.release_id = release_id + self.path = os.path.join(runs_root, release_id, "events.jsonl") + self.actor = _actor() + + def log(self, event: str, source: str = "engine", **fields) -> dict: + rec = {"ts": _now(), "release_id": self.release_id, "actor": self.actor, + "source": source, "event": event} + rec.update({k: v for k, v in fields.items() if v is not None}) + try: + os.makedirs(os.path.dirname(self.path), exist_ok=True) + with open(self.path, "a", encoding="utf-8") as fh: + fh.write(json.dumps(rec, ensure_ascii=False) + "\n") + except Exception: + pass # logging must never break the flow + return rec + + # convenience wrappers for the interaction layer (called by the skill via CLI) + def scout_said(self, text: str, kind: str = "message", options=None) -> dict: + return self.log("scout_output", source="scout", kind=kind, text=text, options=options) + + def user_said(self, text: str, kind: str = "input", choice=None) -> dict: + return self.log("user_input", source="user", kind=kind, text=text, choice=choice) + + def qa(self, question: str, answer: str, phase=None, step=None) -> dict: + """A free-form question the engineer asked about a step and Scout's answer. + One record captures both sides so the log shows the real interaction.""" + return self.log("step_qa", source="scout", kind="qa", question=question, + answer=answer, phase=phase, step=step) + + def read(self, limit: int = None) -> list: + return _read_jsonl(self.path, limit) + + +def _read_jsonl(path: str, limit: int = None) -> list: + if not os.path.isfile(path): + return [] + out = [] + with open(path, "r", encoding="utf-8") as fh: + for ln in fh: + ln = ln.strip() + if not ln: + continue + try: + out.append(json.loads(ln)) + except Exception: + continue + return out[-limit:] if limit else out + + +def summarize(events: list) -> dict: + """Roll up a single release's events for quick debugging.""" + by_event, by_source = {}, {} + gate_decisions, declines, interactions, qas = [], [], 0, 0 + for e in events: + by_event[e.get("event")] = by_event.get(e.get("event"), 0) + 1 + by_source[e.get("source")] = by_source.get(e.get("source"), 0) + 1 + if e.get("source") in ("scout", "user"): + interactions += 1 + if e.get("event") == "step_qa": + qas += 1 + if e.get("event") in ("gate_approved", "gate_denied"): + gate_decisions.append({"phase": e.get("phase"), "step": e.get("step"), + "decision": "approved" if e["event"] == "gate_approved" else "denied", + "driver": e.get("driver"), "actor": e.get("actor"), "ts": e.get("ts")}) + if e.get("event") == "readiness_declined": + declines.append({"items": e.get("items"), "owner_blocked": e.get("owner_blocked"), + "driver": e.get("driver"), "ts": e.get("ts")}) + return { + "release": events[0]["release_id"] if events else None, + "total_events": len(events), + "by_source": by_source, + "by_event": by_event, + "interactions_logged": interactions, + "questions_answered": qas, + "gate_decisions": gate_decisions, + "declines": declines, + } diff --git a/release-agent/orchestrator/infra.py b/release-agent/orchestrator/infra.py new file mode 100644 index 00000000..ea3bdc0d --- /dev/null +++ b/release-agent/orchestrator/infra.py @@ -0,0 +1,180 @@ +"""Infrastructure preflight — verify (and auto-provision) everything the skill +needs on a machine BEFORE the tool-level requirements: CLIs, the launchers that +back MCP servers, and the MCP servers themselves registered into Scout's config. + +Design: + * CLI/host/python deps ("requirements") → shell-checked (same as before). + * MCP servers ("mcp_servers") → live in Scout's own config file + (~/.scout/m-mcp-servers.json), which loads at startup. We can't "install" them, + but we CAN register a missing one into that file (backing it up first) so it + loads on the next Scout restart. Each entry names its `provider` (a shell check + that the launcher exists) so we never register a server whose launcher is absent. + +This module is pure-Python and import-light so bootstrap.ps1 can call it via the +CLI (`python -m orchestrator.cli infra`). It only touches Scout's MCP config when +asked to register, and always backs it up. +""" +from __future__ import annotations + +import json +import os +import shutil +import subprocess +from datetime import datetime + +import yaml + + +def scout_mcp_config_path() -> str: + return os.path.join(os.path.expanduser("~"), ".scout", "m-mcp-servers.json") + + +def expand(s: str) -> str: + """Expand %VARS% / $VARS and ~ in a path string.""" + return os.path.expanduser(os.path.expandvars(s or "")) + + +def load_requirements(path: str) -> dict: + with open(path, "r", encoding="utf-8") as fh: + return yaml.safe_load(fh) or {} + + +def _shell_ok(cmd: str, timeout: int = 30) -> bool: + """True if the shell command exits 0. Used for CLI + provider checks.""" + if not cmd: + return False + try: + r = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=timeout) + return r.returncode == 0 + except (subprocess.TimeoutExpired, OSError): + return False + + +def check_requirements(req: dict) -> list: + """Return [{id,name,ok,install}] for each shell-checkable requirement.""" + out = [] + for r in req.get("requirements", []): + out.append({ + "id": r.get("id", ""), "name": r.get("name", r.get("id", "?")), + "ok": _shell_ok(r.get("check", "")), "install": r.get("install", ""), + }) + return out + + +def _load_scout_config(path: str) -> dict: + if not os.path.exists(path): + return {"servers": {}} + with open(path, "r", encoding="utf-8") as fh: + data = json.load(fh) + data.setdefault("servers", {}) + return data + + +def _backup(path: str) -> str: + stamp = datetime.now().strftime("%Y%m%d-%H%M%S") + dst = f"{path}.bak-{stamp}" + shutil.copy2(path, dst) + return dst + + +def ensure_mcp_servers(req: dict, register: bool) -> list: + """Check each required MCP server against Scout's config. When `register` is + True, add any that are missing (whose provider launcher exists), backing up + the config once before the first write. + + Returns [{id,name,scout_key,status,detail}] where status is one of: + present | registered | would_register | provider_missing | launcher_missing + """ + servers = req.get("mcp_servers", []) + if not servers: + return [] + cfg_path = scout_mcp_config_path() + cfg = _load_scout_config(cfg_path) + existing = cfg.get("servers", {}) + results, dirty, backed_up = [], False, None + + for m in servers: + key = m.get("scout_key") or m.get("id") + name = m.get("name", key) + rec = {"id": m.get("id"), "name": name, "scout_key": key} + if key in existing: + results.append({**rec, "status": "present", "detail": "already in Scout config"}) + continue + # Not registered. Is its launcher present? + provider_ok = _shell_ok(m.get("provider", "")) + cmd = expand(m.get("command", "")) + launcher_ok = bool(cmd) and os.path.exists(cmd) + if not provider_ok and not launcher_ok: + results.append({**rec, "status": "provider_missing", + "detail": m.get("note", "provider/launcher not found")}) + continue + if not launcher_ok: + results.append({**rec, "status": "launcher_missing", + "detail": f"launcher not found at {cmd or '(unset)'}"}) + continue + if not register: + results.append({**rec, "status": "would_register", + "detail": "run bootstrap (or infra --register) to add it"}) + continue + # Build args, expanding any dynamic directive (e.g. Kusto known-services + # from a data list) so multi-cluster config stays pure data. + args = list(m.get("args", [])) + ks_from = m.get("known_services_from") + if ks_from: + clusters = req.get(ks_from, []) or [] + known = [{"service_uri": c.get("service_uri"), + "default_database": c.get("default_database"), + "description": c.get("description", "")} + for c in clusters if c.get("service_uri")] + if known: + args += ["--known-services", json.dumps(known)] + # Register it into the config (backup once). + if not backed_up and os.path.exists(cfg_path): + backed_up = _backup(cfg_path) + existing[key] = { + "builtin": False, + "config": {"name": name.split(" MCP")[0].strip() or key, + "type": "command", "command": cmd, "args": args}, + # Command-based servers: Scout reads this allowlist STATICALLY and drops + # the server if it's empty for a server that needs it. Honor a declared + # `tools` list (e.g. the Teams MCP's 36 tools); default [] otherwise. + "tools": list(m.get("tools", []) or []), + } + dirty = True + results.append({**rec, "status": "registered", + "detail": "added to Scout config — RESTART Scout to load"}) + + if dirty: + os.makedirs(os.path.dirname(cfg_path), exist_ok=True) + tmp = cfg_path + ".tmp" + with open(tmp, "w", encoding="utf-8") as fh: + json.dump(cfg, fh, indent=2, ensure_ascii=False) + os.replace(tmp, cfg_path) + return results + + +def run(req_path: str, register: bool = True) -> dict: + """Full infra preflight. Returns a structured report. + + If Scout itself isn't present (~/.scout missing), MCP registration is skipped + (there's no config to write) and the report flags scout_missing — install Scout + first, then re-run. + """ + req = load_requirements(req_path) + reqs = check_requirements(req) + scout_present = os.path.isdir(os.path.join(os.path.expanduser("~"), ".scout")) + if scout_present: + mcps = ensure_mcp_servers(req, register) + else: + # can't register into a non-existent Scout config — report as pending + mcps = [{"id": m.get("id"), "name": m.get("name", m.get("id")), + "scout_key": m.get("scout_key") or m.get("id"), + "status": "scout_missing", "detail": "install Scout first"} + for m in req.get("mcp_servers", [])] + restart_needed = any(m["status"] == "registered" for m in mcps) + ok = (scout_present and all(r["ok"] for r in reqs) + and all(m["status"] in ("present", "registered") for m in mcps)) + return {"requirements": reqs, "mcp_servers": mcps, + "scout_present": scout_present, + "ok": ok, "restart_needed": restart_needed, + "scout_mcp_config": scout_mcp_config_path()} diff --git a/release-agent/orchestrator/knowledge.py b/release-agent/orchestrator/knowledge.py new file mode 100644 index 00000000..144f47ac --- /dev/null +++ b/release-agent/orchestrator/knowledge.py @@ -0,0 +1,82 @@ +"""Step knowledge base — reference/help content for answering user questions. + +When a user asks a detail question about a step ("what does this do?", "where do +I find the Play Console vitals?", "how do I clear this block?"), the skill pulls +the step's knowledge entry and answers ACCURATELY from it instead of guessing. + +Two composable sources, resolved by `get_knowledge(phase, step)`: + 1. config/knowledge.yaml — central data file, keyed ".". Covers any + step (migrated or stub); editable without touching code. + 2. a step MODULE's `KNOWLEDGE` dict — co-located with the step, overlaid ON TOP of + the yaml (module wins per-field) for step-specific detail. + +Returns a plain dict (summary/what/where/how/links/faqs) or None when nothing is +authored yet — so callers can say "no knowledge entry yet" honestly. +""" +from __future__ import annotations +import os + +import yaml + +_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # release-agent/ +_CACHE = {"path": None, "data": None} + + +def knowledge_path() -> str: + return os.environ.get("RELEASE_AGENT_KNOWLEDGE") or \ + os.path.join(_ROOT, "config", "knowledge.yaml") + + +def _load_file(path: str | None = None) -> dict: + p = path or knowledge_path() + if _CACHE["path"] == p and _CACHE["data"] is not None: + return _CACHE["data"] + data = {} + if os.path.exists(p): + with open(p, "r", encoding="utf-8") as fh: + doc = yaml.safe_load(fh) or {} + data = {k: v for k, v in doc.items() if isinstance(v, dict)} + _CACHE["path"], _CACHE["data"] = p, data + return data + + +def _module_knowledge(phase_id: str, step_id: str) -> dict: + """A migrated step module's KNOWLEDGE dict, if it declares one.""" + try: + import steps + mod = steps.get_step(phase_id, step_id) + except Exception: # noqa: BLE001 + mod = None + return dict(getattr(mod, "KNOWLEDGE", {}) or {}) if mod else {} + + +def get_knowledge(phase_id: str, step_id: str, path: str | None = None): + """Merged knowledge for a step (yaml base + module overlay), or None if none. + Per-field overlay: a field present in the module replaces the yaml's field.""" + base = dict(_load_file(path).get(f"{phase_id}.{step_id}", {}) or {}) + overlay = _module_knowledge(phase_id, step_id) + merged = {**base, **overlay} + return merged or None + + +def render_knowledge(phase_id: str, step_id: str, k: dict) -> str: + """Human-readable markdown for a step's knowledge (for `step-info`).""" + lines = [f"## {phase_id}.{step_id}"] + if k.get("summary"): + lines += ["", f"_{k['summary']}_"] + if k.get("what"): + lines += ["", "**What it does**", k["what"].strip()] + if k.get("who"): + lines += ["", "**Who owns it**", k["who"].strip()] + if k.get("where"): + lines += ["", "**Where to look**"] + [f"- {w}" for w in k["where"]] + if k.get("how"): + lines += ["", "**How to complete / resolve**", k["how"].strip()] + if k.get("links"): + lines += ["", "**Links**"] + [f"- [{l.get('name','link')}]({l['url']})" + for l in k["links"] if l.get("url")] + if k.get("faqs"): + lines += ["", "**FAQ**"] + for f in k["faqs"]: + lines += [f"- **{f.get('q','')}** — {f.get('a','')}"] + return "\n".join(lines) diff --git a/release-agent/orchestrator/mocks.py b/release-agent/orchestrator/mocks.py new file mode 100644 index 00000000..e71d2f4d --- /dev/null +++ b/release-agent/orchestrator/mocks.py @@ -0,0 +1,86 @@ +"""Local step mocks — a personal, gitignored testing hook. + +Drop a `mocks.local.yaml` in the release-agent/ root and the engine will REPLACE +the listed steps with the behavior you declare, on every `next` / `step-action`, +for a real release. Everything not listed runs for real, so your +normal Scout interaction is unchanged — you just run the release and the mocked +steps resolve themselves. + +File format (top-level map of "." → behavior): + + # mocks.local.yaml (gitignored — yours, never pushed) + preflight.cg: + outcome: done # done | blocked + note: "mocked: CG clean" + preflight.wiki: + outcome: blocked + reason: "mocked: pretend the wiki create failed" + +A `mocks:` wrapper key is also accepted. `outcome: done` marks the step complete; +`outcome: blocked` holds it for the owner (with `reason`/`note`). Gate steps are +never mockable (a gate needs a real decision). Point at a different file with the +RELEASE_AGENT_MOCKS env var. +""" +from __future__ import annotations +import os + +import yaml + +from phases.stub_runner import StepResult + +_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # release-agent/ +_DONE = {"done", "pass", "ok", "complete"} +_BLOCK = {"blocked", "block", "attention", "hold", "fail"} + + +def mocks_path() -> str: + """Path to the local mocks file (env override, else release-agent/mocks.local.yaml).""" + return os.environ.get("RELEASE_AGENT_MOCKS") or os.path.join(_ROOT, "mocks.local.yaml") + + +def load_mocks(path: str | None = None) -> dict: + """Load the local mocks map keyed '.'. Missing file → {} (the + normal case for CI and any run without a personal mocks file).""" + p = path or mocks_path() + if not os.path.exists(p): + return {} + with open(p, "r", encoding="utf-8") as fh: + doc = yaml.safe_load(fh) or {} + raw = doc.get("mocks", doc) if isinstance(doc, dict) else {} + return {k: v for k, v in raw.items() + if isinstance(v, dict) and k != "version"} + + +def stepresult_for(mocks: dict, phase_id: str, step_id: str): + """Return the mocked StepResult for a step, or None if it isn't mocked. + + done → StepResult(ok=True, ...) → the engine records it complete. + blocked → StepResult(ok=False, ...) → the engine holds it for the owner. + """ + spec = mocks.get(f"{phase_id}.{step_id}") + if not isinstance(spec, dict): + return None + outcome = str(spec.get("outcome", "")).strip().lower() + if outcome in _DONE: + note = spec.get("note") or f"[MOCK] {phase_id}.{step_id} forced pass" + return StepResult(ok=True, action=str(note), by="mock") + if outcome in _BLOCK: + reason = spec.get("reason") or spec.get("note") or f"[MOCK] {phase_id}.{step_id} forced block" + return StepResult(ok=False, action=str(reason), by="mock") + return None + + +def readiness_result(mocks: dict, item_id: str): + """Return (status, message) for a mocked readiness AUTO item, or None. Lets a + local test clear/fail the entry gate without the real ADO / config checks — + key `readiness.` with `outcome: pass|fail` (+ optional `detail`).""" + spec = mocks.get(f"readiness.{item_id}") + if not isinstance(spec, dict): + return None + outcome = str(spec.get("outcome", "")).strip().lower() + detail = spec.get("detail") or spec.get("note") or spec.get("message") + if outcome in _DONE: + return "pass", detail or f"[MOCK] readiness.{item_id} forced pass" + if outcome in _BLOCK or outcome == "fail": + return "fail", detail or f"[MOCK] readiness.{item_id} forced fail" + return None diff --git a/release-agent/orchestrator/notifications.py b/release-agent/orchestrator/notifications.py new file mode 100644 index 00000000..7796b0f7 --- /dev/null +++ b/release-agent/orchestrator/notifications.py @@ -0,0 +1,86 @@ +"""Digest delivery channels — where the daily `tick` digest is sent. + +`config/notifications.yaml` declares the channels (email always; Teams optional) and +the Teams target. `tick`/`notify` call this to (1) report which channels are on and +(2) build a Teams delivery descriptor when a digest is actually due. The engine still +produces exactly ONE digest (render.notification / _html); this module only fans it +out. + +Teams has TWO possible destinations: + * 'scout' (default) → the Scout Teams bot DM (m_send_teams_message). This is the + release owner's Scout notification channel — plain-text digest. + * an explicit chat id → workiq_send_chat_message to that chat (rich HTML). + +Pure + IO-light (reads one yaml) so it's trivially testable; the actual send +side-effects stay in the automation. +""" +from __future__ import annotations + +import os + +import yaml + +# Conservative default when the file is absent: email only (today's behavior), +# Teams off. Adding the file with channels.teams: true opts in. Teams target +# defaults to the Scout bot. +_DEFAULTS = {"channels": {"email": True, "teams": False}, "teams": {"target": "scout"}} + +# Aliases that all mean "the Scout Teams bot" (delivered via m_send_teams_message). +_SCOUT_ALIASES = {None, "scout", "scout_bot", "bot", "self", "me", "owner"} + + +def notifications_path(config_path: str) -> str: + """config/notifications.yaml sits next to phases.yaml (config_path).""" + return os.path.join(os.path.dirname(config_path), "notifications.yaml") + + +def load_config(config_path: str) -> dict: + """Load + merge notifications.yaml over the defaults. Missing file → defaults.""" + p = notifications_path(config_path) + doc = {} + if os.path.exists(p): + try: + with open(p, "r", encoding="utf-8") as fh: + doc = yaml.safe_load(fh) or {} + except (OSError, yaml.YAMLError): + doc = {} + ch = {**_DEFAULTS["channels"], **(doc.get("channels") or {})} + tm = {**_DEFAULTS["teams"], **(doc.get("teams") or {})} + return {"channels": {"email": bool(ch.get("email", True)), + "teams": bool(ch.get("teams", False))}, + "teams": tm} + + +def channels(cfg: dict) -> dict: + """{'email': bool, 'teams': bool} — which channels are enabled.""" + return dict(cfg.get("channels", {})) + + +def teams_target(cfg: dict) -> str: + """The configured Teams target string ('scout' or an explicit chat id).""" + return (cfg.get("teams") or {}).get("target", "scout") + + +def _is_scout_bot(target) -> bool: + return target in _SCOUT_ALIASES + + +def teams_delivery(cfg: dict, html: str, message: str, markdown: str = None): + """How to deliver the Teams copy, or None if Teams is off. + + Scout bot (default): + {"via": "scout_bot", "text": } + → the automation calls m_send_teams_message(message=text). The Scout bot + renders markdown and collapses single newlines, so we send the markdown + digest (blank-line paragraphs / bullets), NOT the plain-text one. + Explicit chat id: + {"via": "chat", "chatId": , "content": , "contentType": "html"} + → the automation calls workiq_send_chat_message(**block). + """ + if not channels(cfg).get("teams"): + return None + target = teams_target(cfg) + if _is_scout_bot(target): + return {"via": "scout_bot", "text": markdown or message} + return {"via": "chat", "chatId": target, + "content": html or f"
{message}
", "contentType": "html"} diff --git a/release-agent/orchestrator/outcomes.py b/release-agent/orchestrator/outcomes.py new file mode 100644 index 00000000..84becb00 --- /dev/null +++ b/release-agent/orchestrator/outcomes.py @@ -0,0 +1,97 @@ +"""Step outcomes — the ONE uniform return contract for every release step. + +Historically the engine had two disjoint mechanisms: `agent` steps returned a +`StepResult` and were run in-process, while `scout` steps had no runner at all — +the engine just held, and the skill did the work via ad-hoc `prepare-X` commands +scattered across `orchestrator/commands/`. That split is why adding a scout step +touched ~6 files. + +This module gives EVERY step one vocabulary. A step handler returns exactly one +of these, and the engine/skill react uniformly: + + Done — the step is complete (an agent did it, or nothing to do). + Blocked — an agent hit a real problem the owner must resolve. + NeedsHuman — a person must confirm/act (attestation or reminder). + NeedsSkill — scout-assisted: the SKILL must run `tool` with `payload` (an MCP + call the engine can't make), then record the step. The step + DESCRIBES the action as data, so the skill executor is generic — + no per-step instructions in the skill's reference docs. + +Pure data — no IO, no engine imports — so it's trivially testable and shared by +the engine, the CLI, and the step handlers. +""" +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any, Optional + + +@dataclass +class Done: + note: str = "" + by: str = "agent" # 'agent' | 'human' + links: list = field(default_factory=list) # [{name, url}] durable refs + kind: str = "done" + + +@dataclass +class Blocked: + reason: str + links: list = field(default_factory=list) # [{name, url}] durable refs + kind: str = "blocked" + + +@dataclass +class InProgress: + """An agent step whose underlying work is STILL RUNNING (not a failure, not done). + + Used by the Phase-2 MRWP verification when the RC pipeline run's overall status is + notStarted/inProgress: the step must NOT block as 'aborted' (a never-ran stage during + an in-flight run is just not-run-YET). The engine holds the phase as 'waiting on the + pipeline' — no user action — and a poller re-runs the step every `poll_in_min` minutes + until the run completes, at which point the normal Done/Blocked rules apply.""" + note: str = "" + links: list = field(default_factory=list) + poll_in_min: int = 30 + kind: str = "in_progress" + + +@dataclass +class NeedsHuman: + prompt: str + attest: bool = False # True → attestation (confirm), False → plain reminder/to-do + kind: str = "needs_human" + + +@dataclass +class NeedsSkill: + """A scout-assisted action the SKILL must execute (an MCP/browser call the + deterministic engine can't make), described as data so the skill is generic. + + tool — the skill tool/verb to run, e.g. 'workiq_send_email', + 'workiq_send_chat_message', or a follow-up engine command name. + payload — kwargs for that tool (already resolved: recipients, subject, + html body, chat target, …). The skill passes it through. + record_as — the step id to `record-step` once the tool succeeds. + summary — a one-line human description ('email the code-complete notice + to recipients') for the skill to show / log. + note — optional detail stored with the recorded step. + outbound — True when performing this action sends something EXTERNAL + (an email, a Teams post, a pipeline trigger) as opposed to a + local follow-up engine command (e.g. check-lockdown). When an + automation runs the step headless, an outbound action gets a + courtesy copy to the owner's Scout DM so they see what went out. + """ + tool: str + payload: dict = field(default_factory=dict) + record_as: str = "" + summary: str = "" + note: str = "" + outbound: bool = False + kind: str = "needs_skill" + + +def as_dict(outcome: Any) -> dict: + """Serialize any outcome to a plain dict (for `--json` CLI output / the skill).""" + d = {k: v for k, v in vars(outcome).items()} + return d diff --git a/release-agent/orchestrator/readiness.py b/release-agent/orchestrator/readiness.py new file mode 100644 index 00000000..925f6d45 --- /dev/null +++ b/release-agent/orchestrator/readiness.py @@ -0,0 +1,218 @@ +"""Release readiness entry gate (logic only — no presentation). + +A self-contained subsystem: given the readiness config (data) + the release +run-state, it computes the checklist, runs auto verifiers, records attestations +and declines, and decides whether the gate is signed/blocked. + +Returns structured data only. Rendering lives in render.py so a different +interface (web UI, other frontend) can present the same data its own way. + +Model: every item is equally required. The only distinction is WHO resolves it: + auto -> Scout verifies it programmatically (pass/fail) + attest -> the engineer confirms it +If any item is unsatisfied the gate is not signed. If the engineer declares they +cannot satisfy an item (decline), the gate is blocked until resolved / handed off. +""" +from __future__ import annotations + +from . import schedule +from .state import ReleaseState, _now + + +class ReadinessGate: + def __init__(self, config: dict, state: ReleaseState): + # config is the parsed readiness.yaml (or None if not configured) + self.config = config + self.state = state + # Local test mocks (set by the engine). `readiness.` entries force an + # AUTO item pass/fail without the real ADO/config check. + self.mocks = {} + + def _window(self, it: dict): + """Compute a CCD-relative window {start,end} for an item that declares + window_start_anchor / window_end_anchor, using the release CCD. Returns + None when the item has no window or CCD isn't set yet.""" + sa, ea = it.get("window_start_anchor"), it.get("window_end_anchor") + ccd = schedule.parse_date(self.state.ccd) + if not (sa and ea and ccd): + return None + try: + return {"start": schedule.anchor_date(ccd, sa).isoformat(), + "end": schedule.anchor_date(ccd, ea).isoformat()} + except ValueError: + return None + + # ---- queries ---- + def checklist(self) -> dict: + """The entry checklist as structured data (no formatting).""" + if not self.config: + return {"items": [], "signed": True, "title": "", "instructions": ""} + items = [] + for it in self.config.get("items", []): + rec = self.state.readiness_items.get(it["id"], {}) or {} + links = [] + for ln in it.get("links", []): + if isinstance(ln, dict): + links.append({"name": ln.get("name", ln.get("url")), "url": ln.get("url")}) + else: + links.append({"name": ln, "url": ln}) + items.append({ + "id": it["id"], "text": it["text"], + "label": it.get("label", it["id"]), + "detail": it.get("detail"), + "links": links, + "verify": it.get("verify", "attest"), # auto | attest (who resolves it) + "source": it.get("source"), # None (python) | "scout" (skill runs it via MCP) + "verifier": it.get("verifier"), + "team_id": it.get("team_id"), # for scout-assisted checks (e.g. on-call team) + "team_name": it.get("team_name"), + "cluster_uri": it.get("cluster_uri"), # for scout-assisted Kusto checks + "database": it.get("database"), + "required_servers": it.get("required_servers"), # for scout-assisted silent-perms check + "opt_out": it.get("opt_out", False), # soft item: may be waived ("degraded") and still satisfy + "window": self._window(it), # {start,end} for windowed attest items + "status": rec.get("status", "pending"), # pending | pass | fail | attested | unable + "message": rec.get("message"), + "checks": rec.get("checks", []), # per-check results for auto items + "satisfied": self._item_satisfied(it), + }) + return { + "title": self.config.get("title", "Release readiness"), + "instructions": self.config.get("instructions", ""), + "blocked_message": self.config.get("blocked_message", ""), + "items": items, + "auto_items": [i for i in items if i["verify"] == "auto"], + "attest_items": [i for i in items if i["verify"] == "attest"], + "signed": self.state.readiness_signed, + "signed_at": self.state.readiness_signed_at, + "blocked": self.state.blocked, + "blocked_items": list(self.state.blocked_items), + "all_satisfied": all(i["satisfied"] for i in items) if items else True, + } + + @property + def signed(self) -> bool: + return self.state.readiness_signed + + @property + def blocked(self) -> bool: + return self.state.blocked + + # ---- mutations ---- + def verify(self) -> dict: + """Run the AUTO verifiers. Each returns pass or fail — no half-measures. + Items with source: scout are skipped here — the skill runs those via its + MCP tools and records the result with record_check().""" + from phases.readiness_verifiers import get_verifier + from . import mocks as mocks_mod + if not self.config: + return self.checklist() + for it in self.config.get("items", []): + if it.get("verify") != "auto": + continue + # Local test mock: force ANY auto item (python OR scout-assisted) pass/fail, + # skipping the real ADO/config/MCP check — lets a test clear the gate offline. + mock = mocks_mod.readiness_result(self.mocks, it["id"]) + if mock is not None: + status, message = mock + self.state.readiness_items[it["id"]] = { + "status": status, "message": message, "at": _now()} + continue + if it.get("source") == "scout": + continue # skill-run (MCP) — not executable here + vf = get_verifier(it.get("verifier")) + if vf is None: + self.state.readiness_items[it["id"]] = {"status": "fail", + "message": f"no verifier '{it.get('verifier')}' registered", "at": _now()} + continue + res = vf(it) + rec = {"status": res.status, "message": res.message, "at": _now()} + if getattr(res, "details", None): + rec["checks"] = res.details + self.state.readiness_items[it["id"]] = rec + self._refresh_signed() + return self.checklist() + + def record_check(self, item_id: str, status: str, message: str = "") -> dict: + """Record the result of a SCOUT-ASSISTED auto check (source: scout), which + the skill performs via its MCP tools (e.g. the ICM on-call lookup). Only + valid for auto items marked source: scout — a Python-verified auto item + (e.g. build_access) cannot be hand-recorded, and attest items use sign(). + + Status is normally 'pass' | 'fail'. Items marked `opt_out: true` (soft items + the user may proceed WITHOUT — e.g. silent_perms) also accept 'degraded', + which SATISFIES the gate while recording that they chose to proceed without + the capability (the downside is captured in `message`).""" + by_id = {it["id"]: it for it in (self.config or {}).get("items", [])} + it = by_id.get(item_id) + if it is None: + return {"error": f"no such readiness item '{item_id}'"} + if not (it.get("verify") == "auto" and it.get("source") == "scout"): + return {"error": f"'{item_id}' is not a scout-assisted auto item (cannot record a result for it)"} + valid = ("pass", "fail") + (("degraded",) if it.get("opt_out") else ()) + if status not in valid: + return {"error": f"status must be one of: {', '.join(valid)}"} + self.state.readiness_items[item_id] = { + "status": status, "message": message, "at": _now(), "source": "scout"} + self._refresh_signed() + return self.checklist() + + def sign(self, item_ids=None, note=None) -> dict: + """Attest human (attest) items and run auto verifiers. item_ids=None + attests every attest item (library convenience — the CLI never does this; + it requires explicit item ids). Auto items are only set by verification — + they cannot be hand-waved through. `note` records the human's confirmation + as evidence on each attested item. Signs when all items are satisfied.""" + if not self.config: + self.state.readiness_signed = True + self.state.readiness_signed_at = _now() + return self.checklist() + self.verify() # auto items (pass/fail) + all_items = self.config.get("items", []) + attest_ids = [it["id"] for it in all_items if it.get("verify", "attest") == "attest"] + targets = attest_ids if item_ids is None else [i for i in item_ids if i in attest_ids] + for iid in targets: + rec = {"status": "attested", "at": _now()} + if note: + rec["note"] = note + self.state.readiness_items[iid] = rec + self._refresh_signed() + return self.checklist() + + def decline(self, item_ids: list) -> dict: + """The engineer declares they CANNOT satisfy one or more items. Every item + is required, so any declined item blocks the gate until resolved / handed off.""" + if not self.config: + return self.checklist() + by_id = {it["id"]: it for it in self.config.get("items", [])} + for iid in item_ids or []: + if iid not in by_id: + continue + self.state.readiness_items[iid] = {"status": "unable", "at": _now()} + self.state.blocked = True + if iid not in self.state.blocked_items: + self.state.blocked_items.append(iid) + self.state.readiness_signed = False + return self.checklist() + + def blocked_labels(self) -> list: + """Human labels for the currently-blocked item ids.""" + by_id = {it["id"]: it.get("label", it["id"]) for it in (self.config or {}).get("items", [])} + return [by_id.get(b, b) for b in self.state.blocked_items] + + # ---- internals ---- + def _item_satisfied(self, item: dict) -> bool: + status = (self.state.readiness_items.get(item["id"], {}) or {}).get("status") + if item.get("verify", "attest") == "auto": + if item.get("opt_out"): + # soft item: fully verified (pass) OR user chose to proceed (degraded) + return status in ("pass", "degraded") + return status == "pass" # auto is fully verified: pass or nothing + return status == "attested" # attest is human-confirmed + + def _refresh_signed(self) -> None: + items = self.config.get("items", []) if self.config else [] + if items and all(self._item_satisfied(it) for it in items): + if not self.state.readiness_signed: + self.state.readiness_signed = True + self.state.readiness_signed_at = _now() diff --git a/release-agent/orchestrator/registry.py b/release-agent/orchestrator/registry.py new file mode 100644 index 00000000..b004b84a --- /dev/null +++ b/release-agent/orchestrator/registry.py @@ -0,0 +1,173 @@ +"""Automation registry — tracks the Scout automations the orchestrator provisions +for a release, so they can be cleanly torn down at release close. + +The engine/CLI never call Scout's automation API (creating/deleting automations is +the skill's job via m_create_automation / m_delete_automation). This module only +RECORDS which automations exist, tagged by release + scope + the STEPS each drives, +so the skill knows exactly what to remove at the end (nothing gets orphaned) and so +automation<->step linkage is queryable both ways (`list(release=…)` shows an +automation's steps; `list(step=…)` shows which automation owns a step). + +Two scopes: + * shared — machine-wide, reused across releases (e.g. "Release push reminders"). + NOT torn down per release. + * release — provisioned for one release; removed when that release closes. + +Two kinds (what an automation acts on): + * step-driving — runs one or more named steps at a fire time (owns steps[]). + * release-level — operates on the whole release, not a step (e.g. the hourly + `tick` push-reminder). Owns NO steps. + +Storage layout: + * //_automations.json — a release's own automations, co-located + with its release-state.json so ownership is explicit (and they're removed with the + release folder at close). + * /_automations.json — SHARED (machine-wide) automations only; these are + reused across releases and are not tied to any one release folder. +""" +from __future__ import annotations + +import glob +import json +import os +from datetime import datetime, timezone + +KINDS = ("release-level", "step-driving") + + +def kind_of(entry: dict) -> str: + """The automation's kind, deriving it for older entries that predate the field: + an entry that drives steps is step-driving, otherwise release-level.""" + k = entry.get("kind") + if k in KINDS: + return k + return "step-driving" if (entry.get("steps") or []) else "release-level" + + +def _now() -> str: + return datetime.now(timezone.utc).isoformat() + + +class AutomationRegistry: + def __init__(self, runs_root: str, release: str = None): + self.runs_root = runs_root + self.release = release + # SHARED (machine-wide) automations only; release automations live in /. + self.shared_path = os.path.join(runs_root, "_automations.json") + + # ---- paths ---- + def _release_path(self, release: str) -> str: + """A release's own registry file, next to its release-state.json.""" + return os.path.join(self.runs_root, release, "_automations.json") + + def _release_files(self) -> list: + """Every per-release registry file under runs_root.""" + return sorted(glob.glob(os.path.join(self.runs_root, "*", "_automations.json"))) + + # ---- file IO ---- + def _load_file(self, path: str) -> list: + try: + with open(path, "r", encoding="utf-8") as fh: + data = json.load(fh) + return data if isinstance(data, list) else [] + except (OSError, ValueError): + return [] + + def _save_file(self, path: str, entries: list) -> None: + os.makedirs(os.path.dirname(path) or ".", exist_ok=True) + tmp = path + ".tmp" + with open(tmp, "w", encoding="utf-8") as fh: + json.dump(entries, fh, indent=2) + os.replace(tmp, path) + + def _path_for(self, entry: dict) -> str: + """Where an entry is stored: its release folder, else the shared file.""" + if entry.get("scope") == "release" and entry.get("release"): + return self._release_path(entry["release"]) + return self.shared_path + + # ---- api ---- + def register(self, auto_id: str, name: str, release: str = None, + shared: bool = False, purpose: str = "", steps: list = None, + kind: str = None, schedule: str = None, slug: str = None) -> dict: + """Record an automation (upsert by id). Shared automations store release=None and + live in the machine-wide file; release automations live in /. `steps` is + the list of '.' ids this automation drives — the automation<->step + linkage used for traceability. + + `slug` is the stable identity from config/automations.yaml (e.g. 'ccd-noon'). + It's the reliable key for `automation sync` — matching by steps alone is + ambiguous when two automations share a step (the noon trigger and the poller + both drive ccd.localization). `schedule` is the Scout schedule it was created + with, stored so sync can detect when a CCD change made a cron schedule stale. + + `kind` is 'step-driving' (owns steps) or 'release-level' (whole-release, no + steps). Omit to auto-derive from `steps`. The two can't contradict: + step-driving requires steps; release-level forbids them.""" + steps = list(steps or []) + derived = "step-driving" if steps else "release-level" + if kind is None: + kind = derived + elif kind not in KINDS: + raise ValueError(f"kind must be one of {KINDS}, got {kind!r}") + elif kind == "step-driving" and not steps: + raise ValueError("a 'step-driving' automation must declare at least one step") + elif kind == "release-level" and steps: + raise ValueError("a 'release-level' automation must not own steps") + entry = { + "id": auto_id, + "name": name, + "slug": slug or None, + "kind": kind, + "scope": "shared" if shared else "release", + "release": None if shared else (release or self.release), + "purpose": purpose, + "steps": steps, + "schedule": schedule or None, + "registered_at": _now(), + } + # Upsert: drop any prior copy of this id wherever it lived, then write to its + # (possibly new) home file. + self._remove_everywhere(auto_id) + path = self._path_for(entry) + entries = self._load_file(path) + entries.append(entry) + self._save_file(path, entries) + return entry + + def _remove_everywhere(self, auto_id: str) -> bool: + """Drop `auto_id` from the shared file and every per-release file. Returns True + if it was found somewhere.""" + removed = False + for path in [self.shared_path, *self._release_files()]: + entries = self._load_file(path) + kept = [e for e in entries if e.get("id") != auto_id] + if len(kept) != len(entries): + self._save_file(path, kept) + removed = True + return removed + + def deregister(self, auto_id: str) -> bool: + return self._remove_everywhere(auto_id) + + def list(self, release: str = None, scope: str = None, step: str = None, + kind: str = None) -> list: + """List entries. `release` filters to that release's automations (and reads only + that release's file + the shared file). `scope` filters by 'shared'/'release'. + `step` filters to automations that DRIVE that '.' id (reverse + lookup). `kind` filters by 'step-driving'/'release-level'.""" + entries = self._load_file(self.shared_path) + if release is not None: + entries += self._load_file(self._release_path(release)) + else: + for path in self._release_files(): + entries += self._load_file(path) + if release is not None: + entries = [e for e in entries if e.get("release") == release] + if scope is not None: + entries = [e for e in entries if e.get("scope") == scope] + if step is not None: + entries = [e for e in entries if step in (e.get("steps") or [])] + if kind is not None: + entries = [e for e in entries if kind_of(e) == kind] + return entries diff --git a/release-agent/orchestrator/render.py b/release-agent/orchestrator/render.py new file mode 100644 index 00000000..3733cc4f --- /dev/null +++ b/release-agent/orchestrator/render.py @@ -0,0 +1,644 @@ +"""Presentation layer — turns structured data into text/markdown. + +Kept separate from the logic (engine.py / readiness.py) so a different interface +(web UI, TUI, another frontend) can consume the same structured data and present +it its own way. These are pure functions: data in, string out. No state, no IO. +""" +from __future__ import annotations + +import re + +# First URL anywhere in a note (e.g. the wiki page link) — surfaced as a compact +# [link](…) in the Details column instead of a long raw URL. +_URL_RE = re.compile(r"https?://[^\s)>\]]+") + + +def _cell(text: str) -> str: + """Make arbitrary text safe for a single markdown table cell: collapse all + whitespace/newlines to one line and escape pipes.""" + return re.sub(r"\s+", " ", str(text)).replace("|", "\\|").strip() + + +def step_detail(s: dict, limit: int = 160) -> str: + """One-line 'Details' summary of a step's execution, from its stored note. + + Generic — works for ANY step (current or future phases) because every step + records its outcome as `note` (agent report, block reason, …) and may attach + structured `links` ([{name,url}], e.g. the wiki page or CG alerts page). + Prefers the stored `links` for the reference; falls back to the first URL found + in the note. Empty when nothing ran yet; an em dash for outstanding items. + """ + note = s.get("note") + links = s.get("links") or [] + if not note and not links: + state = s.get("state") + return "—" if state in ("pending", "scheduled", "reminder", "gate", "auto") else "" + text = str(note or "").strip() + lead = next((ln.strip() for ln in text.splitlines() if ln.strip()), "") + # Prefer STRUCTURED links (first-class, stored on state) over URL-in-prose. + if links: + link_md = " · ".join(f"[{l.get('name', 'link')}]({l['url']})" + for l in links if l.get("url")) + lead_c = _cell(lead).rstrip(" :.—-") + if len(lead_c) > limit: + lead_c = lead_c[: limit - 1].rstrip() + "…" + return f"{lead_c} · {link_md}" if lead_c else link_md + url = _URL_RE.search(text) + if url and url.group(0) in lead: + # lead itself carries the URL — strip it, we re-add it as a clean link + lead = lead.replace(url.group(0), "") + lead_c = _cell(lead).rstrip(" :.—-") + if len(lead_c) > limit: + lead_c = lead_c[: limit - 1].rstrip() + "…" + if url: + link = f"[link]({url.group(0)})" + return f"{lead_c} · {link}" if lead_c else link + return lead_c + + +# ---- readiness entry gate (the frozen table) ---- +_ICON = {"pass": "✅", "attested": "✅", "fail": "❌", "unable": "⛔", + "degraded": "⚠️", "pending": "⬜"} +_STATUS_WORD = {"pass": "PASS", "attested": "Confirmed", "fail": "FAIL", + "unable": "Unable", "degraded": "Proceeding (not silent)", + "pending": "Outstanding"} + + +def readiness_table(chk: dict, release_id: str) -> str: + """Canonical markdown table for the entry gate (frozen layout). + `chk` is ReadinessGate.checklist().""" + if not chk.get("items"): + return "No readiness checklist configured." + lines = [ + f"### Readiness Entry Gate — {release_id}", + "", + _cell(chk["instructions"]), + "", + "**Type legend:** `[auto]` Scout verifies · `[attest]` you confirm", + "", + "| | Type | Item | Status |", + "|---|---|---|---|", + ] + for it in chk["items"]: + box = _ICON.get(it["status"], "⬜") + typ = "`[auto]`" if it["verify"] == "auto" else "`[attest]`" + label = it.get("label") or it["id"] + if it.get("checks"): + parts = [f"[{c['name']}]({c['url']}) {'✓' if c.get('ok') else '✗'}" + if c.get("url") else f"{c['name']} {'✓' if c.get('ok') else '✗'}" + for c in it["checks"]] + detail = " · ".join(parts) + else: + detail = it.get("detail") or it.get("text") or "" + win = it.get("window") + if win and win.get("start") and win.get("end"): + detail = f"{detail} (window: {win['start']} → {win['end']})".strip() + item_cell = f"**{_cell(label)}** — {_cell(detail)}" + lines.append(f"| {box} | {typ} | {item_cell} | {_STATUS_WORD.get(it['status'], it['status'])} |") + lines.append("") + + if chk["blocked"]: + labels = [next((i["label"] for i in chk["items"] if i["id"] == b), b) + for b in chk["blocked_items"]] + lines.append("⛔ **Blocked** — cannot start: " + ", ".join(labels) + ".") + lines.append(_cell(chk["blocked_message"])) + elif chk["signed"]: + lines.append("✅ **All items satisfied — entry gate cleared.** Ready to start Phase 0.") + else: + pending = [i["label"] for i in chk["items"] if not i["satisfied"]] + lines.append("**Outstanding:** " + ", ".join(pending)) + return "\n".join(lines) + + +def attest_prompt(chk: dict) -> str: + """Public: the '✋ Your confirmation needed' block on its own (no table), or a + short status line when it's not time yet / nothing to attest. Rendered as the + SECOND readiness output — after the full table (render #1) and the silent auto + checks — so the table appears exactly ONCE while the attestation ask still comes + deterministically from the engine (the item list is not hand-built by the skill).""" + block = _attest_prompt_block(chk) + if block: + return block + if chk.get("signed"): + return "✅ All items satisfied — entry gate cleared. Ready to start Phase 0." + # auto checks not yet complete + auto_pending = [i["label"] for i in chk["items"] + if i["verify"] == "auto" and not i["satisfied"]] + if auto_pending: + return ("Automated checks still pending: " + ", ".join(auto_pending) + + " — resolve these before the attestations.") + return "No attestations outstanding." + + +def _attest_prompt_block(chk: dict) -> str: + """The 'your confirmation needed' section, or '' when it's not time yet. + Only returned when: gate unsigned, not blocked, ALL auto items satisfied, and + at least one attest item is still outstanding.""" + auto = [i for i in chk["items"] if i["verify"] == "auto"] + if auto and not all(i["satisfied"] for i in auto): + return "" # auto checks not finished — don't prompt for attestations yet + outstanding = [i for i in chk["items"] + if i["verify"] == "attest" and not i["satisfied"]] + if not outstanding: + return "" + out = ["---", "### ✋ Your confirmation needed", + "The automated checks are done. I'll sign **only** what you explicitly " + "confirm — please verify each:", ""] + for it in outstanding: + detail = it.get("detail") or it.get("text") or "" + win = it.get("window") + if win and win.get("start") and win.get("end"): + detail = f"{detail} (window: {win['start']} → {win['end']})".strip() + out.append(f"- **{_cell(it.get('label') or it['id'])}** — {_cell(detail)}") + return "\n".join(out) + + +# Short "can't satisfy" card title + one-line description per attest item, so the +# m_ask_user card is self-describing even when no markdown table renders. +_ATTEST_CARD = { + "play_console_access": ("I can't open Play Console", "Play Console dashboard doesn't load"), + "oncall_window": ("I'm on-call during the window", "Scheduled Android on-call during the release window"), + "saw_ame": ("I don't have a SAW machine", "Can't access SAW / AME"), + "yubikey": ("I don't have a YubiKey", "No YubiKey in hand"), +} + + +def attest_prompt_payload(chk: dict, release_id: str) -> dict: + """DETERMINISTIC m_ask_user payload for the readiness attestations — the engine + owns the exact question + answer cards so the (always-rendered) Scout card is the + source of truth, independent of whether the markdown table rendered. + + Returns a dict: + { ready: bool, release, question, answers:[{title,description,action,item?}], + confirm_items:[ids], recommendedIndex, reason } + `ready` is False (with `reason`) when it's not time to attest yet (auto checks + pending) or nothing is outstanding — the skill should not prompt in that case.""" + auto = [i for i in chk["items"] if i["verify"] == "auto"] + if auto and not all(i["satisfied"] for i in auto): + pend = ", ".join(i["label"] for i in auto if not i["satisfied"]) + return {"ready": False, "release": release_id, + "reason": f"auto checks pending: {pend}"} + outstanding = [i for i in chk["items"] + if i["verify"] == "attest" and not i["satisfied"]] + if not outstanding: + return {"ready": False, "release": release_id, + "reason": "no attestations outstanding"} + + # "All confirmed" description: concise per-item summary (with the on-call dates). + bits = [] + for it in outstanding: + lbl = it.get("label") or it["id"] + win = it.get("window") + if it["id"] == "oncall_window" and win and win.get("start") and win.get("end"): + bits.append(f"free of on-call {win['start']}–{win['end']}") + else: + bits.append(lbl) + confirm_desc = " · ".join(bits) + + answers = [{"title": "All confirmed", "description": _cell(confirm_desc), + "action": "confirm_all"}] + for it in outstanding: + title, desc = _ATTEST_CARD.get( + it["id"], (f"Can't satisfy {it.get('label') or it['id']}", "")) + answers.append({"title": title, "description": desc, + "action": "decline", "item": it["id"]}) + + return { + "ready": True, + "release": release_id, + "question": (f"Automated readiness checks passed — confirm all " + f"{len(outstanding)} remaining item(s) to start release {release_id}?"), + "answers": answers, + "confirm_items": [it["id"] for it in outstanding], + "recommendedIndex": 0, + } + + + +# Plain-language labels for internal engine states (never show raw state names). +_STATE_LABEL = { + "not_started": "Not started", + "running": "In progress", + "scheduled": "Scheduled — waiting for the window to open", + "awaiting_action": "Action needed from you", + "holding_gate": "Waiting for your approval", + "readiness_gate": "Entry gate — checklist pending", + "blocked": "Blocked", + "halted": "Halted", + "complete": "Complete", +} +_PHASE_ICON = {"done": "✅", "current": "⏸", "pending": "⬜", "scheduled": "🗓"} +_STEP_ICON = {"done": "✅", "gate": "⏸", "reminder": "📌", "scheduled": "🗓", + "pending": "⬜", "skipped": "⏭️", "scout": "🤖", "auto": "🤖", "blocked": "⛔", + "in_flight": "⏳"} +_STEP_STATE_WORD = {"done": "Done", "gate": "Awaiting your approval", + "reminder": "Do this — then mark done", "scheduled": "Not open yet", + "pending": "Pending", "skipped": "Skipped", + # 'auto' (engine-run agent step) and 'scout' (skill-run MCP step) are BOTH + # Scout's automatic work — shown identically so the human sees one message + # (no action needed), never a confusing "Pending" next to "automatic". + "auto": "Scout runs this — automatic", + "scout": "Scout runs this — automatic", "blocked": "Blocked — needs you", + # in_flight = the pipeline run is still executing. No user action — Scout + # is polling every 30 min and re-evaluates when the run completes. + "in_flight": "RC running — Scout is polling"} + + +def _pipelines_line(r: dict) -> str: + """Compact one-line summary of the Phase-2 release-pipeline runs recorded on state + (checker → orchestrator → the LATEST RC's two MRWP runs). Empty string when none + resolved yet. Reads the nested pipeline_runs schema; no live az call in the render path.""" + from tools.pipelines import format_versions + pr = r.get("pipeline_runs") or {} + parts = [] + ch = pr.get("checker") or {} + if ch.get("run_id"): + parts.append(f"checker {ch['run_id']}") + o = pr.get("orchestrator") or {} + if o.get("run_id"): + vstr = format_versions(o.get("versions")) + parts.append(f"orchestrator {o['run_id']}" + (f" ({vstr})" if vstr else "")) + rcs = pr.get("rcs") or [] + rc = rcs[-1] if rcs else {} + mr = [] + if (rc.get("ecs") or {}).get("run_id"): + mr.append(f"ECS {rc['ecs']['run_id']}") + if (rc.get("local") or {}).get("run_id"): + mr.append(f"Local {rc['local']['run_id']}") + if mr: + tag = f" (RC{rc['rc']})" if rc.get("rc") and len(rcs) > 1 else "" + parts.append("MRWP" + tag + " " + " / ".join(mr)) + return " · ".join(parts) + + +def status_view(r: dict) -> str: + """Human-readable status: next-action headline → phase map → current-phase steps. + `r` is Orchestrator.status_report().""" + bars = 20 + filled = round(bars * r["done"] / r["total"]) if r["total"] else 0 + bar = "█" * filled + "░" * (bars - filled) + label = _STATE_LABEL.get(r["status"], r["status"]) + + lines = [ + f"## Release {r['release_id']} · {r['done']}/{r['total']} ({r['percent']}%)", + f"`{bar}`", + ] + # Code Complete Date anchor line (when known). + if r.get("ccd"): + src = {"override": "override", "manual": "confirmed", + "default": "2nd Wednesday"}.get(r.get("ccd_source"), r.get("ccd_source") or "") + srctag = f" ({src})" if src else "" + skip = " · ⚠ release SKIP set in pipeline" if r.get("skip_release") else "" + lines.append(f"**Code Complete:** {r['ccd']}{srctag} · **today:** {r.get('as_of','')}{skip}") + if r.get("ccd_conflict"): + lines.append(f"⚠ **Confirm the date** — the pipeline override is **{r['ccd_conflict']}**, " + f"which differs from the 2nd-Wednesday default (**{r['ccd']}**). " + f"Which is the real Code Complete Date — the default, or the pipeline's?") + if r.get("owner_email"): + who = (f"{r['owner_name']} " if r.get("owner_name") else "") + f"{r['owner_email']}" + lines.append(f"**Owner:** {who}") + lines.append("") + + # 1) Next-action headline — the single most important thing. + if r.get("halted"): + rsn = f" — {r['halt_reason']}" if r.get("halt_reason") else "" + lines.append(f"⛔ **HALTED**{rsn}. Nothing advances until you resume.") + elif r["blocked"]: + lines.append(f"⛔ **Blocked** — cannot start: {', '.join(r['blocked_items'])}. " + "Resolve it, or hand the release to someone who can.") + elif not r["readiness_signed"]: + lines.append("▣ **Entry gate** — the readiness checklist isn't signed yet.") + elif r.get("scheduled"): + sc = r["scheduled"] + when = _delta_phrase(sc.get("opens_in_days")) + lines.append(f"🗓 **Scheduled** — **{sc['phase_name']}** opens **{sc.get('opens','')}** " + f"({when}). Nothing to do yet.") + elif r.get("action"): + a = r["action"] + lines.append(f"📌 **Action needed** — you need to: **{a['step_name']}** " + f"(Phase {_phase_num(r, a['phase'])} · {a['phase_name']}). Mark it done when complete.") + elif r["gate"]: + g = r["gate"] + lines.append(f"⏸ **Next: your decision** — approve or deny **{g['step_name']}** " + f"(Phase {_phase_num(r, g['phase'])} · {g['phase_name']}).") + elif r["status"] == "complete": + lines.append("✔ **Release complete.** All phases done.") + elif r["status"] == "not_started": + lines.append("▶ **Not started yet** — run the next step to begin.") + else: + # in progress, between gates + nxt = _next_pending_step(r) + if nxt: + lines.append(f"▶ **In progress** — next up: **{nxt}** " + f"({r.get('current_phase_name') or ''}).") + else: + lines.append(f"▶ **{label}.**") + + # 2) Phase map (overview). Icon is prefixed onto the Phase name (no separate + # empty-header icon column — that renders with a huge gap in Scout's tables). + if r.get("phases"): + lines += ["", "### Phases", "| # | Phase | Done |", "|---|---|---|"] + for p in r["phases"]: + icon = _PHASE_ICON.get(p["state"], "⬜") + note = "" + if p["current"]: + note = " ← you are here" + elif p["state"] == "scheduled" and p.get("opens"): + note = f" · opens {p['opens']} ({_delta_phrase(p.get('opens_in_days'))})" + lines.append(f"| {p['num']} | {icon} {p['name']}{note} | {p['done']}/{p['total']} |") + # Blank line BEFORE the legend so markdown ends the table and renders the + # legend as its own caption paragraph — otherwise it's absorbed as a row + # (all glued into one column). + lines += ["", "_✅ done · ⏸ in progress · 🗓 scheduled · ⬜ not started_"] + + # 3) Current-phase detail (drill-down). Icon prefixed onto the Step name. + # A third "Details" column captures each step's execution outcome (from its + # stored note): where a lockdown clashed, the breaking change, CG alerts found, + # the created wiki link, etc. Generic — any step that records a note shows it. + if r.get("current_steps"): + lines += ["", f"### ▶ Current phase — {r.get('current_phase_name','')}", + "| Step | State | Details |", "|---|---|---|"] + for s in r["current_steps"]: + icon = _STEP_ICON.get(s["state"], "⬜") + word = _STEP_STATE_WORD.get(s["state"], s["state"]) + tag = " 🚦" if s["gate"] else (" 📌" if s.get("reminder") else "") + detail = step_detail(s) + lines.append(f"| {icon} {s['name']}{tag} | {word} | {detail} |") + + # Phase-2 pipeline run ids (checker/orchestrator/MRWP) — recorded on state as + # the verification steps resolve the chain, so they show without a live read. + pl = _pipelines_line(r) + if pl: + lines += ["", f"**Pipelines:** {pl}"] + + # 3b) Expanded detail — only for steps whose note has MORE than the one-line + # summary the column shows (e.g. the CG report's full High/Critical CVE list, + # or the breaking-change draft comms). Simple one-line notes (lockdown "no + # overlap", cron "firing") are already fully shown in the column, so they're + # not repeated here. Generic: any step with a multi-line note expands. + def _multiline(note): + return len([ln for ln in str(note).splitlines() if ln.strip()]) > 1 + + rich = [s for s in r["current_steps"] + if (s.get("note") and _multiline(s["note"])) or s.get("links")] + if rich: + lines += ["", "### Step details"] + for s in rich: + mark = _STEP_ICON.get(s["state"], "•") + note_lines = [ln.rstrip() for ln in str(s.get("note") or "").strip().split("\n")] + body = " \n".join(ln for ln in note_lines if ln) # markdown hard breaks + lines += ["", f"{mark} **{s['name']}** ", body] + for l in (s.get("links") or []): + if l.get("url"): + lines.append(f"🔗 [{l.get('name', 'link')}]({l['url']}) ") + + return "\n".join(lines) + + +def _delta_phrase(days) -> str: + if days is None: + return "" + if days == 0: + return "today" + if days == 1: + return "tomorrow" + if days == -1: + return "yesterday" + return f"in {days} days" if days > 0 else f"{-days} days ago" + + +def _phase_num(r: dict, phase_id: str): + for p in r.get("phases", []): + if p["id"] == phase_id: + return p["num"] + return "?" + + +def _next_pending_step(r: dict): + for s in r.get("current_steps", []): + if s["state"] in ("pending", "gate", "auto"): + return s["name"] + return None + + +def notification_subject(r: dict) -> str: + """Email subject for the daily phase digest (empty if nothing to send).""" + ap = r.get("active_phase") + if not ap: + return f"Release {r.get('release_id','?')} — update" + return f"Release {r.get('release_id','?')} — Phase {ap.get('num')} status" + + +def _digest_model(r: dict): + """The digest's content model, or None to STAY SILENT — the single source of + truth shared by the plain-text and markdown renderers (the HTML email uses a + fuller step table but reuses this as its silence gate). + + Silence rules (established with the user): + * Setup (readiness + CCD) is interactive in Scout — NO push. An unsigned + release, a blocked entry gate, or a halted/complete release stay silent. + * The FIRST push is a phase opening (Phase 0 at CCD-7). Nothing before it. + * While a phase is open (due) with outstanding steps, report status daily. + """ + if (r.get("halted") or r.get("blocked") or r.get("status") == "complete" + or not r.get("readiness_signed")): + return None # setup / paused — no push + ap = r.get("active_phase") + if not ap or not ap.get("due"): + return None # nothing open yet (scheduled) — no push + # Scout still owes automatic steps on the open phase (notice / reminders / lockdown + # not yet run). The digest reports the *settled* "here's what needs YOU" picture, so + # sending it now would be premature and half-run — stay silent until Scout drains its + # own steps (a blocked scout step is status 'blocked', not 'scout', so it still pushes). + if r.get("scout_pending"): + return None + + hold = None + if r.get("gate"): + hold = ("gate", r["gate"]["step_name"]) + elif r.get("action"): + hold = ("action", r["action"]["step_name"]) + human_all = [o for o in ap.get("outstanding", []) if o["gate"] or o["reminder"]] + completed_all = ap.get("completed") or [] + # Phase-2 RC one-liner — only while a phase that opts in (show_pipeline_runs) is + # active, best-effort (reads state.pipeline_runs; never a live call). Empty until the + # chain resolves. + rc_line = _pipelines_line(r) if ap.get("show_pipeline_runs") else "" + return { + "rid": r.get("release_id", "?"), + "ap": ap, + "started": bool(ap.get("started")), + "completed": completed_all[:8], + "completed_total": len(completed_all), + "hold": hold, # (kind, step_name) or None + "human": human_all[:6], + "human_total": len(human_all), + "pipelines": rc_line, # RC id one-liner (build_verify only) + } + + +def _progress_line(m: dict, bold: bool = False) -> str: + """The opened/progress line, shared by both text renderers.""" + ap = m["ap"] + if not m["started"]: + opened = f" (opened {ap['opens']})" if ap.get("opens") else "" + total = f"**{ap['total']}**" if bold else str(ap["total"]) + return f"Phase {ap['num']} has opened{opened} — {total} steps to work through, none done yet." + prog = f"**{ap['done']} of {ap['total']}**" if bold else f"{ap['done']} of {ap['total']}" + return f"Progress: {prog} steps done." + + +def notification(r: dict) -> str: + """The DAILY PHASE DIGEST emailed to the release owner, or "" to stay silent. + `r` is Orchestrator.status_report(). Plain-text form (email fallback / logs); + the markdown and HTML forms render the same model differently. The once-per-day + cadence is enforced by the CLI (last_notified_date).""" + m = _digest_model(r) + if m is None: + return "" + ap = m["ap"] + lines = [f"Release {m['rid']} — Phase {ap['num']}: {ap['name']}", _progress_line(m)] + if m.get("pipelines"): + lines.append(f"RC pipelines: {m['pipelines']}") + if m["completed"]: + lines.append(f"Completed ({m['completed_total']}):") + lines += [f" ✓ {name}" for name in m["completed"]] + if m["hold"]: + kind, name = m["hold"] + lines.append(f"Waiting on your decision: {name} (approve or deny)." if kind == "gate" + else f"Action needed now: {name} (do it, then mark done).") + if m["human"]: + lines.append(f"Still needs you ({m['human_total']}):") + lines += [f" • {o['name']} — {'your approval' if o['gate'] else 'your action'}" + for o in m["human"]] + lines.append("Open Scout to continue the release.") + return "\n".join(lines) + + +def notification_markdown(r: dict) -> str: + """Teams-bot-friendly MARKDOWN digest — same content as notification() but with + blank-line paragraph breaks, `-` bullets and `**bold**`. The Scout bot renders + markdown and COLLAPSES single newlines, so the plain-text form would arrive as + one run-on paragraph. Empty under the same silence rules.""" + m = _digest_model(r) + if m is None: + return "" + ap = m["ap"] + blocks = [f"**Release {m['rid']} — Phase {ap['num']}: {ap['name']}**", _progress_line(m, bold=True)] + if m.get("pipelines"): + blocks.append(f"**RC pipelines:** {m['pipelines']}") + if m["completed"]: + blocks.append("\n".join([f"**Completed ({m['completed_total']}):**"] + + [f"- ✓ {n}" for n in m["completed"]])) + if m["hold"]: + kind, name = m["hold"] + blocks.append(f"**Waiting on your decision:** {name} (approve or deny)." if kind == "gate" + else f"**Action needed now:** {name} (do it, then mark done).") + if m["human"]: + rows = [f"**Still needs you ({m['human_total']}):**"] + rows += [f"- {o['name']} — {'your approval' if o['gate'] else 'your action'}" + for o in m["human"]] + blocks.append("\n".join(rows)) + blocks.append("_Open Scout to continue the release._") + return "\n\n".join(blocks) # blank line between blocks survives markdown collapse + + +# ---- HTML digest (nice email UX) ------------------------------------------- +# Email-safe: inline styles + table layout (Outlook-friendly), no external CSS. + +def _esc(s: str) -> str: + return (str(s or "").replace("&", "&").replace("<", "<") + .replace(">", ">").replace('"', """)) + + +# Per-status pill styling (label, text color, background). +_PILL = { + "done": ("✓ Done", "#1a7f37", "#e6f4ea"), + "now": ("⚠ Needs you now", "#b42318", "#fef3f2"), + "blocked": ("⛔ Blocked — fix & rerun", "#b42318", "#fef3f2"), + "approval": ("Your approval", "#b54708", "#fffaeb"), + "confirm": ("Your confirmation", "#b54708", "#fffaeb"), + "action": ("Your action", "#b54708", "#fffaeb"), + "scout": ("Scout runs this", "#475467", "#f2f4f7"), + "auto": ("Automatic — pending", "#475467", "#f2f4f7"), + "in_flight": ("⏳ RC running — polling", "#475467", "#f2f4f7"), +} + + +def _pill(status: str) -> str: + label, fg, bg = _PILL.get(status, _PILL["auto"]) + return (f'{label}') + + +def notification_html(r: dict) -> str: + """HTML version of the daily phase digest. Returns "" under the exact same + silence rules as the other renderers (shares `_digest_model`). Presents EVERY + step in the active phase with a status pill, and flags what needs the owner now.""" + if _digest_model(r) is None: # same silence rules / dedup gate + return "" + rid = _esc(r.get("release_id", "?")) + ap = r.get("active_phase") or {} + phase_title = _esc(f"Phase {ap.get('num')}: {ap.get('name','')}") + done, total = ap.get("done", 0), ap.get("total", 0) + pct = round(100 * done / total) if total else 0 + steps = ap.get("steps", []) + + # The single item that needs the owner right now (the live hold), if any. + hold = r.get("gate") or r.get("action") + hold_name = _esc(hold["step_name"]) if hold else "" + hold_kind = "approve or deny" if r.get("gate") else "do it, then mark it done" + + # Rows: every step, with the active hold promoted to the "now" pill. + rows = [] + for s in steps: + st = "now" if s.get("now") else s.get("status", "auto") + name = _esc(s.get("name", "")) + star = (' ⚑' + if s.get("needs_owner") else "") + rows.append( + f'{name}{star}' + f'{_pill(st)}' + ) + rows_html = "\n".join(rows) + + attention = "" + if hold: + attention = ( + f'' + f'
' + f'⚑ Needs your attention: {hold_name} ' + f'— {hold_kind}.
' + ) + + return f"""\ +
+ + + + {attention} + + +
+
Release {rid}
+
{phase_title}
+
+
Progress: {done} of {total} steps done ({pct}%)
+
+
+
+
+ + + + {rows_html} +
TaskStatus
+
+
Items marked ⚑ need you. Open Scout to continue the release.
+
+
""" + diff --git a/release-agent/orchestrator/schedule.py b/release-agent/orchestrator/schedule.py new file mode 100644 index 00000000..326e4d57 --- /dev/null +++ b/release-agent/orchestrator/schedule.py @@ -0,0 +1,176 @@ +"""Schedule math for CCD-anchored phases — pure functions, no IO. + +The Code Complete Date (CCD) is the anchor the whole release hangs off of. +This module mirrors the logic of ADO pipeline 3038 "Code Complete Calendar +Checker" so the orchestrator resolves the *same* date the pipeline would: + + * Default : the 2nd Wednesday of the release month. + * Override: a full YYYY-MM-DD, but only if it belongs to the release month + (a stale cross-month override is ignored, exactly like the pipeline). + +Phases anchor to CCD via a spec like "CCD-7" (7 days before) or "CCD+1". +Kept IO-free so it stays deterministic and unit-testable; reading/writing the +pipeline variable lives in tools/checks.py, and CCD is stored on ReleaseState. +""" +from __future__ import annotations + +import calendar +import re +from datetime import date, datetime, time, timedelta +from typing import Optional + +# The release runs on the OWNER's wall clock, not the host's. On a UTC host, a bare +# date.today() rolls to the next day at UTC-midnight (evening the day before, Pacific), +# which opened phases — and fired timed comms — hours early. So "today"/"now" are +# computed in this zone unless a caller overrides it. Needs the `tzdata` package on +# Windows (no system IANA db); falls back to host-local if the zone can't be loaded. +DEFAULT_TZ = "America/Los_Angeles" + + +def get_tz(name: Optional[str] = None): + """A tzinfo for `name` (default DEFAULT_TZ), or None if it can't be loaded + (missing tzdata) — callers then fall back to host-local time.""" + try: + from zoneinfo import ZoneInfo + return ZoneInfo(name or DEFAULT_TZ) + except (ImportError, KeyError, ValueError): # no tzdata / unknown zone name + return None + + +def detect_local_tz() -> Optional[str]: + """The IANA name of the machine's local timezone (e.g. 'America/Los_Angeles'), or + None if it can't be determined. Captured at `init` (on the owner's interactive + machine) and persisted, so later headless automation runs — which may execute in a + UTC process context — use the OWNER's zone, not the host's.""" + try: + import tzlocal + return tzlocal.get_localzone_name() + except Exception: + return None + + +def now_local(tz=None) -> datetime: + """Timezone-aware 'now' in the release zone (default DEFAULT_TZ). Falls back to a + naive host-local now only if the zone can't be loaded.""" + z = tz if tz is not None else get_tz() + return datetime.now(z) if z is not None else datetime.now() + + +def parse_release_month(release_id: str) -> Tuple[int, int]: + """'2026-07' -> (2026, 7).""" + parts = release_id.split("-") + return int(parts[0]), int(parts[1]) + + +def second_wednesday(year: int, month: int) -> date: + """The 2nd Wednesday of the month (pipeline 3038's default rule).""" + weeks = calendar.monthcalendar(year, month) + wednesdays = [w[calendar.WEDNESDAY] for w in weeks if w[calendar.WEDNESDAY] != 0] + return date(year, month, wednesdays[1]) + + +def parse_date(s: Optional[str]) -> Optional[date]: + """Parse 'YYYY-MM-DD' -> date, or None if empty/invalid.""" + if not s or not str(s).strip(): + return None + try: + y, m, d = (int(x) for x in str(s).strip().split("-")) + return date(y, m, d) + except (ValueError, TypeError): + return None + + +def default_ccd(release_id: str) -> date: + """The canonical CCD for a release month: the 2nd Wednesday. This is the + source of truth — a differing pipeline override is treated as a *question* + to confirm, not a value to adopt silently.""" + year, month = parse_release_month(release_id) + return second_wednesday(year, month) + + +def pipeline_conflict(release_id: str, override: Optional[str], stored_ccd: Optional[str] = None): + """Return the pipeline override date IF it is a valid in-month date that + DIFFERS from our reference CCD — i.e. a divergence the user must resolve. + Otherwise None (override empty, cross-month, or already in agreement). + + The reference is `stored_ccd` when provided (what this release is anchored + to), else the 2nd-Wednesday default (used at init before anything is stored). + """ + od = parse_date(override) + if not od: + return None + year, month = parse_release_month(release_id) + if (od.year, od.month) != (year, month): + return None # month-scoped, like the pipeline + reference = parse_date(stored_ccd) or default_ccd(release_id) + return od if od != reference else None + + +_ANCHOR_RE = re.compile(r"^CCD\s*([+-]\s*\d+)?$", re.IGNORECASE) + + +def anchor_offset(spec: str) -> int: + """'CCD-7' -> -7, 'CCD+1' -> 1, 'CCD' -> 0.""" + m = _ANCHOR_RE.match((spec or "").strip()) + if not m: + raise ValueError(f"bad anchor spec: {spec!r} (expected e.g. 'CCD-7')") + grp = m.group(1) + return int(grp.replace(" ", "")) if grp else 0 + + +def anchor_date(ccd: date, spec: str) -> date: + """The calendar date a phase with `spec` opens, given CCD.""" + return ccd + timedelta(days=anchor_offset(spec)) + + +def today(tz=None) -> date: + """'Now' at date granularity, in the release timezone (default DEFAULT_TZ) — NOT + the host's, so a UTC host doesn't roll the date early. `--as-of` overrides this.""" + return now_local(tz).date() + + +def humanize_delta(days: int) -> str: + """'in 3 days' / 'today' / '2 days ago' — for countdowns.""" + if days == 0: + return "today" + if days == 1: + return "tomorrow" + if days == -1: + return "yesterday" + if days > 0: + return f"in {days} days" + return f"{-days} days ago" + + +def ccd_viability(ccd: date, as_of: date, open_spec: str = "CCD-7") -> dict: + """Is a CCD temporally viable, measured against the clock `as_of`? + + Reconciliation with the pipeline (pipeline_conflict) answers *which* date; + this answers whether that date is even runnable on the calendar. `open_spec` + is the earliest phase anchor (Phase 0 opens "CCD-7"), so the normal prep + window is open_spec..CCD. + + Returns: + days_to_ccd : (ccd - as_of).days (negative ⇒ CCD already past) + past : ccd < as_of (INVALID — can't code-complete in the past) + phase0_open : the CCD-7 date (when prep normally starts) + normal_window: len of a full prep window in days (e.g. 7 for CCD-7) + runway_days : prep days actually left = days from max(as_of, phase0_open)..ccd (>=0) + compressed : not past, but as_of is already inside the CCD-7 window + (runway_days < normal_window) ⇒ Phase 0 is squeezed — WARN, don't block + """ + days_to_ccd = (ccd - as_of).days + phase0_open = anchor_date(ccd, open_spec) + normal_window = -anchor_offset(open_spec) # "CCD-7" -> 7 + past = days_to_ccd < 0 + start = max(as_of, phase0_open) + runway_days = max((ccd - start).days, 0) + compressed = (not past) and runway_days < normal_window + return { + "days_to_ccd": days_to_ccd, + "past": past, + "phase0_open": phase0_open.isoformat(), + "normal_window": normal_window, + "runway_days": runway_days, + "compressed": compressed, + } diff --git a/release-agent/orchestrator/sim.py b/release-agent/orchestrator/sim.py new file mode 100644 index 00000000..79304ed0 --- /dev/null +++ b/release-agent/orchestrator/sim.py @@ -0,0 +1,401 @@ +"""Release simulator — drop into the middle of a release to validate a phase. + +WHY: testing a late phase from scratch is expensive (drive every earlier phase by +hand). Instead a *scenario* declares a target point; this module fast-forwards the +**real engine** to it in an isolated runs-root, so what you validate is exactly what +production would produce — no parallel reimplementation of the state machine. + +HOW (Option A + freeze): + * Earlier phases are fast-forwarded with `outcome: done` step-mocks (instant, offline) + and gate auto-approval — the engine's own transitions, just short-circuited. + * The TARGET phase runs for real: `data: live` exercises real `az` reads against real + runs; `data: mock` uses the scenario's fine-grained input mocks (offline, deterministic). + * Readiness is signed offline (auto items via `readiness.` pass-mocks, attest items + via gate.sign()). + * `--freeze` snapshots the engine-produced state to tests/fixtures/.json — a cache + that is *generated by* this driver, so it can never drift into a hand-maintained lie. + +A scenario is data, not code: + + name: mid_build_verify_open + release_id: 2026-08 + ccd: 2026-08-26 + ccd_source: confirmed # default: confirmed + as_of: CCD+1 # CCD-relative or absolute YYYY-MM-DD; default: ccd + target: {phase: build_verify, at: open} # at: open | gate | done + data: live # live | mock (default: mock) + approve_gates: [bash_done] # gates to auto-approve while fast-forwarding (see note) + mocks: # merged last (win); fine-grained inputs or outcome mocks + build_verify.checker_fired: { triggering: {...} } + seed: # direct ReleaseState field overrides (e.g. pipeline_runs) + owner_email: dev@example.com + +`at` semantics (target phase T): + open — every phase before T complete; positioned at T's entry (nothing in T run). + gate — T's non-gate steps run; halted at T's first gate (error if T has no gate). + done — T fully complete (its gate auto-approved); positioned at the next phase. +""" +from __future__ import annotations + +import os +from dataclasses import dataclass, field +from datetime import date +from typing import Optional + +import yaml + +from orchestrator import schedule +from orchestrator.state import ReleaseState +from orchestrator.engine import Orchestrator +from orchestrator import cli_common as C + +HERE = os.path.dirname(os.path.abspath(__file__)) +ROOT = os.path.dirname(HERE) # release-agent/ +SCENARIO_DIR = os.path.join(ROOT, "config", "scenarios") +FIXTURE_DIR = os.path.join(ROOT, "tests", "fixtures") +# A scenario seeds the REAL runs-root by default — the whole point is that the release +# skill then reads it as a normal release ("status" / "next" / "approve" just work). +# Any existing state at that id is backed up first (see _backup_existing). Pass an +# explicit runs_root to run_scenario() to target a throwaway sandbox instead. +DEFAULT_SEED_RUNS = C.DEFAULT_RUNS_ROOT + +_AT_MODES = ("open", "gate", "done") + + +# ---------------------------------------------------------------- scenario I/O +def scenario_path(name: str) -> str: + base = name if name.endswith((".yaml", ".yml")) else f"{name}.yaml" + return base if os.path.isabs(base) else os.path.join(SCENARIO_DIR, base) + + +def load_scenario(name: str) -> dict: + p = scenario_path(name) + if not os.path.exists(p): + raise FileNotFoundError(f"no scenario '{name}' (looked in {p})") + with open(p, "r", encoding="utf-8") as fh: + sc = yaml.safe_load(fh) or {} + sc.setdefault("name", os.path.splitext(os.path.basename(p))[0]) + return sc + + +def list_scenarios() -> list: + if not os.path.isdir(SCENARIO_DIR): + return [] + return sorted(os.path.splitext(f)[0] for f in os.listdir(SCENARIO_DIR) + if f.endswith((".yaml", ".yml"))) + + +# ---------------------------------------------------------------- result model +@dataclass +class SimResult: + scenario: str + release_id: str + ccd: Optional[str] + as_of: str + target: dict + reached: bool # did we land exactly at the requested target? + stop_kind: str # 'open' | 'gate' | 'done' | 'blocked' | 'halted' | 'scheduled' | 'complete' + stop_message: str + steps_forwarded: int + gates_approved: list = field(default_factory=list) + problems: list = field(default_factory=list) + frozen_to: Optional[str] = None + backed_up_to: Optional[str] = None + runs_root: Optional[str] = None + state: Optional[ReleaseState] = None + + +# ---------------------------------------------------------------- profile build +def _config_phase_ids(config: dict) -> list: + return [p["id"] for p in config["phases"]] + + +def _phase_by_id(config: dict, pid: str) -> Optional[dict]: + return next((p for p in config["phases"] if p["id"] == pid), None) + + +def build_mock_profile(config: dict, target_phase: str, data: str, mocks: dict) -> dict: + """Compose the engine mock profile for a scenario. + + * Every non-gate step in a phase BEFORE the target gets `outcome: done` so the + fast-forward is instant and offline (gates are approved by the driver instead). + * The target phase runs for real: under `data: mock` any target step WITHOUT a + scenario mock also gets `outcome: done` so gate/done mode can still advance; + under `data: live` the target gets no base mock (real logic hits az). + * Every auto readiness item gets `readiness.: pass` so the entry gate signs + offline. Scenario `mocks` win over all of the above (merged last). + """ + ids = _config_phase_ids(config) + if target_phase not in ids: + raise ValueError(f"target phase '{target_phase}' not in config (have {ids})") + tgt_idx = ids.index(target_phase) + prof: dict = {} + + for i, phase in enumerate(config["phases"]): + before = i < tgt_idx + is_target = i == tgt_idx + for step in phase["steps"]: + if step.get("gate"): + continue # gates are never step-mockable + key = f"{phase['id']}.{step['id']}" + if before: + prof[key] = {"outcome": "done", "note": "[sim] fast-forwarded"} + elif is_target and data != "live": + prof[key] = {"outcome": "done", "note": "[sim] target auto-advanced"} + + # Readiness auto items → pass offline (covers python-auto AND scout-assisted auto). + for it in _readiness_items(config): + if it.get("verify") == "auto": + prof[f"readiness.{it['id']}"] = {"outcome": "pass", "detail": "[sim] readiness auto-pass"} + + prof.update(mocks or {}) # scenario overrides win + return prof + + +def _readiness_items(config: dict) -> list: + path = os.path.join(ROOT, "config", "readiness.yaml") + if not os.path.exists(path): + return [] + with open(path, "r", encoding="utf-8") as fh: + return (yaml.safe_load(fh) or {}).get("items", []) or [] + + +# ---------------------------------------------------------------- clock +def resolve_as_of(scenario: dict, ccd: Optional[date]) -> Optional[date]: + """Scenario `as_of`: absolute YYYY-MM-DD, or CCD-relative ('CCD+1', 'CCD-7', 'CCD'). + Defaults to CCD when omitted so the target phase's window is open.""" + spec = scenario.get("as_of") + if not spec: + return ccd + spec = str(spec).strip() + if spec.upper().startswith("CCD"): + if ccd is None: + raise ValueError("as_of is CCD-relative but the scenario has no ccd") + return schedule.anchor_date(ccd, spec) + d = schedule.parse_date(spec) + if d is None: + raise ValueError(f"bad as_of {spec!r} (expected YYYY-MM-DD or CCD±N)") + return d + + +# ---------------------------------------------------------------- readiness sign +def _sign_readiness(orch: Orchestrator) -> None: + """Sign the entry gate offline. verify() clears auto items via the readiness + pass-mocks in the profile; sign() attests every attest item. If readiness has no + config, sign() flips it directly.""" + attest_ids = [it["id"] for it in (orch.gate.config or {}).get("items", []) + if it.get("verify", "attest") == "attest"] + orch.gate.sign(item_ids=attest_ids, note="[sim] fast-forward attest") + + +# ---------------------------------------------------------------- driver +def _is_auto_agent(step: dict) -> bool: + return (step.get("owner", "agent") == "agent" and step.get("source") != "scout" + and not step.get("attest") and not step.get("gate")) + + +def _fast_forward(orch: Orchestrator, config: dict, target_phase: str, mode: str, + approve_gates: set, max_iter: int = 2000) -> dict: + """Drive the REAL engine until it reaches the target. Returns a stop dict + {kind, message, forwarded, approved, problems}. Never reimplements transitions — + it only reacts to what step_once() reports.""" + ids = _config_phase_ids(config) + tgt_idx = ids.index(target_phase) + approved, problems = [], [] + forwarded = 0 + attempted: set = set() # threaded into step_once so a re-blocking PARALLEL step + # can't be re-run every iteration (else it spins to the cap) + + for _ in range(max_iter): + cur_id = orch.current_phase_id() + if cur_id is None: + return _stop("complete", "Release complete.", forwarded, approved, problems) + ci = ids.index(cur_id) + + # Arrived at / past the target — decide whether to stop. + if ci > tgt_idx: + # In 'gate' mode we should have halted AT the target's gate; passing it means + # the target had no gate (or none was reached) — surface that, don't pretend. + if mode == "gate": + problems.append(f"target '{target_phase}' has no gate (or it was never reached)") + return _stop("done", f"Past target — now at '{cur_id}'.", forwarded, approved, problems) + if ci == tgt_idx and mode == "open": + return _stop("open", f"Positioned at entry of '{target_phase}'.", + forwarded, approved, problems) + + act = orch.step_once(attempted) + + if act.kind == "ran": + # In a PARALLEL phase a blocked auto step returns kind 'ran' (block_holds= + # False), not 'reminder'. Detect that here so a genuine block is surfaced + # instead of being re-run to the iteration cap. + if act.phase and act.step: + stp = orch.state.get_step(act.phase, act.step) + step = _find_step(config, act.phase, act.step) + if stp is not None and stp.status == "blocked" and _is_auto_agent(step or {}): + problems.append(f"blocked: {act.phase}.{act.step} — {stp.note}") + return _stop("blocked", f"{act.phase}.{act.step} blocked: {stp.note}", + forwarded, approved, problems) + forwarded += 1 + continue + if act.kind == "complete": + return _stop("complete", act.message, forwarded, approved, problems) + if act.kind in ("readiness", "blocked", "halted", "scheduled"): + # A hard stop the scenario didn't expect (e.g. as_of before the phase + # window ⇒ scheduled; unsatisfied entry gate ⇒ readiness). + problems.append(f"{act.kind}: {act.message}") + return _stop(act.kind, act.message, forwarded, approved, problems) + + if act.kind == "gate": + step_id = act.step + at_target_gate = (act.phase == target_phase) + if mode == "gate" and at_target_gate: + return _stop("gate", act.message, forwarded, approved, problems) + # Approve if allowed: earlier-phase gates always; target gate only in + # 'done' mode or when explicitly listed. + if (act.phase != target_phase) or mode == "done" or step_id in approve_gates: + orch.approve_gate("[sim] auto-approved") + approved.append(f"{act.phase}.{step_id}") + forwarded += 1 + continue + problems.append(f"gate not approved: {act.phase}.{step_id} " + f"(add it to approve_gates or use at: done)") + return _stop("gate", act.message, forwarded, approved, problems) + + if act.kind == "reminder": + step = _find_step(config, act.phase, act.step) + stp = orch.state.get_step(act.phase, act.step) + # A blocked AUTO agent step in the target is a genuine finding — stop and + # surface it (that's exactly what live validation is meant to catch). + if stp.status == "blocked" and _is_auto_agent(step or {}): + problems.append(f"blocked: {act.phase}.{act.step} — {stp.note}") + return _stop("blocked", f"{act.phase}.{act.step} blocked: {stp.note}", + forwarded, approved, problems) + # Otherwise it's an expected scout/attest/human hold — clear it. + if (step or {}).get("source") == "scout": + orch.record_scout_step(act.phase, act.step, "pass", "[sim] auto-pass") + else: + orch.complete_step(act.phase, act.step, "[sim] auto-complete") + forwarded += 1 + continue + + # idle / unknown — nothing to do. + return _stop(act.kind or "idle", act.message, forwarded, approved, problems) + + problems.append(f"fast-forward exceeded {max_iter} iterations (possible loop)") + return _stop("halted", "iteration cap hit", forwarded, approved, problems) + + +def _stop(kind, message, forwarded, approved, problems) -> dict: + return {"kind": kind, "message": message, "forwarded": forwarded, + "approved": approved, "problems": problems} + + +def _find_step(config: dict, phase_id: str, step_id: str) -> Optional[dict]: + ph = _phase_by_id(config, phase_id) + if not ph: + return None + return next((s for s in ph["steps"] if s["id"] == step_id), None) + + +# ---------------------------------------------------------------- entry point +def run_scenario(name_or_dict, runs_root: str = None, config_path: str = None, + freeze: bool = False, save: bool = True) -> SimResult: + """Fast-forward the engine to a scenario's target and SEED that state as the real + release, returning a SimResult. + + `name_or_dict` is a scenario name (config/scenarios/.yaml) or a scenario + dict. State is written under `runs_root`/ — default is the REAL runs-root + so the release skill drives it as a normal release. Any existing state at that id is + backed up first. `freeze` also snapshots the produced state to tests/fixtures/.json. + """ + sc = name_or_dict if isinstance(name_or_dict, dict) else load_scenario(name_or_dict) + config_path = config_path or C.DEFAULT_CONFIG + runs_root = runs_root or DEFAULT_SEED_RUNS + + with open(config_path, "r", encoding="utf-8") as fh: + config = yaml.safe_load(fh) + + release_id = sc.get("release_id") + if not release_id: + raise ValueError("scenario missing 'release_id'") + target = sc.get("target") or {} + tphase = target.get("phase") + mode = (target.get("at") or "open").lower() + if not tphase: + raise ValueError("scenario target missing 'phase'") + if mode not in _AT_MODES: + raise ValueError(f"target.at must be one of {_AT_MODES} (got {mode!r})") + data = (sc.get("data") or "mock").lower() + + # YAML may parse an unquoted `ccd: 2026-08-26` as a date object — normalize to an + # ISO string so it serializes and the schedule helpers get their expected type. + ccd_raw = sc.get("ccd") + ccd_str = ccd_raw.isoformat() if isinstance(ccd_raw, date) else ccd_raw + ccd = schedule.parse_date(ccd_str) + as_of = resolve_as_of(sc, ccd) + + # Seed state (+ any direct field overrides) and build the mock profile. + st = ReleaseState(release_id=release_id, ccd=ccd_str, + ccd_source=sc.get("ccd_source", "confirmed"), + owner_email=sc.get("owner_email"), owner_name=sc.get("owner_name")) + for k, v in (sc.get("seed") or {}).items(): + if hasattr(st, k): + setattr(st, k, v) + + profile = build_mock_profile(config, tphase, data, sc.get("mocks") or {}) + orch = Orchestrator(config_path, st, as_of=as_of, mocks=profile) + + _sign_readiness(orch) + + approve = set(sc.get("approve_gates") or []) + stop = _fast_forward(orch, config, tphase, mode, approve) + + # In 'open' mode we break BEFORE the engine acts on the target, so its cached + # cursor still points at the last completed phase. Sync it to the derived target + # so status' "you are here" marker is correct (position itself is always derived). + if stop["kind"] == "open": + st.current_phase = tphase + tp = _phase_by_id(config, tphase) + first = next((s for s in (tp or {}).get("steps", []) + if not st.is_done(tphase, s["id"])), None) + st.current_step = first["id"] if first else None + st.status = "running" + + reached = stop["kind"] == mode + res = SimResult( + scenario=sc.get("name", "scenario"), release_id=release_id, + ccd=ccd_str, as_of=as_of.isoformat() if as_of else "", + target={"phase": tphase, "at": mode}, reached=reached, + stop_kind=stop["kind"], stop_message=stop["message"], + steps_forwarded=stop["forwarded"], gates_approved=stop["approved"], + problems=stop["problems"], runs_root=runs_root, state=st) + + if save: + res.backed_up_to = _backup_existing(runs_root, release_id) + C.save_state(st, runs_root, release_id) + if freeze: + res.frozen_to = _freeze(sc.get("name", release_id), st) + return res + + +def _backup_existing(runs_root: str, release_id: str) -> Optional[str]: + """If a state file already exists at this id, copy it aside before the seed + overwrites it (so a real release is never lost to a seed). Returns the backup path + or None when there was nothing to back up.""" + import shutil + from datetime import datetime + src = C.state_path(runs_root, release_id) + if not os.path.exists(src): + return None + ts = datetime.now().strftime("%Y%m%d-%H%M%S") + dst = os.path.join(os.path.dirname(src), f"release-state.pre-seed-{ts}.json") + shutil.copy2(src, dst) + return dst + + +def _freeze(name: str, st: ReleaseState) -> str: + os.makedirs(FIXTURE_DIR, exist_ok=True) + path = os.path.join(FIXTURE_DIR, f"{name}.json") + st.save(path) + return path diff --git a/release-agent/orchestrator/state.py b/release-agent/orchestrator/state.py new file mode 100644 index 00000000..9b6feb54 --- /dev/null +++ b/release-agent/orchestrator/state.py @@ -0,0 +1,134 @@ +"""Release Orchestrator — run-state model (X5). + +Two kinds of state, per the architecture: + * DERIVED : recomputed from systems of record (ADO/Git/Play Console/ADX). Never stored here. + * PERSISTED: decisions/intent, step completion, pending human actions, notes. + Stored in release-state.json (the Release State Record). + +This module owns ONLY the persisted state. Reconcile-on-resume (deriving live +state) is a separate concern handled by tools/reconcile.py (stubbed for now). +""" +from __future__ import annotations +import json +import os +from dataclasses import dataclass, field, asdict, fields +from datetime import datetime, timezone +from typing import Optional + + +SCHEMA_VERSION = 1 + + +def _now() -> str: + return datetime.now(timezone.utc).isoformat() + + +@dataclass +class StepState: + """Persisted state for a single step.""" + status: str = "pending" # pending | done | skipped | blocked | in_flight + completed_at: Optional[str] = None + note: Optional[str] = None + by: Optional[str] = None # 'agent' (stub) or 'human' + links: list = field(default_factory=list) # [{name, url}] — durable refs (wiki page, CG alerts) + data: dict = field(default_factory=dict) # step-private scratch (e.g. localization build id/start) + + +@dataclass +class GateDecision: + """A recorded human decision at a gate (audit trail).""" + step: str + decision: str # approved | denied | held + at: str + by: str = "human" + comment: Optional[str] = None + + +@dataclass +class ReleaseState: + """The Release State Record — one per monthly release.""" + schema_version: int = SCHEMA_VERSION + release_id: str = "" # e.g. 2026-07 + created_at: str = field(default_factory=_now) + updated_at: str = field(default_factory=_now) + # Release owner — the engineer running this release (release metadata). The + # push reminders email this address; resolved from the signed-in user at init. + owner_email: Optional[str] = None + owner_name: Optional[str] = None + # The owner's IANA timezone (e.g. 'America/Los_Angeles'), captured at init from the + # owner's machine. Phase due-ness + every fire_at_local are evaluated on THIS zone, + # so a headless automation running in a UTC process still uses the owner's clock. + timezone: Optional[str] = None + # Code Complete Date — the anchor the phases hang off of (orchestrator's truth, + # seeded from / written back to pipeline 3038). ccd is 'YYYY-MM-DD'. + ccd: Optional[str] = None + ccd_source: Optional[str] = None # 'default' (2nd Wed) | 'override' | 'manual' + ccd_conflict: Optional[str] = None # a pipeline override date that DIFFERS from ccd (unresolved) + skip_release: bool = False # mirrors the pipeline 'skipRelease' switch (display) + # readiness entry gate (must be signed before Phase 0) + readiness_signed: bool = False + readiness_signed_at: Optional[str] = None + readiness_items: dict = field(default_factory=dict) # item_id -> {status,...} + blocked: bool = False # an item was declared unsatisfiable + blocked_items: list = field(default_factory=list) + # manual overrides (human-driven transitions, §7.1) + halted: bool = False # emergency hold + halt_reason: Optional[str] = None + # cursor + current_phase: Optional[str] = None + current_step: Optional[str] = None + status: str = "not_started" # not_started | running | scheduled | awaiting_action | holding_gate | halted | blocked | complete + # persisted detail + steps: dict = field(default_factory=dict) # "phase.step" -> StepState (as dict) + gate_decisions: list = field(default_factory=list) + pending_human: list = field(default_factory=list) # outstanding human actions + last_notified_date: Optional[str] = None # YYYY-MM-DD of the last daily digest sent + # Phase-2 release-pipeline runs — the RECORD of what verification resolved, reused by + # the RC report + gate (no re-discovery). Because a re-triggered 'Trigger RC Testing' + # stage spawns NEW MRWP runs, these are re-resolved (newest wins) — not a fixed cache. + # Nested schema: + # { checker: {run_id, when, resolved_at}, + # orchestrator: {run_id, versions:{Common,Msal,Broker}, parked, resolved_at}, + # rcs: [ {rc, ecs:{run_id,id_source,complete,ran,total,failed_stages, + # yellow_stages,never_ran,tests,failed_suites,resolved_at}, + # local:{...same...}, resolved_at} ] } + # There is exactly ONE checker + ONE orchestrator, but MULTIPLE RC iterations (each a + # re-trigger of RC Testing spawns a new ecs/local pair). The LATEST RC is rcs[-1] — the + # report + gate always use it. A per-provider id change appends a new rc entry. + pipeline_runs: dict = field(default_factory=dict) + notes: list = field(default_factory=list) + + # ---- persistence ---- + @classmethod + def load(cls, path: str) -> "ReleaseState": + with open(path, "r", encoding="utf-8") as fh: + data = json.load(fh) + # Drop unknown keys: this loader runs in an unattended automation, so a + # hand-edited or forward-version state file must never hard-crash it. Only keys + # matching a declared field are applied. + known = {f.name for f in fields(cls)} + obj = cls(**{k: v for k, v in data.items() if k in known}) + return obj + + def save(self, path: str) -> None: + self.updated_at = _now() + os.makedirs(os.path.dirname(path), exist_ok=True) + tmp = path + ".tmp" + with open(tmp, "w", encoding="utf-8") as fh: + json.dump(asdict(self), fh, indent=2) + os.replace(tmp, path) + + # ---- step helpers ---- + @staticmethod + def key(phase: str, step: str) -> str: + return f"{phase}.{step}" + + def get_step(self, phase: str, step: str) -> StepState: + raw = self.steps.get(self.key(phase, step)) + return StepState(**raw) if raw else StepState() + + def set_step(self, phase: str, step: str, state: StepState) -> None: + self.steps[self.key(phase, step)] = asdict(state) + + def is_done(self, phase: str, step: str) -> bool: + return self.get_step(phase, step).status in ("done", "skipped") diff --git a/release-agent/orchestrator/status_views.py b/release-agent/orchestrator/status_views.py new file mode 100644 index 00000000..ce20b6ab --- /dev/null +++ b/release-agent/orchestrator/status_views.py @@ -0,0 +1,279 @@ +"""Status view-model builder — the presentation half of the Orchestrator. + +Extracted from engine.py (which owns the state machine) so the report-model builder is a +separate responsibility. This is a MIXIN on Orchestrator: the methods read engine +internals via self (self.state, self.config, self._phase_due, ...) and return the plain +dict that render.py turns into a view. Behaviour is identical to the in-engine version. +""" +from __future__ import annotations + +from typing import Optional + + +class StatusViewMixin: + def _active_phase_report(self) -> Optional[dict]: + """The first incomplete included phase, with its outstanding steps and + whether its time-window is open (due). This is what the daily phase + notification reports on — independent of state.current_phase (which is + only set once the release has been advanced).""" + for phase in self.config["phases"]: + if not self._phase_included(phase): + continue + steps = phase["steps"] + done = sum(1 for s in steps if self.state.is_done(phase["id"], s["id"])) + if done == len(steps): + continue # phase complete — look at the next one + outstanding = [ + {"id": s["id"], "name": s["name"], "gate": bool(s.get("gate")), + "reminder": self._is_reminder(s), "owner": s.get("owner", "agent")} + for s in steps if not self.state.is_done(phase["id"], s["id"]) + ] + completed = [s["name"] for s in steps + if self.state.is_done(phase["id"], s["id"])] + cur = self.state.current_step + steps_view = [] + for s in steps: + sid = s["id"] + stp = self.state.get_step(phase["id"], sid) + s_done = self.state.is_done(phase["id"], sid) + s_blocked = stp.status == "blocked" + s_inflight = stp.status == "in_flight" + is_gate = bool(s.get("gate")) + is_rem = self._is_reminder(s) + is_scout = s.get("source") == "scout" + is_attest = bool(s.get("attest")) + if s_done: + status = "done" + elif s_blocked: + status = "blocked" + elif s_inflight: + status = "in_flight" # pipeline run still executing — Scout polling + elif is_gate: + status = "approval" + elif is_attest: + status = "confirm" + elif is_rem: + status = "action" # a human to-do — the user must act + elif is_scout: + status = "scout" # Scout runs it automatically (scrape/send via MCP) + else: + status = "auto" + # needs_owner = a genuine USER task. A pending scout step is Scout's + # automatic work (not the user's) until it BLOCKS (s_blocked), so it is + # NOT flagged — only gates, reminders, attests, and blocks are. + needs = bool((is_gate or is_rem or is_attest or s_blocked) and not s_done) + steps_view.append({ + "id": sid, "name": s["name"], "status": status, + "needs_owner": needs, + "time_ready": self._step_time_ready(phase, s), # False = waits for its fire_at_local + "note": stp.note, # agent result / block reason / detail + "links": list(getattr(stp, "links", None) or []), # durable refs to items evaluated + "now": bool(sid == cur and not s_done and (is_gate or is_rem or is_attest or s_blocked)), + }) + opens = self._phase_anchor_date(phase) + return { + "id": phase["id"], "name": phase["name"], + "num": phase.get("checklist_phase"), + "show_pipeline_runs": bool(phase.get("show_pipeline_runs")), + "done": done, "total": len(steps), + "due": self._phase_due(phase), "started": done > 0, + "opens": opens.isoformat() if opens else None, + "opens_in_days": (opens - self.as_of).days if opens else None, + "outstanding": outstanding, + "completed": completed, + "steps": steps_view, + } + return None + + def _phase_map(self): + """Build the phase overview + running totals. Returns + (phases, total, done, current_phase_name, current_phase_obj, current_step_name).""" + phases = [] + total = done = 0 + current_phase_name = current_step_name = None + current_phase_obj = None + # The authoritative CURRENT phase is the FRONTIER — the first included phase with + # incomplete steps — NOT merely "any phase with progress". Deriving it here (rather + # than trusting p_done > 0) means exactly ONE phase shows as current, even when an + # upstream reopen left stale progress in a later phase (which would otherwise render + # as a confusing second "in progress" phase). + frontier = self._current_phase() + frontier_id = frontier["id"] if frontier else None + for idx, phase in enumerate(self.config["phases"]): + if not self._phase_included(phase): + continue + p_total = len(phase["steps"]) + p_done = sum(1 for s in phase["steps"] if self.state.is_done(phase["id"], s["id"])) + total += p_total + done += p_done + is_current = phase["id"] == frontier_id + due = self._phase_due(phase) + opens = self._phase_anchor_date(phase) + if p_total and p_done == p_total: + state = "done" + elif not due and p_done == 0: + state = "scheduled" # not open yet — even if it's the frontier + elif is_current: + state = "current" + else: + state = "pending" # not the frontier — a later phase, even if it has + # stale partial progress, is not "in progress" now + if is_current: + current_phase_name = phase["name"] + current_phase_obj = phase + phases.append({ + "id": phase["id"], "name": phase["name"], + "num": phase.get("checklist_phase", idx), + "done": p_done, "total": p_total, "state": state, + "current": is_current, + "anchor": phase.get("anchor"), + "opens": opens.isoformat() if opens else None, + "opens_in_days": (opens - self.as_of).days if opens else None, + }) + for s in phase["steps"]: + if s["id"] == self.state.current_step and phase["id"] == frontier_id: + current_step_name = s["name"] + return phases, total, done, current_phase_name, current_phase_obj, current_step_name + + def _current_steps(self, current_phase_obj) -> list: + """The current phase's steps, each tagged with a display state.""" + if not current_phase_obj: + return [] + phase_due = self._phase_due(current_phase_obj) + out = [] + for s in current_phase_obj["steps"]: + rec = self.state.steps.get(self.state.key(current_phase_obj["id"], s["id"]), {}) or {} + is_scout = s.get("source") == "scout" and not s.get("attest") + if rec.get("status") == "skipped": + s_state = "skipped" + elif self.state.is_done(current_phase_obj["id"], s["id"]): + s_state = "done" + elif rec.get("status") == "blocked": + s_state = "blocked" # a step hit a real problem — needs the owner + elif rec.get("status") == "in_flight": + s_state = "in_flight" # pipeline run still executing — Scout is polling + elif s["id"] == self.state.current_step and self.state.status == "holding_gate": + s_state = "gate" + elif is_scout: + s_state = "scout" # Scout's automatic work — never a user "do this" + elif s["id"] == self.state.current_step and self.state.status == "awaiting_action": + s_state = "reminder" + elif not phase_due: + s_state = "scheduled" + elif s.get("owner") == "human" or s.get("attest") or self._is_reminder(s): + s_state = "pending" # a human step queued behind a dependency + else: + s_state = "auto" # an agent step Scout runs itself — no user action + out.append({ + "id": s["id"], "name": s["name"], + "gate": bool(s.get("gate")), + "reminder": self._is_reminder(s), + "owner": s.get("owner", "agent"), + "state": s_state, + "note": rec.get("note"), # agent result / block reason / detail + "links": rec.get("links") or [], # durable refs (wiki page, CG alerts) + }) + return out + + def _hold_view(self, phase_name, step_name) -> dict: + """Detail of the current hold (gate or action-needed) — same shape for both.""" + return { + "phase": self.state.current_phase, + "phase_name": phase_name, + "step": self.state.current_step, + "step_name": step_name, + } + + def _scheduled_view(self) -> Optional[dict]: + """The phase we're waiting on the clock for — derived from the first + incomplete phase's due-ness, so `status` shows it even before `next`.""" + first_incomplete = next( + (p for p in self.config["phases"] + if self._phase_included(p) + and not all(self.state.is_done(p["id"], s["id"]) for s in p["steps"])), + None) + if (first_incomplete is None or self._phase_due(first_incomplete) + or self.state.status in ("complete", "halted", "blocked")): + return None + opens = self._phase_anchor_date(first_incomplete) + return { + "phase": first_incomplete["id"], + "phase_name": first_incomplete["name"], + "opens": opens.isoformat() if opens else None, + "opens_in_days": (opens - self.as_of).days if opens else None, + } + + def status_report(self) -> dict: + """Structured status — presentation layer (render.py) turns this into a view. + Deterministic; no formatting baked in. Assembled from focused builders: + phase map, current-phase steps, current hold, scheduled window, active phase.""" + (phases, total, done, current_phase_name, + current_phase_obj, current_step_name) = self._phase_map() + current_steps = self._current_steps(current_phase_obj) + + gate = action = None + if self.state.status == "holding_gate" and self.state.current_phase: + gate = self._hold_view(current_phase_name, current_step_name) + elif self.state.status == "awaiting_action" and self.state.current_phase: + # A scout step is the SKILL's work (run via step-action), not a USER action — + # never surface it as `action` (which the digest reads as "Action needed now"). + phase = next((p for p in self.config["phases"] + if p["id"] == self.state.current_phase), None) + cur = next((s for s in (phase or {}).get("steps", []) + if s["id"] == self.state.current_step), None) if self.state.current_step else None + is_scout_focus = bool(cur and cur.get("source") == "scout" and not cur.get("attest")) + cur_blocked = (self.state.get_step(self.state.current_phase, self.state.current_step).status + == "blocked") if self.state.current_step else False + if not is_scout_focus or cur_blocked: + action = self._hold_view(current_phase_name, current_step_name) + scheduled = self._scheduled_view() + + chk = self.gate.checklist() + active_phase = self._active_phase_report() + # Scout steps ready for the SKILL to execute (perform the MCP send/scrape, then + # record-step). They are NOT user holds — the skill drains these itself; only if + # a scout step records 'attention' does it become a blocked user task. + # GATED ON PHASE DUE: a phase that hasn't reached its anchor (e.g. Code Complete + # Day before the CCD) must expose NO pending scout work — otherwise the autonomous + # automation would drain those steps early, running CCD-day comms ahead of the CCD. + # ALSO GATED ON fire_at_local: a timed step (e.g. the 09:00 CCD comms) is excluded + # until its wall-clock time arrives, so the every-hour worker doesn't fire it early + # — its dedicated cron automation runs it at the pinned time. + scout_pending = ([s["id"] for s in (active_phase or {}).get("steps", []) + if s.get("status") == "scout" and s.get("time_ready", True)] + if (active_phase and active_phase.get("due")) else []) + return { + "release_id": self.state.release_id, + "status": self.state.status, + "owner_email": self.state.owner_email, + "owner_name": self.state.owner_name, + "ccd": self.state.ccd, + "ccd_source": self.state.ccd_source, + "ccd_conflict": self.state.ccd_conflict, + "as_of": self.as_of.isoformat(), + "skip_release": self.state.skip_release, + "readiness_signed": self.state.readiness_signed, + "readiness_pending": [i["id"] for i in chk["items"] if not i["satisfied"]], + "blocked": self.state.blocked, + "blocked_items": list(self.state.blocked_items), + "blocked_message": chk.get("blocked_message", ""), + "halted": self.state.halted, + "halt_reason": self.state.halt_reason, + "done": done, "total": total, + "percent": round(100 * done / total) if total else 0, + "phases": phases, + "current_phase": self.state.current_phase, + "current_phase_name": current_phase_name, + "current_step": self.state.current_step, + "current_step_name": current_step_name, + "current_steps": current_steps, + "gate": gate, + "action": action, + "scheduled": scheduled, + "active_phase": active_phase, + "scout_pending": scout_pending, + "pending_human": list(self.state.pending_human), + "gate_decisions": len(self.state.gate_decisions), + "pipeline_runs": dict(getattr(self.state, "pipeline_runs", {}) or {}), + "updated_at": self.state.updated_at, + } diff --git a/release-agent/phases/__init__.py b/release-agent/phases/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/release-agent/phases/readiness_verifiers.py b/release-agent/phases/readiness_verifiers.py new file mode 100644 index 00000000..21f0630e --- /dev/null +++ b/release-agent/phases/readiness_verifiers.py @@ -0,0 +1,87 @@ +"""Readiness AUTO verifiers. + +An auto verifier must FULLY verify its item — it returns pass or fail, never a +half-measure. If something cannot be fully proven programmatically, it must NOT +be an auto item (make it an attest item in readiness.yaml instead). + +Contract: verify(item) -> VerifyResult(status, message) + status: "pass" | "fail" +""" +from __future__ import annotations +from dataclasses import dataclass +import os + + +@dataclass +class VerifyResult: + status: str # "pass" | "fail" + message: str + details: list = None # optional per-check breakdown: [{name, url, ok, detail}] + + @property + def ok(self) -> bool: + return self.status == "pass" + + +def verify_build_defs(item: dict) -> VerifyResult: + """Confirm the engineer can access every configured ADO build definition, + using `az pipelines build definition show`. Fully verified access — pass/fail. + Returns per-check details (name, url, ok) so the display can link each one.""" + from tools.checks import check_ado_build_def + + checks = [c for c in item.get("checks", []) if c.get("type") == "ado_build_def"] + if not checks: + return VerifyResult("fail", "no build definitions configured to check", []) + details, any_fail = [], False + for c in checks: + r = check_ado_build_def(c["org"], c["project"], c["id"]) + details.append({"name": c.get("name", str(c["id"])), "url": c.get("url"), + "ok": r.ok, "detail": r.detail}) + if not r.ok: + any_fail = True + msg = "; ".join(f"{'OK' if d['ok'] else 'FAIL'} {d['name']}" for d in details) + return VerifyResult("fail" if any_fail else "pass", msg, details) + + +def verify_mcp_servers(item: dict) -> VerifyResult: + """Confirm every MCP server the skill needs (ICM, Kusto/ADX, Teams) is registered + in Scout's config. Reuses the infra preflight in READ-ONLY mode (register=False) + against config/requirements.yaml — the single source of truth for MCP deps. + Fully verified: pass only if all are `present`, else fail listing the missing + ones (fix = run bootstrap / `infra --register`, then RESTART Scout).""" + from orchestrator import infra + + root = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + req_path = os.path.join(root, "config", "requirements.yaml") + try: + req = infra.load_requirements(req_path) + except OSError as e: + return VerifyResult("fail", f"cannot read requirements.yaml: {e}", []) + results = infra.ensure_mcp_servers(req, register=False) + if not results: + return VerifyResult("fail", "no MCP servers configured to check", []) + details, missing = [], [] + for r in results: + ok = r.get("status") == "present" + details.append({"name": r.get("name", r.get("scout_key")), "url": None, + "ok": ok, "detail": r.get("detail", "")}) + if not ok: + missing.append(r.get("scout_key") or r.get("id")) + if missing: + return VerifyResult( + "fail", + "not registered: " + ", ".join(missing) + + " — run bootstrap (or `python -m orchestrator.cli infra`), then RESTART Scout", + details) + return VerifyResult( + "pass", "registered: " + ", ".join(r.get("scout_key", "?") for r in results), details) + + +REGISTRY = { + "build_defs": verify_build_defs, + "mcp_servers": verify_mcp_servers, +} + + +def get_verifier(verifier_id: str): + return REGISTRY.get(verifier_id) diff --git a/release-agent/phases/stub_runner.py b/release-agent/phases/stub_runner.py new file mode 100644 index 00000000..8dc43d08 --- /dev/null +++ b/release-agent/phases/stub_runner.py @@ -0,0 +1,49 @@ +"""Stub phase runner — the fallback for steps that don't yet have a real agent. + +Phase 0 has real agents (phases/agents/preflight.py); every other step still +maps to `agent: stub`. The stub does NOT perform the real action — it returns a +mock result telling the conductor what a human would do, so the end-to-end flow +can be driven and tested before each real agent exists. + +When a real phase agent is built, it implements the same contract: + run(phase_id, step, state) -> StepResult +and replaces the stub for that step's `agent` id (registered in phases/agents/). +""" +from __future__ import annotations +from dataclasses import dataclass + + +@dataclass +class StepResult: + ok: bool + action: str # human-readable description of what happened / should happen + by: str # 'agent' (stub did it) or 'human' (needs a person) + links: list = None # optional [{name, url}] durable refs (wiki page, CG alerts) + in_flight: bool = False # True → the work is still RUNNING (poll again); not done, not blocked + poll_in_min: int = 30 # re-check cadence when in_flight + + +def run_stub(phase_id: str, step: dict, state=None) -> StepResult: + """Mock action for a step. Agent-owned steps are 'auto-completed' (mock); + human-owned steps return a reminder that a person must act.""" + owner = step.get("owner", "agent") + name = step.get("name", step["id"]) + if owner == "human": + return StepResult( + ok=True, + action=f"[STUB] Reminder — a human must: {name}", + by="human", + ) + return StepResult( + ok=True, + action=f"[STUB] Would run agent for: {name} (mock success)", + by="agent", + ) + + +# Registry: maps an agent id -> runner. For now everything is the stub. +REGISTRY = {"stub": run_stub} + + +def get_runner(agent_id: str): + return REGISTRY.get(agent_id, run_stub) diff --git a/release-agent/setup/bootstrap.ps1 b/release-agent/setup/bootstrap.ps1 new file mode 100644 index 00000000..f56433a2 --- /dev/null +++ b/release-agent/setup/bootstrap.ps1 @@ -0,0 +1,273 @@ +<# +.SYNOPSIS + One-time setup for the Release Orchestrator (/release-agent) on this machine. + Small by design: it only prepares Scout + the environment so the real work + can run inside Scout. It does NOT run a release. + +.DESCRIPTION + Steps: + 1. Infrastructure preflight — check CLIs/host deps AND register + verify the + MCP servers the skill needs inside Scout (from config/requirements.yaml). + If Python 3.9+ is missing it is installed automatically via winget + (per-user, no admin); PyYAML is then installed via pip. Pass + -NoAutoInstallPython to opt out of the Python auto-install. + 2. Install the /release-agent skill into the Scout skills folder — only if + Microsoft Scout is detected (~/.scout present); the copy is then verified. + 3. Print next steps. By default the script closes and relaunches Scout so the + new skill / MCP servers load (skipped automatically when this setup is run + from inside a Scout session, to avoid killing itself). Pass -NoRestartScout + to opt out and restart Scout yourself. + + HOW TO RUN + Run this from the `release-agent` folder of your android-complete clone, + using PowerShell 7 (pwsh): + + cd \android-complete\release-agent + pwsh .\setup\bootstrap.ps1 + + Example (default clone location): + + cd C:\repos\android-complete\release-agent + pwsh .\setup\bootstrap.ps1 + + You can launch it from any working directory as long as you give the full + path to the script (the script locates its own folder), e.g.: + + pwsh C:\repos\android-complete\release-agent\setup\bootstrap.ps1 + +.EXAMPLE + cd C:\repos\android-complete\release-agent + pwsh .\setup\bootstrap.ps1 +#> +[CmdletBinding()] +param( + [string]$ScoutSkillsDir = "$env:USERPROFILE\.scout\m-skills", + [switch]$SkipSkillInstall, + [switch]$NoAutoInstallPython, # by default, if Python is missing we install it via winget + [switch]$NoRestartScout # by default we restart Scout so new skill/MCP servers load +) + +$ErrorActionPreference = "Stop" + +# The Python engine prints UTF-8 (em-dashes, etc.). Make sure this console reads/writes +# UTF-8 so echoed output doesn't turn into mojibake like "ΓÇö". +try { + [Console]::OutputEncoding = [System.Text.Encoding]::UTF8 + $OutputEncoding = [System.Text.Encoding]::UTF8 + $env:PYTHONIOENCODING = "utf-8" + $env:PYTHONUTF8 = "1" +} catch {} + +$AgentRoot = Split-Path -Parent $PSScriptRoot # release-agent/ +$RepoRoot = Split-Path -Parent $AgentRoot # android-complete/ +$ReqFile = Join-Path $AgentRoot "config\requirements.yaml" + +Write-Host "Release Orchestrator bootstrap`n" -ForegroundColor Cyan +Write-Host " Running from: $AgentRoot" -ForegroundColor DarkGray +Write-Host " (run this from the 'release-agent' folder of your android-complete clone)`n" -ForegroundColor DarkGray + +# Sanity check: make sure we're actually in the release-agent folder. +if (-not (Test-Path (Join-Path $AgentRoot 'setup\bootstrap.ps1'))) { + Write-Host " This does not look like the release-agent folder." -ForegroundColor Red + Write-Host " cd into \android-complete\release-agent and run: pwsh .\setup\bootstrap.ps1`n" -ForegroundColor Red + exit 1 +} + +# True when THIS script was launched from a shell inside a Scout session — Scout sets +# these env vars for its child processes. Restarting Scout from there would kill us. +$InsideScout = [bool]($env:COPILOT_AGENT_SESSION_ID -or $env:COPILOT_CLI) + +# Stop all Scout processes and relaunch the app. Returns $true if it relaunched. +function Restart-Scout { + $procs = @(Get-Process -Name 'scout' -ErrorAction SilentlyContinue) + $exe = ($procs | Where-Object { $_.Path } | Select-Object -First 1).Path + if (-not $exe) { $exe = Join-Path $env:LOCALAPPDATA 'Programs\Microsoft Scout\scout.exe' } + if (-not (Test-Path $exe)) { + Write-Host " Could not locate scout.exe to relaunch." -ForegroundColor Yellow + return $false + } + if ($procs.Count) { + Write-Host " Closing Scout ($($procs.Count) process(es)) ..." -ForegroundColor DarkYellow + $procs | ForEach-Object { try { Stop-Process -Id $_.Id -Force -ErrorAction Stop } catch {} } + # Wait for the processes to actually exit before relaunching (single-instance lock). + for ($i = 0; $i -lt 20 -and (Get-Process -Name 'scout' -ErrorAction SilentlyContinue); $i++) { + Start-Sleep -Milliseconds 250 + } + } + Write-Host " Relaunching Scout ..." -ForegroundColor DarkYellow + # Launch fully DETACHED so Scout does NOT inherit this console — otherwise its + # Electron startup logs spill into the terminal after setup returns. Win32_Process + # Create starts it in a brand-new process with no console attachment. + $launched = $false + try { + $r = Invoke-CimMethod -ClassName Win32_Process -MethodName Create -Arguments @{ CommandLine = "`"$exe`"" } -ErrorAction Stop + if ($r.ReturnValue -eq 0) { $launched = $true } + } catch {} + if (-not $launched) { + # Fallback: explorer.exe launches the app as its child, also detached from our console. + try { Start-Process explorer.exe -ArgumentList "`"$exe`""; $launched = $true } catch {} + } + return $launched +} + +# ---- 1. Infrastructure preflight (CLIs + MCP servers), data-driven ---- +# Delegates to the engine (python -m orchestrator.cli infra), which reads +# config/requirements.yaml, checks every CLI/host dependency, and REGISTERS any +# missing MCP servers into Scout's config (backing it up first). One home for the +# logic; bootstrap just needs python+pyyaml to call it. +Write-Host "1. Infrastructure preflight (from config/requirements.yaml)" +if (-not (Test-Path $ReqFile)) { Write-Host " requirements.yaml not found at $ReqFile" -ForegroundColor Red; exit 1 } + +# Resolve a working Python (3.9+). Prefer `python`/`python3`, fall back to `py -3`, +# then probe well-known winget/python.org install dirs (PATH may be stale in this session). +function Resolve-Python { + foreach ($cand in @('python','python3')) { + try { & $cand -c "import sys; sys.exit(0 if sys.version_info[:2] >= (3,9) else 1)" 2>$null; if ($LASTEXITCODE -eq 0) { return $cand } } catch {} + } + try { & py -3 -c "import sys; sys.exit(0 if sys.version_info[:2] >= (3,9) else 1)" 2>$null; if ($LASTEXITCODE -eq 0) { return 'py -3' } } catch {} + # Direct probe of standard per-user install locations (newest first) + $globs = @( + "$env:LOCALAPPDATA\Programs\Python\Python3*\python.exe", + "$env:ProgramFiles\Python3*\python.exe" + ) + foreach ($g in $globs) { + $hit = Get-ChildItem $g -ErrorAction SilentlyContinue | Sort-Object FullName -Descending | Select-Object -First 1 + if ($hit) { + try { & $hit.FullName -c "import sys; sys.exit(0 if sys.version_info[:2] >= (3,9) else 1)" 2>$null; if ($LASTEXITCODE -eq 0) { return "`"$($hit.FullName)`"" } } catch {} + } + } + return $null +} + +$PyExe = Resolve-Python + +# If Python is missing, install it for the user via winget (per-user, no admin), then re-resolve. +if (-not $PyExe -and -not $NoAutoInstallPython) { + $winget = (Get-Command winget -ErrorAction SilentlyContinue) + if ($winget) { + Write-Host " [ ] Python 3.9+ not found - installing Python 3.12 via winget (no admin needed) ..." -ForegroundColor DarkYellow + winget install --id Python.Python.3.12 -e --source winget --scope user ` + --accept-package-agreements --accept-source-agreements --disable-interactivity 2>&1 | + ForEach-Object { Write-Host " $_" -ForegroundColor DarkGray } + # Refresh PATH from the registry so the just-installed python is visible this session. + $env:Path = [Environment]::GetEnvironmentVariable('Path','Machine') + ';' + + [Environment]::GetEnvironmentVariable('Path','User') + $PyExe = Resolve-Python + if ($PyExe) { Write-Host " [x] Python installed" -ForegroundColor Green } + else { Write-Host " [ ] Python installed but not detected yet - close and reopen your terminal, then re-run setup." -ForegroundColor Yellow } + } else { + Write-Host " [ ] Python 3.9+ missing and winget is unavailable for auto-install." -ForegroundColor Yellow + } +} + +$haveInfra = $false +if ($PyExe) { + # Python is present. Ensure PyYAML — install it silently via pip, no user action needed. + & cmd /c "$PyExe -c ""import yaml"" 2>nul" + if ($LASTEXITCODE -eq 0) { + $haveInfra = $true + } else { + Write-Host " [ ] PyYAML missing - installing via pip ..." -ForegroundColor DarkYellow + & cmd /c "$PyExe -m pip install --quiet --disable-pip-version-check pyyaml" + & cmd /c "$PyExe -c ""import yaml"" 2>nul" + if ($LASTEXITCODE -eq 0) { $haveInfra = $true; Write-Host " [x] PyYAML installed" -ForegroundColor Green } + } +} + +$ok = $true +$restartNeeded = $false +if (-not $haveInfra) { + if (-not $PyExe) { + Write-Host " [ ] Python 3.9+ ... still MISSING (needed to run the preflight)" -ForegroundColor Yellow + if ($NoAutoInstallPython) { + Write-Host " Auto-install was disabled (-NoAutoInstallPython)." -ForegroundColor DarkYellow + } + Write-Host " Install Python 3.9+ ('winget install Python.Python.3.12' or https://aka.ms/python)," -ForegroundColor DarkYellow + Write-Host " reopen your terminal, then re-run setup." -ForegroundColor DarkYellow + } else { + Write-Host " [ ] PyYAML ... could not be installed automatically" -ForegroundColor Yellow + } + $ok = $false +} else { + Push-Location $AgentRoot + try { + $out = & cmd /c "$PyExe -m orchestrator.cli infra 2>&1" + $out | ForEach-Object { Write-Host " $_" } + if ($LASTEXITCODE -ne 0) { $ok = $false } + if ($out -match "RESTART Scout") { $restartNeeded = $true } + } finally { Pop-Location } + + # engine config presence (cheap local sanity check) + if (Test-Path (Join-Path $AgentRoot 'config\phases.yaml')) { + Write-Host " [OK] engine config (phases.yaml)" -ForegroundColor Green + } else { + Write-Host " [MISSING] engine config (phases.yaml)" -ForegroundColor Yellow; $ok = $false + } +} + +if (-not $ok) { + Write-Host "`nSome infrastructure is missing — resolve the items above, then re-run." -ForegroundColor Yellow +} + +Write-Host "`n2. Skill install" +if ($SkipSkillInstall) { + Write-Host " Skipped (--SkipSkillInstall)." +} else { + $src = Join-Path $AgentRoot "skill\SKILL.md" + $ScoutRoot = Split-Path -Parent $ScoutSkillsDir # ~/.scout + $destDir = Join-Path $ScoutSkillsDir "release-agent" + $destFile = Join-Path $destDir "SKILL.md" + + if (-not (Test-Path $src)) { + Write-Host " [ ] Source skill not found at $src" -ForegroundColor Red + $ok = $false + } + # Is Scout actually installed? Its per-user data folder (~/.scout) is created on + # first launch. If it's absent, Scout isn't installed/run yet — don't fabricate it. + elseif (-not (Test-Path $ScoutRoot)) { + Write-Host " [ ] Microsoft Scout not detected ($ScoutRoot is missing)." -ForegroundColor Yellow + Write-Host " Install Microsoft Scout and launch it once, then re-run this setup" -ForegroundColor DarkYellow + Write-Host " to install the /release-agent skill." -ForegroundColor DarkYellow + $ok = $false + } + else { + New-Item -ItemType Directory -Force -Path $destDir | Out-Null + Copy-Item $src $destFile -Force + # Verify the skill actually landed (exists + non-empty + size matches source). + $srcLen = (Get-Item $src).Length + if ((Test-Path $destFile) -and ((Get-Item $destFile).Length -eq $srcLen) -and ($srcLen -gt 0)) { + Write-Host " [x] Installed /release-agent skill -> $destFile ($srcLen bytes)" -ForegroundColor Green + $skillInstalled = $true + } else { + Write-Host " [ ] Skill copy could not be verified at $destFile" -ForegroundColor Red + $ok = $false + } + } + + # Report every release-agent skill Scout can currently see. + if (Test-Path $ScoutSkillsDir) { + $installed = Get-ChildItem $ScoutSkillsDir -Directory -ErrorAction SilentlyContinue | + Where-Object { Test-Path (Join-Path $_.FullName 'SKILL.md') } | + Select-Object -ExpandProperty Name + if ($installed) { Write-Host " Skills currently installed in Scout: $($installed -join ', ')" -ForegroundColor DarkGray } + } +} + +$needsReload = ($restartNeeded -or $skillInstalled) +if ($needsReload) { + Write-Host "`nScout must reload to pick up the newly-installed skill / MCP server(s)." + if ($NoRestartScout) { + Write-Host " Auto-restart disabled (-NoRestartScout). Restart Scout yourself so the changes load." -ForegroundColor DarkYellow + } elseif ($InsideScout) { + Write-Host " Skipping auto-restart: this setup is running inside a Scout session," -ForegroundColor Yellow + Write-Host " so restarting would kill it. Close and reopen Scout manually." -ForegroundColor Yellow + } else { + $done = Restart-Scout + if ($done) { Write-Host " Scout restarted — the skill/MCP servers will load on launch." -ForegroundColor Green } + else { Write-Host " Please restart Scout manually so the changes load." -ForegroundColor Yellow } + } +} + +Write-Host "`n3. Next steps" -ForegroundColor Cyan +Write-Host " * Open Scout and run: /release-agent" +Write-Host " * The agent drives the whole release from inside Scout - just follow its prompts.`n" diff --git a/release-agent/skill/SKILL.md b/release-agent/skill/SKILL.md new file mode 100644 index 00000000..050e0db0 --- /dev/null +++ b/release-agent/skill/SKILL.md @@ -0,0 +1,70 @@ +--- +name: release-agent +description: Drive an Android release end-to-end using the Release Orchestrator backbone. Use when the user invokes /release-agent, says "start a release", "continue the release", "advance the release", "approve the gate", "release status", or asks about release run-state. The engine is deterministic and does the real work; this skill is the conversation layer that discovers releases, presents gate briefs and status, and relays the human decision. +--- + +# /release-agent — Release Orchestrator conductor + +> **Recommended model:** run on a high-reasoning model (e.g. **claude-opus-4.8**). Release work involves gate decisions, Component Governance / incident judgment, and multi-step state reconciliation. Scout skills can't self-select a model, so switch the session model before invoking if you're on a lighter one. (The unattended "Release push reminders" automation is already pinned to a strong model.) + +You are the conversation layer over the **Release Orchestrator engine** (deterministic Python). The engine decides what happens next; you discover releases, present status/gates, and relay decisions. **Never decide the release flow yourself, and never invent a release — always call the engine.** + +## Where things live +- Engine + config: `C:\repos\android-complete\release-agent\` — **run all `python -m orchestrator.cli …` commands from here.** +- Run-state: `C:\repos\android-complete\.release-runs\\release-state.json` (gitignored; one per month, e.g. `2026-08`). +- **Reference docs (this skill's detail):** `C:\repos\android-complete\release-agent\skill\reference\` — read the relevant one on demand (routing table below). The core stays lean; the details live there. +- `setup/bootstrap.ps1` only prepares the machine (infra preflight + installs this skill). If an infra check fails (an MCP server isn't registered, or Scout wasn't restarted), run `python -m orchestrator.cli infra` and tell the user to restart Scout; manifest is `config/requirements.yaml`. + +## GOLDEN RULES (always apply — the deduped essentials) +1. **Discover first, always.** On ANY release request, run `python -m orchestrator.cli list --json` and branch on `resolution`: `none` → offer to start (via `m_ask_user`); `one` → use `release.release_id`; `ambiguous` → list `all`, let the user pick; `explicit` → use it. Never run `status`/`next`/`approve` against an unconfirmed id. +2. **Render CLI output as LIVE MARKDOWN — never fenced.** `checklist`, `status`, `next`, etc. print finished markdown tables. Reproduce their stdout **verbatim as normal message content** so Scout renders the table — do NOT wrap in a ``` code fence, and do NOT rebuild/re-order/re-type from memory (you'll introduce stale icons / broken URLs). A sentence before/after is fine; the block must match. Use `--json` only for your own branching. **Running the command is NOT the same as showing it** — the CLI auto-logs, but the user only sees what YOU paste into your reply. If you ran `checklist`/`status` and didn't paste its table, the user saw nothing. +2b. **NEVER ask for a gate decision or attestation in a message that doesn't contain the freshly-rendered table.** Before any `m_ask_user` for attestations (entry gate) or Approve/Deny (a gate), the SAME assistant message must first show the current `checklist`/`status` table pasted verbatim. A bare list of items is not acceptable — the table is the context. If you're about to ask and haven't pasted the table in this message, run the command and paste it first. +3. **The engine is the source of truth.** It owns sequencing and gate state. When unsure, `status --json`. Never hand-edit checklist/status output. +4. **Prompt, don't interrogate.** For any discrete choice (start? which release? approve/deny?) use the `m_ask_user` clickable prompt, not free-text. Reserve free-text for genuinely open values (an unusual month). +5. **Never assume a human decision.** An `m_ask_user` result that merely echoes the offered options is NOT confirmation. Never attest, approve, sign, or mark done until the user explicitly said so. Attesting/approving on an assumption is a release-integrity violation. +6. **Gates are human-decided.** Present and relay Approve/Deny; never authorize yourself. +7. **Runs are real; mock for safety.** There is no dry-run — every run makes real calls (real reads, real sends, real writes). For testing, the engineer keeps a personal `mocks.local.yaml` (gitignored) that skips, blocks, redirects (`send_to`), or injects inputs per step. `[STUB…]` output = an unbuilt later-phase step; say so, don't imply real work. See `mock-spec` for what each step exposes. +8. **Never hardcode a recipient.** Reminders/notices go to the release `owner_email` from metadata (or engine-resolved DLs). +9. **Log silently.** Human-readable commands auto-log. YOU must journal user choices: `journal --release --source user --kind choice --text "" --choice "