From 7d23dfc054a27e2c3569736dc6108cede5d365bd Mon Sep 17 00:00:00 2001 From: James Kessler Date: Thu, 13 Aug 2026 17:51:10 -0700 Subject: [PATCH 1/2] Smoke test: add weekly schedule The existing `workflow_run` trigger only fires after releases of *this* repo, so it never catches changes underneath us -- a new coverage-reporter release, a runner-image update, or a Homebrew change. Those are precisely the drifts that break the end-user install chain silently. Concrete example: coverage-reporter v0.6.18 shipped tonight and nothing here fired. Verifying it required remembering this workflow existed and dispatching it by hand, which is exactly the kind of step that gets skipped. Adds a weekly cron as a safety net. The job's existing `if:` guard already passes for schedule events (`github.event_name` is 'schedule', so the workflow_run conclusion check is short-circuited), so no other change needed. Chose a weekly schedule over wiring a `repository_dispatch` from coverage-reporter's build.yml: cross-repo dispatch cannot use `secrets.GITHUB_TOKEN` and would require creating, storing, and rotating a PAT. A schedule needs no credentials and additionally covers drift that no coverage-reporter release would trigger -- which is where two of tonight's three breakages actually came from. Cron is set off the hour to avoid GitHub's peak-of-hour scheduling delays. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/smoke-test.yml | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/.github/workflows/smoke-test.yml b/.github/workflows/smoke-test.yml index 903db92d..900d2217 100644 --- a/.github/workflows/smoke-test.yml +++ b/.github/workflows/smoke-test.yml @@ -3,8 +3,9 @@ name: Smoke Test (published action) # Verifies the full end-user chain of the *published* v2 tag on every # supported platform: action resolution -> reporter install (Homebrew on # macOS, GitHub-release download on Linux/Windows) -> report upload. -# Runs automatically after each release (once the v2 tag has been moved -# by the Update Major Version Tag workflow). +# Runs automatically after each release of *this* repo (once the v2 tag has +# been moved by the Update Major Version Tag workflow), and weekly on a +# schedule to catch changes underneath us that no release here would trigger. # # Run it manually after any change worth sanity-checking against the # published action -- e.g. a Homebrew major version, a runner-image @@ -18,6 +19,12 @@ on: workflow_run: workflows: ["Update Major Version Tag"] types: [completed] + # Weekly safety net. The workflow_run trigger above only fires on *this* + # repo's releases, so it does not catch changes underneath us -- a new + # coverage-reporter release, a runner-image update, or a Homebrew change. + # Those are exactly the drifts that break the end-user chain silently. + schedule: + - cron: "23 12 * * 1" # Mondays 12:23 UTC (off the hour to dodge peak load) jobs: smoke: From bf68ae0ab1286130245ce8af93e4630914f225b7 Mon Sep 17 00:00:00 2001 From: James Kessler Date: Thu, 13 Aug 2026 18:00:10 -0700 Subject: [PATCH 2/2] Smoke test: open an issue on failure Email is a weak alarm for a scheduled job. GitHub sends scheduled-run failure notifications only to whoever last edited the cron expression -- not to repo watchers -- and if that person's notification settings are wrong, the failure is silent. We would have automated the check without automating the alert, which is worse than not automating it: it looks watched when it is not. Adds a `report-failure` job that opens an issue when any smoke matrix job fails. Visible, persistent, and independent of anyone's inbox configuration. Details: - Reuses the existing open issue (via the `smoke-test-failure` label) and comments on it, so a recurring weekly failure does not open 52 issues. - Creates the label on first use, so no manual setup is required. - Scoped `permissions: issues: write` on the job rather than the workflow. - The issue body explains what the failure implies (end users are likely broken right now, even though nothing in this repo changed) and lists the usual causes, so whoever picks it up has context rather than a bare link. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/smoke-test.yml | 58 ++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/.github/workflows/smoke-test.yml b/.github/workflows/smoke-test.yml index 900d2217..8ef61be6 100644 --- a/.github/workflows/smoke-test.yml +++ b/.github/workflows/smoke-test.yml @@ -63,3 +63,61 @@ jobs: run: | which coveralls coveralls --version + + # Email alone is a weak alarm for a scheduled job: GitHub sends scheduled-run + # failure notifications only to whoever last edited the cron, and a wrong + # notification setting fails silently. Open an issue instead -- visible, + # persistent, and independent of anyone's inbox config. + report-failure: + needs: smoke + if: failure() + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - name: Open or update the smoke-test failure issue + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GH_REPO: ${{ github.repository }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + TRIGGER: ${{ github.event_name }} + run: | + set -euo pipefail + + label="smoke-test-failure" + # Ensure the label exists; harmless if it already does. + gh label create "$label" --color B60205 \ + --description "Published action failed its end-user smoke test" 2>/dev/null || true + + body=$(cat <