test(github): stop the pre-release install test failing after a stable release - #1182
Conversation
Move stub_command, along with its setup, teardown and assertion helpers, out of metrics_test.bats and into test_helper.sh, so that other bats suites can stub the tools that their code shells out to. Add stub_argv, which replaces the direct reads of $STUB_ARGS_DIR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NWz3v5hdZRcyB6RLPzDgGZ
install_latest_github_release with pre-releases enabled installs the newest release of any kind, because latest_github_release_version only drops --exclude-pre-releases. `gh release list` returns releases newest-first, so a stable release wins from the moment it is published until the next commit re-creates the repo's `unstable` release. The pre-release test asserted that the installed version matched (rc|unstable), so it failed for the whole of that window. Cover the flags that the function builds with a stubbed `gh` instead, which also keeps that coverage off the GitHub API rate limit, and skip the live pre-release install while the newest getoutreach/stencil release is a stable one. Correct the doc comments, which described the stricter behavior that the test assumed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NWz3v5hdZRcyB6RLPzDgGZ
marnagy
left a comment
marnagy
left a comment
There was a problem hiding this comment.
LGTM — verified build/lint/tests pass locally (13/13 bats, incl. live GitHub tests). Fix correctly targets the newest-release-vs-pre-release selection bug; new stub-based tests decouple coverage from live GitHub state.
- Generated using AI
| run stub_argv gh | ||
| assert_line "--exclude-drafts" | ||
| refute_line "--exclude-pre-releases" | ||
| } |
There was a problem hiding this comment.
Nitpick: This test only asserts --exclude-drafts/--exclude-pre-releases, not the repo slug like the test above it. Not a real gap — just flagging for symmetry, feel free to ignore.
- Generated using AI
There was a problem hiding this comment.
The split is deliberate. The test above pins the full argument shape, including the slug, and this one pins only what differs between the two modes. Repeating the slug assertion here would not catch anything the other test misses.
Generated by Claude Code
|
🎉 This PR is included in version 2.39.3-rc.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
3 participants
What this PR does / why we need it
latest_github_release_versiononly drops--exclude-pre-releaseswhen pre-releases are requested, so it returns the newest release of any kind. Between a stable stencil release and the next commit that re-creates that repo'sunstablerelease, the newest release is a stable one, and the test's assertion that the installed version matches(rc|unstable)failed.Cover the flags the function builds with a stubbed
gh, so that coverage no longer depends on another repo's release state or on the GitHub API rate limit. Skip the live install test while the newest release is stable. Correct the doc comments, which described the stricter behavior the test assumed.Notes for your reviewers
The first commit is almost all relocation.
stub_commandandassert_stub_not_calledmove frommetrics_test.batstotest_helper.shbyte-identical, and onlystub_argvis new.The skip guard spends one extra
gh release listcall. Reading the tag from the install log would avoid it, but(rc|unstable)also matches CI paths such as/home/circleci/....🤖 Generated by Claude Code