Skip to content

Issue 4940: Add skip deployment flag for PR bot tests#4947

Open
stuart-bass-cgi wants to merge 4 commits into
microsoft:mainfrom
stuart-bass-cgi:issue-4940-skip-deployment-pr-bot
Open

Issue 4940: Add skip deployment flag for PR bot tests#4947
stuart-bass-cgi wants to merge 4 commits into
microsoft:mainfrom
stuart-bass-cgi:issue-4940-skip-deployment-pr-bot

Conversation

@stuart-bass-cgi

Copy link
Copy Markdown

Resolves #4940

What is being addressed

PR bot /test and /test-extended commands currently always run the full validation flow, including build and deployment jobs, even when the PR validation environment has already been deployed. This makes it slow to re-run smoke or extended E2E tests after transient test failures or test-only changes.

How is this addressed

  • Adds a skip_deployment flag for the /test and /test-extended PR bot commands.
  • Passes the parsed flag from the PR comment bot into the reusable TRE deployment workflow.
  • When skip_deployment is true, skips the deployment/build/register jobs and allows the smoke and requested custom E2E jobs to run against the existing deterministic PR validation environment.
  • Keeps the existing external PR SHA validation in place, including when skip_deployment is supplied.
  • Updates PR bot command documentation with the new syntax and fork PR examples.
  • Adds unit test coverage for the new command parsing behaviour.

@stuart-bass-cgi stuart-bass-cgi requested a review from a team as a code owner July 3, 2026 11:59
Copilot AI review requested due to automatic review settings July 3, 2026 11:59
@github-actions github-actions Bot added the external PR from an external contributor label Jul 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a skip_deployment flag to PR comment bot /test and /test-extended commands so maintainers can rerun smoke/extended E2E tests against an already-deployed deterministic PR validation environment, avoiding the full build/deploy pipeline when appropriate.

Changes:

  • Extended PR bot command parsing to recognize skip_deployment and pass it through to the reusable deployment workflow.
  • Updated the reusable workflow to conditionally bypass deployment/build stages and still run smoke/custom E2E jobs.
  • Updated developer docs and added unit tests covering the new command parsing behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
docs/tre-developers/github-pr-bot-commands.md Documents the new skip_deployment flag syntax and fork PR examples for /test and /test-extended.
.github/workflows/pr_comment_bot.yml Plumbs skipDeployment output from the parsing script into the reusable workflow call.
.github/workflows/deploy_tre_reusable.yml Adds skipDeployment input and adjusts E2E job conditions to allow running tests when deployment is skipped.
.github/scripts/build.js Parses skip_deployment, emits skipDeployment output, and updates external-PR SHA extraction to ignore the flag.
.github/scripts/build.test.js Adds unit tests validating skipDeployment output behavior for internal and external PR command forms.

Comment thread .github/workflows/pr_comment_bot.yml
Comment thread .github/workflows/pr_comment_bot.yml
Comment thread .github/scripts/build.js
@stuart-bass-cgi

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="CGI"

@marrobi

marrobi commented Jul 4, 2026

Copy link
Copy Markdown
Member

@stuart-bass-cgi thanks for the PR.

A review with Opus 4.8 says:

I


Overall: Well-structured, focused change that solves a real developer pain point (re-running smoke/extended E2E tests without a full redeploy). Good test coverage and docs, plus a couple of nice opportunistic bug fixes. A few items to confirm before approval.

🟢 Done well

  • Clean separation of commandHasSkipDeploymentFlag() and getShaFromCommandParts() — correctly supports passing a SHA and the flag in any order on fork PRs.
  • Good unit-test coverage for internal true/default-false, fork /test <sha> skip_deployment, and /test-extended.
  • Fixes real pre-existing bugs: branchRefidbranchRefId typo, and the debug echo that printed prRefId under the branchRefId label.
  • Docs updated in step with the code, including fork examples.

🟠 To address

  1. Check status reporting on skip-deployment runs. With skipDeployment == true, deploy_management is skipped, so the entire needs chain feeding e2e_tests_smoke/e2e_tests_custom is skipped. The always() && (inputs.skipDeployment || ...) guards make the tests run, which works — but please confirm the summary job still reports the "Deploy PR / Run E2E Tests (Smoke)" check back to the PR head SHA so the PR status check updates correctly. A short code comment explaining why always() + skipped upstreams produces the intended behavior would help future maintainers.
  2. skip_deployment is silently ignored by /test-extended-aad and /test-shared-services. The flag is only wired into /test and /test-extended. A user appending it to the other commands gets a full deploy with no feedback. Consider warning on unsupported use, or confirm this asymmetry is intentional (docs already omit it for those commands).

🟡 Nits

  • skip_deployment is a magic string in multiple places — consider a shared constant to avoid drift.
  • const { create } = require('domain'); (build.js) is unused dead code; low-risk cleanup while you're here.
  • The string→bool round-trip (== 'true') for the typed workflow input is correct — just flagging it's intentional.

Verdict: LGTM pending confirmation of (1) smoke-test check reporting and (2) the intentional flag asymmetry.


Let me know if you agree!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external PR from an external contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip deployment parameter for PR bot test commands

4 participants