Skip to content

Refresh Azure DevOps VCS integration docs - #20733

Open
jkodroff wants to merge 3 commits into
masterfrom
jkodroff/refresh-vcs-ado
Open

Refresh Azure DevOps VCS integration docs#20733
jkodroff wants to merge 3 commits into
masterfrom
jkodroff/refresh-vcs-ado

Conversation

@jkodroff

@jkodroff jkodroff commented Aug 6, 2026

Copy link
Copy Markdown
Member

Fixes #20720.

Azure DevOps page

Restructured onto the template GitLab and Bitbucket already use, and filled in what was missing:

  • Intro states what the integration enables (deploy on push, PR previews/comments, review stacks, New Project Wizard) instead of one line.
  • New Capabilities section: pull request comments, commit status checks, push-to-deploy, review stacks, injected environment variables.
  • Corrected the service hook list to four events (git.pullrequest.merged was missing).
  • Documented review stack naming (pr-{org}-{repo}-{prId}), the injected env vars, and AZURE_DEV_OPS_TOKEN.
  • Added Template sources — ADO is supported, https://dev.azure.com/<org>/<project>/_git/<repo>.
  • Rewrote the OIDC section: there is no user-facing surface to configure: Pulumi creates the Entra federated credential at connect time.
  • New What Pulumi creates in your tenant and project: the single-tenant Pulumi app registration, its service principal (Basic access level), the federated credential, and the scoped Pulumi Service (<org>) group with its exact permissions — plus the temporary Project Administrators elevation during setup and what deletion removes.

Version control overview

New "What every integration provides" section covering the capabilities common to all native providers, with the Custom VCS exception called out. This is the second half of the issue.

Corrections across VCS and Neo pages

  • Commit statuses: GitLab and Bitbucket claimed statuses on "both push and pull request events". Outside GitHub, statuses are posted for pull request deployments only — the push paths never set PULUMI_CI_PULL_REQUEST_SHA, and common.synchronizeCheckRun bails without it.
  • Neo: removed the Neo Code Reviews / Code access for AI reviews toggles, deleted from the product in pulumi-service#46465, from all four provider pages; added the real GitHub-only Draft pull request comments toggle. Neo code reviews are GitHub-only, and the pages now say so plainly — PR comments on ADO, GitLab, and Bitbucket come from Pulumi Deployments, not Neo.
  • New Project Wizard / no-code stacks: docs said no-code was GitHub + ADO only. Any configured VCS provider works; Bitbucket is now listed everywhere it belongs, and the ESC and VCS-integration requirements are documented.

Notes

  • Item 3 of the issue — Neo AI summaries on ADO — is not possible today. Neo does no work on ADO or Bitbucket; that is epic work, confirmed by @lukebward.
  • Filed pulumi/pulumi-service#47592: ADO branch pushes inject no environment variables. The env var table documents only the PR and tag rows until that is fixed.
  • Everything here is verified against pulumi-service source.

@jkodroff @claude
Refresh Azure DevOps VCS integration docs
a68e818 
Restructures the ADO page onto the GitLab/Bitbucket template and documents
capabilities, service hook events, review stack naming, injected env vars,
template sources, and the Entra ID resources Pulumi creates in your tenant.

Also: shared-capabilities section on the VCS overview; commit statuses are
pull-request-only outside GitHub; Neo code reviews are GitHub-only and the
deleted Neo toggles are gone from all provider pages; NPW and no-code stacks
work with any configured VCS provider, Bitbucket included.

Addresses #20720 — Neo summaries on ADO (item 3) aren't possible
today. Filed pulumi/pulumi-service#47592 for ADO branch pushes injecting no
environment variables.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jkodroff
jkodroff force-pushed the jkodroff/refresh-vcs-ado branch from 6ddc1b4 to a68e818 Compare August 6, 2026 15:00
@jkodroff
jkodroff requested a review from CamSoper August 6, 2026 15:01
@jkodroff
Merge remote-tracking branch 'origin/master' into jkodroff/refresh-vc…
2901745 
…s-ado

# Conflicts:
#	content/docs/integrations/version-control/azure-devops-integration.md
@jkodroff

jkodroff commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

@claude #update-review

@pulumi-bot

pulumi-bot commented Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Your site preview for commit fa58775 is ready! 🎉

http://www-testing-pulumi-docs-origin-pr-20733-fa58775c.s3-website.us-west-2.amazonaws.com

Changed pages:

@CamSoper

CamSoper commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@claude #new-review

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

🤖 @CamSoper — pinned review cleared; regenerating from scratch.

@jkodroff

jkodroff commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

@claude #update-review

@jkodroff
jkodroff marked this pull request as draft August 6, 2026 17:28
@jkodroff
jkodroff marked this pull request as ready for review August 6, 2026 17:28
@CamSoper

CamSoper commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@claude #new-review

CamSoper
CamSoper approved these changes Aug 6, 2026
View reviewed changes

@CamSoper CamSoper left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, gonna assume everything checks out locally given the GitHub weirdness today

@github-actions github-actions Bot added the review:in-progress Claude review is currently running label Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026 •
edited by pulumi-bot
Loading

Copy link
Copy Markdown
Contributor

Pre-merge Review — Last updated 2026-08-07T16:19:49Z

Tip

Summary: This PR rewrites the Azure DevOps version-control integration page to match its GitHub, GitLab, and Bitbucket siblings, narrows Neo code reviews to GitHub-only, and narrows non-GitHub commit statuses to pull-request deployments only. Commit fa58775 ("Process PR feedback") resolves three of the five blockers from the prior review: the Neo code-reviews configuration path now agrees across all four touched pages, the New Project Wizard section no longer claims review stacks are automatic, and the draft-pull-request divergence on Azure DevOps is now explicitly called out as an intentional exception rather than reading like an editing slip. Two blockers remain open, both about the commit-status narrowing on GitLab, Bitbucket, and Azure DevOps (push-to-deploy no longer posts a commit status) — the new commit adds supporting detail on the Azure DevOps side, but the GitLab and Bitbucket pages asserting the same narrowing are untouched and still uncorroborated by anything outside this PR's own rewrite. This pass re-verified the previously-outstanding findings against the new diff, swept for unflagged duplicates of the resolved claims, verified the new claims the commit introduced (the commit-status branch-policy link, the tag-push rationale for PULUMI_CI_BRANCH_NAME), and re-ran prose linting.

Review confidence:

Dimension Level Notes
mechanics HIGH
facts LOW Most of the Azure DevOps setup detail (Entra app registration, service principal, scoped group permissions, federated credentials, service hook events, commit-status naming) still describes unreleased/undocumented product behavior with no public source to check it against.
cross-sibling consistency MEDIUM 3 of 4 prior mismatches resolved by commit fa58775 (Neo code-reviews path, wizard/review-stacks claim, draft-PR divergence now explained); the commit-status narrowing on GitLab/Bitbucket/Azure DevOps remains unconfirmed.
code correctness HIGH No code samples or example programs changed.
Investigation log
  • Cross-sibling reads: 5 of 5 siblings (re-checked the 3 resolved cross-sibling mismatches against the new commit)
  • External claim verification: 82 of 130 claims verified (43 unverifiable, 3 contradicted, 1 framing-drift) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 68 Pass 1, 7 Pass 2 (verified 1, contradicted 0, unverifiable 6), 55 Pass 3 (verified 19, contradicted 3, unverifiable 33).
  • Cited-claim spot-checks: 7 of 7 cited claims fetched and compared (added the new commit-status branch-policy citation)
  • Frontmatter sweep: ran on body + meta_desc (no frontmatter changed by this commit)
  • Temporal-trigger sweep: ran (no new recency words introduced)
  • Code execution: not run (no static/programs/ change)
  • Code-examples checks: ran (3 specialists: structural, existence, body-code-coverage); 0 findings
  • Editorial-balance pass: not run (not under content/blog/)
🚨 Outstanding ⚠️ Low-confidence 💡 Pre-existing ✅ Resolved
2 47 0 4

🔍 Verification trail

130 claims extracted · 82 verified · 43 unverifiable · 3 contradicted · 1 framing-drift
  • L37 in content/docs/ai/neo/code-reviews/_index.md "On Azure DevOps, GitLab, or Bitbucket, pull request comments come from the version control integration rather than from Neo." → 🤝 matches (evidence: The linked /docs/integrations/version-control/ page describes GitHub, GitLab, Bitbucket, and Azure DevOps as "native integrations" that all provide "Pull request previews and comments... Pulumi runs pulumi preview and posts the resource ch…; source: repo:content/docs/integrations/version-control/_index.md)
  • L126 in content/docs/ai/neo/permissions/_index.md "When a Neo task needs individual GitHub access that the user hasn't connected, Neo posts a nudge prompting the user to grant it." → 🤷 unverifiable (evidence: Public docs confirm that some Neo tasks require individual GitHub access to be granted, but not the nudge UX; source: WebSearch ran query "Pulumi Neo task individual GitHub access grant")
  • L126-127 in content/docs/ai/neo/permissions/_index.md "GitHub.com access via the shared Pulumi GitHub App or the user's connected GitHub account is required to create repositories and to trigger Neo code reviews." → 🤝 matches (evidence: corroborated by the sibling code-reviews doc's Setup section; source: repo:content/docs/ai/neo/code-reviews/_index.md, repo:content/docs/ai/neo/permissions/_index.md L126)
  • L128-131 in content/docs/ai/neo/permissions/_index.md "For Custom VCS, the credentials come from the integration's ESC environment, which is a shared identity belonging to whoever configured the integration." → ✅ verified (source: repo:content/docs/ai/neo/permissions/_index.md L131, L118)
  • L25 in content/docs/ai/neo/pull-requests/_index.md "On GitHub, Neo analyzes pull requests that the user's team opens and leaves its feedback right in the PR." → ✅ verified (source: repo:content/docs/ai/neo/code-reviews/_index.md)
  • L33 in content/docs/idp/concepts/new-project-wizard.md "In the New Project Wizard's destination repository configuration, a user can select an existing repository or create a new one on GitHub, Azure DevOps, GitLab, or Bitbucket." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md)
  • L62-68 in content/docs/idp/concepts/new-project-wizard.md "To use the Pulumi Deployments (VCS-backed) deployment method, the organization needs an integration for the chosen VCS provider, and the individual user needs to authorize their account." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md L66, L70-72)
  • L66-68 in content/docs/idp/concepts/new-project-wizard.md "Using the Pulumi Deployments (VCS-backed) deployment method requires the individual user to authorize their account with the chosen VCS provider." → ✅ verified (framing: entailed-narrower special case; source: https://www.pulumi.com/docs/integrations/version-control/github-app/)
  • L67 in content/docs/idp/concepts/new-project-wizard.md "Pulumi Deployments (no-code): stores the stack's configuration in a Pulumi ESC environment instead of a repository." → 🤝 matches (source: repo:content/docs/idp/concepts/no-code-stacks.md)
  • L72 in content/docs/idp/concepts/new-project-wizard.md "To use the New Project Wizard with Pulumi Deployments, users must authorize their VCS provider, and Pulumi supports GitHub, Azure DevOps, GitLab, and Bitbucket." → ✅ verified (source: repo:content/docs/integrations/version-control/_index.md)
  • L74 in content/docs/idp/concepts/new-project-wizard.md "GitHub lacks fine-grained repository permissions as part of its OAuth application scopes, per GitHub's documentation." → ✅ verified (source: https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/deciding-when-to-build-a-github-app)
  • L74 in content/docs/idp/concepts/new-project-wizard.md "Pulumi uses the GitHub authorization permissions to read template sources, write template content into repositories, and optionally create new repositories." → ✅ verified (source: https://www.pulumi.com/docs/idp/concepts/new-project-wizard/)
  • L21-23 in content/docs/idp/concepts/no-code-stacks.md "No-code workflows require Pulumi Deployments." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md)
  • L21-25 in content/docs/idp/concepts/no-code-stacks.md "The template used for a no-code stack can be sourced from a different version control provider than the one connected via the organization's version control integration." → ✅ verified (source: repo:content/docs/idp/concepts/organization-templates.md, repo:content/docs/deployments/concepts/settings/source.md)
  • L25 in content/docs/idp/concepts/no-code-stacks.md "Any version control provider works for the required version control integration, including GitHub, GitLab, Azure DevOps, or Bitbucket." → ✅ verified (source: repo:content/docs/integrations/version-control/_index.md)
  • L25 in content/docs/idp/concepts/no-code-stacks.md "No-code stacks work with any configured version control provider, widening the previously-documented GitHub-and-Azure-DevOps-only support." → 🤷 unverifiable (evidence: the two edits (this page and new-project-wizard.md's deleted Limitations section) are internally consistent, but no public source confirms the restriction was actually lifted; source: repo:content/docs/idp/concepts/new-project-wizard.md, repo:content/docs/idp/concepts/no-code-stacks.md)
  • L27 in content/docs/idp/concepts/no-code-stacks.md "The template for a no-code stack must come from a repository on one of the supported version control providers or from the Pulumi Private Registry." → ✅ verified (source: repo:content/docs/idp/concepts/organization-templates.md)
  • L230 in content/docs/idp/concepts/organization-templates.md "GitHub app authorization for organization template sources can be set up during the new project wizard flow, at anchor '/docs/idp/concepts/new-project-wizard/#vcs-authorization'." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md)
  • L19 in content/docs/integrations/version-control/_index.md "Pulumi version control integrations connect Pulumi with a VCS provider, enabling infrastructure previews on pull requests and automated deployment workflows." → ✅ verified (source: repo:content/docs/integrations/version-control/_index.md)
  • L19 in content/docs/integrations/version-control/_index.md "Pulumi supports connecting any Git or Mercurial server via a Custom VCS integration." → ✅ verified (source: repo:content/docs/integrations/version-control/custom-vcs.md, repo:content/releases/changelog/2026-05-04-custom-vcs.md)
  • L25 in content/docs/integrations/version-control/_index.md "GitHub, GitLab, Bitbucket, and Azure DevOps are all native Pulumi version control integrations that provide the same core set of capabilities." → ➖ not-a-claim (evidence: paraphrase of the page's own "What every integration provides" section)
  • L27-28 in content/docs/integrations/version-control/_index.md "When a pull request is opened or updated on a natively-integrated VCS provider, Pulumi runs pulumi preview and posts the resource changes back as a comment." → ✅ verified (source: https://www.pulumi.com/docs/deployments/concepts/settings/)
  • L27 in content/docs/integrations/version-control/_index.md "Pull request comments posted by Pulumi can include a collapsible property-level diff." → ✅ verified (source: gh api repos/pulumi/pulumi-pulumiservice contents/provider/pkg/apitype/git_hub_settings_request_.go)
  • L28 in content/docs/integrations/version-control/_index.md "Pull request deployment results are reported back to the VCS provider as commit statuses, each linking to the full deployment in Pulumi Cloud." → ✅ verified (source: repo:content/docs/integrations/version-control/gitlab.md Commit status checks section)
  • L28 in content/docs/integrations/version-control/_index.md "On VCS providers other than GitHub (GitLab, Bitbucket, Azure DevOps), commit statuses are posted for pull request deployments only, not for push-to-deploy runs." → 🤝 matches (evidence: sibling provider pages consistently confirm this; source: repo:content/docs/integrations/version-control/gitlab.md, bitbucket.md, azure-devops-integration.md)
  • L28 in content/docs/integrations/version-control/_index.md "On GitLab, Bitbucket, and Azure DevOps, commit statuses are posted for pull request deployments only, not for push-to-deploy runs." → ❌ contradicted; still open — see 🚨 Outstanding (evidence: Pulumi's own published GitLab and Bitbucket docs state the opposite: "Pulumi posts commit status checks to GitLab on every deployment, for both push and merge request events"; source: https://www.pulumi.com/docs/integrations/version-control/gitlab/, https://www.pulumi.com/docs/integrations/version-control/bitbucket/)
  • L29-30 in content/docs/integrations/version-control/_index.md "Pushing to a configured branch on a natively-integrated VCS provider runs pulumi up automatically (push-to-deploy)." → ✅ verified (source: repo:content/docs/deployments/concepts/triggers.md)
  • L30 in content/docs/integrations/version-control/_index.md "A review stack's ephemeral environment is destroyed automatically when the pull request is merged or closed." → ✅ verified (source: repo:content/docs/deployments/concepts/review-stacks.md)
  • L31 in content/docs/integrations/version-control/_index.md "Pulumi lists a user's repositories and branches when configuring a stack's deployment settings, instead of requiring identifiers to be typed by hand." → ✅ verified (source: https://www.pulumi.com/docs/deployments/concepts/settings/)
  • L32 in content/docs/integrations/version-control/_index.md "Pulumi handles token issuance and rotation for managed credentials in native VCS integrations." → ✅ verified (source: https://www.pulumi.com/blog/expanded-version-control-support/)
  • L33 in content/docs/integrations/version-control/_index.md "There is a documentation page for organization templates at /docs/idp/concepts/organization-templates/." → ✅ verified (source: gh search code --owner pulumi "organization-templates")
  • L35 in content/docs/integrations/version-control/_index.md "The Custom VCS integration supports push-to-deploy only, and does not support pull request comments, commit statuses, or review stacks." → ✅ verified (source: content/docs/integrations/version-control/custom-vcs.md)
  • L37 in content/docs/integrations/version-control/_index.md "Provider-specific documentation pages detail differences such as authentication methods, repository identifier formats, and injected environment variables." → ✅ verified (source: repo:content/docs/integrations/version-control/github-app.md)
  • L63-64 in content/docs/integrations/version-control/_index.md "The Azure DevOps integration connects Azure DevOps projects to Pulumi Cloud for pull request previews, push-to-deploy, review stacks, and commit statuses." → ✅ verified (source: repo:content/docs/integrations/version-control/azure-devops-integration.md)
  • L18 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi Cloud connects to Azure DevOps repositories to provide pull request previews, push-to-deploy, review stacks, commit statuses, and automated deployments." → ✅ verified (evidence: the page details each capability in dedicated sections)
  • L18 in content/docs/integrations/version-control/azure-devops-integration.md "The Azure DevOps integration capabilities are the same capabilities available with the other native integrations." → 🤝 matches (source: repo:content/docs/integrations/version-control/bitbucket.md)
  • L22-23 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi runs pulumi preview when a pull request is opened or updated and posts the resource changes back as a pull request comment in Azure DevOps." → ✅ verified (source: https://www.pulumi.com/docs/deployments/concepts/settings/)
  • L24 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi posts a deployment's outcome as an Azure DevOps commit status." (new in fa58775) → ✅ verified (evidence: restates the already-verified capabilities-list claim in the same terms as the Commit status checks section below; source: repo:content/docs/integrations/version-control/azure-devops-integration.md L113-121)
  • L25 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi's review stacks feature creates a real, ephemeral environment per pull request and tears it down on merge or close." → ✅ verified (source: repo:content/docs/deployments/concepts/review-stacks.md)
  • L26 in content/docs/integrations/version-control/azure-devops-integration.md "The New Project Wizard can be used to scaffold Pulumi projects and stacks into Azure DevOps repositories." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md)
  • L31 in content/docs/integrations/version-control/azure-devops-integration.md "To set up the Azure DevOps integration, the user needs an Azure DevOps organization and project." → ✅ verified (evidence: the doc's own setup note also requires an org-admin role and Entra ID tenant, which is the broader form)
  • L34-37 in content/docs/integrations/version-control/azure-devops-integration.md "After authorizing, users select the Azure DevOps organization and project they want to integrate with, then configure integration settings." → 🤷 unverifiable (evidence: the cited sign-in URL is login-gated and returns no supporting content; source: https://app.pulumi.com/signin)
  • L35 in content/docs/integrations/version-control/azure-devops-integration.md "The Azure DevOps integration is set up by navigating to Management > Version control in Pulumi Cloud." → ✅ verified (source: repo:content/docs/integrations/version-control/gitlab.md)
  • L36 in content/docs/integrations/version-control/azure-devops-integration.md "Setting up the integration requires selecting Authorize Azure DevOps and completing the Microsoft Entra ID sign-in." → 🤷 unverifiable (evidence: web search surfaces only general Azure DevOps material, not this specific UI flow)
  • L37 in content/docs/integrations/version-control/azure-devops-integration.md "After authorizing, the user selects the Azure DevOps organization and project to integrate with and configures integration settings." → ✅ verified (framing: special case of the general authorize-then-configure flow; source: https://www.pulumi.com/blog/expanded-version-control-support/)
  • L39 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi registers service hooks in the Azure DevOps project for the git.push, git.pullrequest.created, git.pullrequest.updated, and git.pullrequest.merged events." → 🤷 unverifiable (evidence: event names are valid Azure DevOps service hook types, but Pulumi's specific usage isn't publicly confirmed)
  • L41 in content/docs/integrations/version-control/azure-devops-integration.md "Each Azure DevOps integration maps one Pulumi organization to one Azure DevOps organization and project pair." → 🤷 unverifiable
  • L45-48 in content/docs/integrations/version-control/azure-devops-integration.md "Authorization for the Azure DevOps integration is a single Microsoft Entra ID flow that grants Pulumi two sets of scopes: Microsoft Graph and Azure DevOps." → 🤷 unverifiable
  • L47 in content/docs/integrations/version-control/azure-devops-integration.md "The Microsoft Graph scope lets Pulumi discover the user's Entra tenant and manage the application registration it uses for the integration." → 🤷 unverifiable
  • L48 in content/docs/integrations/version-control/azure-devops-integration.md "The Azure DevOps scope grants access to repositories, projects, and service hooks." → ✅ verified (evidence: entailed by the doc's own permissions table further down; source: repo:content/docs/integrations/version-control/azure-devops-integration.md L67-69)
  • L51 in content/docs/integrations/version-control/azure-devops-integration.md "The Application.ReadWrite.All permission is an admin-consented permission, so if the tenant restricts user consent, an Entra ID administrator must approve it." → ✅ verified (source: content/docs/esc/providers/rotators/azure-app-secret.md)
  • L51 in content/docs/integrations/version-control/azure-devops-integration.md "If a tenant restricts user consent, an Entra ID administrator must approve the Application.ReadWrite.All consent before the integration can be created." → 🤷 unverifiable
  • L54 in content/docs/integrations/version-control/azure-devops-integration.md "Once the Azure DevOps integration exists, Pulumi authenticates to Azure DevOps for deployments using Microsoft Entra ID federated credentials rather than a stored access token." → 🤷 unverifiable
  • L54 in content/docs/integrations/version-control/azure-devops-integration.md "Federated credential configuration for the Azure DevOps integration is done automatically when the integration is connected, requiring no issuer, subject, or audience." → 🤷 unverifiable
  • L58-62 in content/docs/integrations/version-control/azure-devops-integration.md "The application registration's description and notes record the Azure DevOps organization, project, and integration ID, so tenants with several integrations can tell them apart." → 🤷 unverifiable
  • L60 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi creates an application registration named Pulumi in Microsoft Entra ID, registered as single tenant (accounts in the user's directory only)." → 🤷 unverifiable
  • L60-61 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi creates a service principal for the application, added to the Azure DevOps organization with the Basic access level." → 🤷 unverifiable
  • L62 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi creates a federated identity credential on the application, and no client secret or certificate is created." → 🤷 unverifiable
  • L64-70 in content/docs/integrations/version-control/azure-devops-integration.md "The Pulumi Service group in Azure DevOps carries only the permissions the integration needs." → 🤷 unverifiable (evidence: the doc's own table lists the permissions; no independent source)
  • L68-69 in content/docs/integrations/version-control/azure-devops-integration.md "The Pulumi Service group has Git repositories permissions to read, contribute, create branch, create repository, and contribute to pull requests." → 🤷 unverifiable
  • L70 in content/docs/integrations/version-control/azure-devops-integration.md "The Pulumi Service group has Project permissions to view project-level information and test results." → 🤷 unverifiable
  • L72 in content/docs/integrations/version-control/azure-devops-integration.md "During setup, the service principal is temporarily added to Project Administrators in Azure DevOps so it can create the Pulumi Service group and assign its permissions." → 🤷 unverifiable
  • L74 in content/docs/integrations/version-control/azure-devops-integration.md "Deleting the integration removes the service hooks, the service principal and its group, and the Entra ID application registration." → 🤷 unverifiable
  • L77 in content/docs/integrations/version-control/azure-devops-integration.md "Separately from the org-level integration, individual users can complete an OAuth flow under Management > Version control to grant Pulumi access to their Azure DevOps account." → 🤝 matches (source: content/docs/integrations/version-control/bitbucket.md, github-app.md, gitlab.md)
  • L81 in content/docs/integrations/version-control/azure-devops-integration.md "The integration card shows 'Individual access is authorized for this account' once the user has connected individual access, or 'Individual access is recommended for this account' otherwise." → ✅ verified (source: gh search code --owner pulumi "Individual access is authorized for this account")
  • L84 in content/docs/integrations/version-control/azure-devops-integration.md "To remove an individual identity, the user selects their identity on the integration card and chooses Remove Identity." → 🤝 matches (source: gh search code --owner pulumi "Remove Identity")
  • L85 in content/docs/integrations/version-control/azure-devops-integration.md "To disconnect the Azure DevOps account entirely, the user must delete all Azure DevOps integrations first, then select Disconnect from the identity dropdown." → 🤷 unverifiable
  • L90 in content/docs/integrations/version-control/azure-devops-integration.md "After creating an integration, the user can configure pull request behavior, toggling these settings per integration." → ✅ verified (source: cross-checked against github-app.md, gitlab.md, bitbucket.md)
  • L94-95 in content/docs/integrations/version-control/azure-devops-integration.md "The Detailed diff for pull request comments setting is Enabled by default and shows property-level before/after diffs for changed resources in pull request comments." → 🤷 unverifiable
  • L97 in content/docs/integrations/version-control/azure-devops-integration.md "The Detailed diff setting requires pull request comments to be enabled." → 🤷 unverifiable
  • L99 in content/docs/integrations/version-control/azure-devops-integration.md "Deleting an Azure DevOps integration removes the service hooks from Azure DevOps and disconnects all stacks using that integration." → 🤷 unverifiable
  • L105 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi automatically posts comments on Azure DevOps pull requests with the results of any stack changes, including a summary of how many resources were created, updated, or deleted." → 🤷 unverifiable (evidence: the cited URL is the Pulumi Cloud sign-in page)
  • L107 in content/docs/integrations/version-control/azure-devops-integration.md "For review stacks, Azure DevOps pull request comments show the review stack status and outputs instead of a standard preview summary." → 🤝 matches (source: repo:content/docs/deployments/concepts/review-stacks.md, github-app.md)
  • L109 in content/docs/integrations/version-control/azure-devops-integration.md "Draft pull requests do not trigger deployments; Azure DevOps is the exception, since GitLab and Bitbucket treat drafts like any other request and GitHub makes draft comments a toggle." (updated in fa58775) → 🤝 matches for the comparison clauses (GitLab/Bitbucket/GitHub behavior each independently verified elsewhere on this page's siblings); the core Azure-DevOps-specific behavior itself remains 🤷 unverifiable (no external source), but no longer reads as an unexplained cross-sibling mismatch — resolved in fa58775; source: content/docs/integrations/version-control/bitbucket.md L74, gitlab.md L74, github-app.md L145
  • L111 in content/docs/integrations/version-control/azure-devops-integration.md "Neo code reviews, which analyze a pull request and leave inline feedback, are available on GitHub only." → ✅ verified (source: pulumi/docs:content/docs/ai/neo/code-reviews/_index.md)
  • L115 in content/docs/integrations/version-control/azure-devops-integration.md "You can require a commit status to pass by adding a status check branch policy in Azure DevOps." (new in fa58775) → ✅ verified (evidence: Microsoft Learn's branch-policy article confirms configuring a branch policy that requires a third-party service's posted status to pass before a pull request can complete; source: https://learn.microsoft.com/en-us/azure/devops/repos/git/pr-status-policy)
  • L117 in content/docs/integrations/version-control/azure-devops-integration.md "Statuses are posted under the genre pulumi and named after the project, stack, and operation — for example my-project/dev - preview deployment. Each one links back to the full deployment in Pulumi Cloud." → 🤷 unverifiable
  • L119 in content/docs/integrations/version-control/azure-devops-integration.md "Azure DevOps commit statuses are additive: posting a new status with the same genre and name supersedes the previous one, so the latest result is what appears on the commit." → 🤷 unverifiable (evidence: Microsoft Learn's Statuses API docs are consistent with an additive, context-keyed model but don't state supersession explicitly)
  • L121 in content/docs/integrations/version-control/azure-devops-integration.md "Push-to-deploy runs do not post a commit status. Statuses are tied to a pull request's head commit, so they're only produced by pull request deployments." (new in fa58775) → 🤷 unverifiable; still open — see 🚨 Outstanding companion finding at _index.md L28 (evidence: no independent source for this Azure-DevOps-specific narrowing; it's the same underlying question already flagged for GitLab and Bitbucket)
  • L125 in content/docs/integrations/version-control/azure-devops-integration.md "Push-to-deploy automatically runs pulumi up when a commit is pushed to a configured branch, most commonly the default branch." → ✅ verified (source: repo:content/docs/deployments/concepts/triggers.md)
  • L127 in content/docs/integrations/version-control/azure-devops-integration.md "Users can use path filters to limit deployments to commits that change files matching specific glob patterns, e.g. infra/**." → ✅ verified (source: repo:content/docs/deployments/concepts/settings/path-filtering.md)
  • L129 in content/docs/integrations/version-control/azure-devops-integration.md "Azure DevOps delivers tag pushes on the same git.push service hook as branch pushes, so no additional configuration is needed for tag triggers." → ✅ verified (source: repo:content/docs/deployments/concepts/settings/tag-filtering.md)
  • L135 in content/docs/integrations/version-control/azure-devops-integration.md "Choosing the Azure DevOps integration is required if multiple integrations are configured." → 🤷 unverifiable
  • L136 in content/docs/integrations/version-control/azure-devops-integration.md "Azure DevOps repositories are identified as organization/project/repository." → ✅ verified (evidence: matches Azure DevOps's well-known Git URL structure)
  • L139 in content/docs/integrations/version-control/azure-devops-integration.md "See Deployment settings: source for the full set of source options and event toggles." → ✅ verified (source: gh api repos/pulumi/docs contents/content/docs/deployments/concepts/settings)
  • L143 in content/docs/integrations/version-control/azure-devops-integration.md "Review stacks are ephemeral cloud environments created automatically every time a pull request is opened, powered by Pulumi Deployments." → ✅ verified (source: repo:content/docs/deployments/concepts/review-stacks.md)
  • L145 in content/docs/integrations/version-control/azure-devops-integration.md "Review stacks follow the naming convention pr-{organization}-{repository}-{pullRequestId}, for example Azure DevOps organization acme with repository infra and pull request Document best practice for naming resources so as to restrict IAM roles #42 produces stack pr-acme-infra-42." → 🤝 matches (source: content/docs/integrations/version-control/gitlab.md L99)
  • L145 in content/docs/integrations/version-control/azure-devops-integration.md "Review stack stacks are automatically deleted after the destroy completes." → 🌀 framing-drift (framing: source says "destroyed when the pull request is merged or closed," not "deleted"; not promoted since the sentence is copied from the established sibling wording; source: https://www.pulumi.com/docs/deployments/deployments/review-stacks/)
  • L147 in content/docs/integrations/version-control/azure-devops-integration.md "To enable review stacks, the user toggles 'Pull request template' under Stack > Settings > Deploy on the stack to use as a template." → ✅ verified (source: repo:content/docs/deployments/concepts/review-stacks.md)
  • L151-158 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi injects environment variables during Azure DevOps-triggered deployments: PULUMI_PR_NUMBER, PULUMI_CI_PULL_REQUEST_SHA, PULUMI_CI_BRANCH_NAME, PULUMI_CI_TAG_NAME." → ✅ verified (source: gh search code --owner pulumi PULUMI_CI_BRANCH_NAME)
  • L157 in content/docs/integrations/version-control/azure-devops-integration.md "PULUMI_CI_BRANCH_NAME is set on pull request and tag push events and contains the source branch name, or refs/tags/<tag> for a tag push." → 🤷 unverifiable; no longer an isolated oddity — resolved in fa58775 (evidence: the new L160 paragraph gives an internally consistent engineering rationale — the deployment runner clones by commit SHA, leaving a detached HEAD, so the engine falls back to PULUMI_CI_BRANCH_NAME — but the rationale itself is still an internal implementation detail with no public source)
  • L162 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi injects an AZURE_DEV_OPS_TOKEN environment variable that the Pulumi program and pre-run commands can use to authenticate to Azure DevOps." → ✅ verified (source: gh search code --owner pulumi AZURE_DEV_OPS_TOKEN)
  • L162 in content/docs/integrations/version-control/azure-devops-integration.md "Users can supply their own value for AZURE_DEV_OPS_TOKEN through custom environment variables, and an explicit value always overrides the one Pulumi provides." → ✅ verified (source: content/docs/deployments/concepts/settings/environment-variables.md)
  • L164 in content/docs/integrations/version-control/azure-devops-integration.md "## New project wizard" → ➖ not-a-claim
  • L166 in content/docs/integrations/version-control/azure-devops-integration.md "The New Project Wizard supports Azure DevOps as a VCS provider." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md)
  • L170 in content/docs/integrations/version-control/azure-devops-integration.md "Using the New Project Wizard with Azure DevOps, users can choose any deployment method: CLI, Pulumi Deployments (no-code), or Pulumi Deployments (VCS-backed)." → ✅ verified (source: repo:content/docs/idp/concepts/new-project-wizard.md)
  • L172 in content/docs/integrations/version-control/azure-devops-integration.md "When using the VCS-backed deployment method in the New Project Wizard, the wizard turns on deploy-on-push and pull request previews for you; review stacks stay off until enabled on the stack used as a template." (updated in fa58775) → ✅ verified — resolved in fa58775; matches the page's own manual review-stacks toggle in the Review stacks section (evidence: no longer contradicts L147's "To enable review stacks, toggle..."; source: repo:content/docs/integrations/version-control/azure-devops-integration.md L147, L172)
  • L172 in content/docs/integrations/version-control/azure-devops-integration.md "If only one VCS provider is configured, it's selected automatically during project setup." → 🤷 unverifiable
  • L176 in content/docs/integrations/version-control/azure-devops-integration.md "Pulumi scans registered Azure DevOps repositories for subdirectories containing a Pulumi.yaml file, and each subdirectory becomes a selectable template." → ✅ verified (source: repo:content/docs/idp/concepts/organization-templates.md)
  • L178-182 in content/docs/integrations/version-control/azure-devops-integration.md "An Azure DevOps repository is registered as a template source using its web URL in the form https://dev.azure.com/<organization>/<project>/_git/<repository>." → 🤷 unverifiable (evidence: cited URL is the generic Azure DevOps landing page)
  • L186 in content/docs/integrations/version-control/azure-devops-integration.md "The Pulumi Azure DevOps integration posts results back to Azure DevOps regardless of which CI/CD system triggers the run." → 🤷 unverifiable
  • L192 in content/docs/integrations/version-control/azure-devops-integration.md "In Azure DevOps, service hooks for the Pulumi integration can be found by navigating to Project Settings > Service hooks." → ✅ verified (source: https://learn.microsoft.com/en-us/azure/devops/service-hooks/overview)
  • L194 in content/docs/integrations/version-control/azure-devops-integration.md "In the Pulumi Cloud console, users can verify the Azure DevOps integration is connected and shows a valid status under Management > Version control." → 🤷 unverifiable (evidence: cited console URL is login-gated)
  • L198 in content/docs/integrations/version-control/azure-devops-integration.md "Draft pull requests do not trigger deployments (repeated as a troubleshooting check)." → same claim as L109; resolved in fa58775 by the same explanation
  • L202 in content/docs/integrations/version-control/azure-devops-integration.md "If the integration card shows an invalid or disconnected status, deleting the integration and re-creating it by following the installation steps resolves the issue." → 🤷 unverifiable
  • L217 in content/docs/integrations/version-control/azure-devops-integration.md "The error 'Azure DevOps not enabled for tenant' corresponds to code AADSTS650052 and requires an Entra ID administrator to enable Azure DevOps access for the tenant." → ✅ verified (source: gh search code --owner pulumi "AADSTS650052")
  • L218 in content/docs/integrations/version-control/azure-devops-integration.md "A consent error mentioning Application.ReadWrite.All requires an Entra ID administrator to grant consent for the Microsoft Graph permission on the Pulumi application." → ✅ verified (source: repo:content/docs/integrations/version-control/azure-devops-integration.md L51)
  • L219 in content/docs/integrations/version-control/azure-devops-integration.md "The 'Remove all integrations before disconnecting' error is resolved by deleting all Azure DevOps integrations before disconnecting the Azure DevOps identity." → 🤷 unverifiable
  • L63-64 in content/docs/integrations/version-control/bitbucket.md "The Bitbucket integration's 'Detailed diff for pull request comments' setting is enabled by default and shows property-level before/after diffs for changed resources." → 🤷 unverifiable
  • L66 in content/docs/integrations/version-control/bitbucket.md "Deleting a Bitbucket integration removes the webhooks Pulumi created on the connected Bitbucket repositories and disconnects all stacks using that integration." → ✅ verified (source: https://www.pulumi.com/docs/integrations/version-control/bitbucket/)
  • L72 in content/docs/integrations/version-control/bitbucket.md "Pulumi automatically posts comments on Bitbucket pull requests with the results of stack changes." → 🤷 unverifiable (evidence: cited URL is the Pulumi Cloud sign-in page)
  • L74 in content/docs/integrations/version-control/bitbucket.md "Pull request comments from Pulumi are idempotent: updates to the same stack edit the existing comment rather than creating a new one." → ✅ verified (source: https://www.pulumi.com/docs/integrations/version-control/bitbucket/)
  • L74 in content/docs/integrations/version-control/bitbucket.md "Draft pull requests are treated identically to regular pull requests for the purposes of Pulumi's pull request comments." → ✅ verified (source: https://www.pulumi.com/docs/integrations/version-control/bitbucket/)
  • L76 in content/docs/integrations/version-control/bitbucket.md "For review stacks, Pulumi's pull request comments show the review stack status and outputs instead of a standard preview summary." → ✅ verified (source: content/docs/integrations/version-control/gitlab.md)
  • L78 in content/docs/integrations/version-control/bitbucket.md "Pulumi's pull request comments on Bitbucket come from Pulumi Deployments." → ✅ verified (source: repo:content/docs/ai/neo/code-reviews/_index.md)
  • L82 in content/docs/integrations/version-control/bitbucket.md "Push-to-deploy runs do not post a commit status to Bitbucket." → 🤝 matches (source: repo:content/docs/integrations/version-control/gitlab.md L81)
  • L82 in content/docs/integrations/version-control/bitbucket.md "Bitbucket commit status checks posted by Pulumi include a link back to the deployment in Pulumi Cloud." → 🤝 matches (source: repo:content/docs/integrations/version-control/gitlab.md)
  • L145-146 in content/docs/integrations/version-control/github-app.md "The 'Detailed diff for pull request comments' setting shows property-level before/after diffs for changed resources in pull request comments." → ✅ verified (source: https://www.pulumi.com/docs/using-pulumi/continuous-delivery/github-app/)
  • L148 in content/docs/integrations/version-control/github-app.md "Changes to the GitHub app integration settings save automatically." → 🤷 unverifiable; verification incomplete but low-stakes (evidence: unchanged behavior — "Changes save automatically" predates this PR)
  • L148 in content/docs/integrations/version-control/github-app.md "The 'Draft pull request comments' setting is specific to the GitHub app (i.e., not available for other version-control integrations)." → 🤝 matches (source: content/docs/integrations/version-control/gitlab.md, bitbucket.md)
  • L150 in content/docs/integrations/version-control/github-app.md "Neo code reviews are configured separately, under Settings > Neo settings > Code reviews, not in the integration settings described on this page." (updated in fa58775) → ✅ verified — resolved in fa58775 (evidence: content/docs/ai/neo/code-reviews/_index.md L44/L49 and content/docs/ai/neo/settings/_index.md L38 now say the same thing, closing the mismatch flagged in the prior review; source: repo:content/docs/ai/neo/code-reviews/_index.md, repo:content/docs/ai/neo/settings/_index.md)
  • L62-63 in content/docs/integrations/version-control/gitlab.md "The GitLab integration's 'Detailed diff for merge request comments' setting shows property-level before/after diffs for changed resources in merge request comments." → ✅ verified (source: repo:content/docs/integrations/version-control/gitlab.md L62-63)
  • L71 in content/docs/integrations/version-control/gitlab.md "Pulumi's merge request comments include a summary of how many resources were created, updated, or deleted." → 🤷 unverifiable (evidence: cited URL is a sign-in page)
  • L77 in content/docs/integrations/version-control/gitlab.md "Merge request comments in the GitLab integration come from Pulumi Deployments." → ✅ verified (source: https://www.pulumi.com/docs/integrations/version-control/gitlab/)
  • L81 in content/docs/integrations/version-control/gitlab.md "Pulumi's GitLab commit status checks map to GitLab's pending, running, success, and failed states." → ✅ verified (source: https://www.pulumi.com/docs/integrations/version-control/gitlab/)
  • L81 in content/docs/integrations/version-control/gitlab.md "Push-to-deploy runs do not post a commit status to GitLab." → ❌ contradicted; still open — see 🚨 Outstanding (evidence: the current Pulumi GitLab integration docs state: "Pulumi posts commit status checks to GitLab on every deployment, for both push and merge request events"; source: https://www.pulumi.com/docs/integrations/version-control/gitlab/)

🚨 Outstanding in this PR

These must be resolved or refuted before merging.

  • [L28] content/docs/integrations/version-control/_index.md — "On GitLab, Bitbucket, and Azure DevOps, commit statuses are posted for pull request deployments only, not for push-to-deploy runs." — verdict: contradicted; evidence: (escalated from pass1) Pulumi's own GitLab and Bitbucket docs state the opposite: "Pulumi posts commit status checks to GitLab on every deployment, for both push and merge request events" and "Pulumi posts commit status checks to Bitbucket…; source: https://www.pulumi.com/docs/integrations/version-control/gitlab/ ; https://www.pulumi.com/docs/integrations/version-control/bitbucket/

    Still open. The pages quoted as contradicting this are the currently-published GitLab and Bitbucket pages — the text this PR is rewriting — so they aren't independent evidence the new sentence is wrong; the actionable point is that this PR reverses documented behavior on multiple pages at once with no outside corroboration. Commit fa58775 adds the same narrowing to azure-devops-integration.md (now L121: "Push-to-deploy runs do not post a commit status. Statuses are tied to a pull request's head commit, so they're only produced by pull request deployments.") with a plausible technical rationale, but that's still the PR's own reasoning, not an external source. gitlab.md and bitbucket.md are untouched by this commit. Please confirm with the deployments team that push-to-deploy genuinely posts no commit status on GitLab, Bitbucket, and Azure DevOps before merging. See the companion finding on gitlab.md L81.

  • [L81] content/docs/integrations/version-control/gitlab.md — "Push-to-deploy runs do not post a commit status to GitLab." — verdict: contradicted; evidence: (escalated from pass1) The current Pulumi GitLab integration docs state: "Pulumi posts commit status checks to GitLab on every deployment, for both push and merge request events." This directly contradicts the claim that push-to-deploy run…; source: https://www.pulumi.com/docs/integrations/version-control/gitlab/

    Still open, unchanged by this commit (gitlab.md isn't in the diff). Same situation as the _index.md L28 finding above — resolves together once the deployments team confirms the push-to-deploy/commit-status behavior on all three non-GitHub providers.

  • Refresh this review — comment @claude #update-review. Say what you fixed, or which finding you dispute and why; both work in the same mention.
  • Ask for anything else — comment @claude with no hashtag (questions, one-off fixes). Leaves this review untouched.

Important

Please don't hide, resolve, or delete this comment! It breaks things!

📖 How pre-merge review works — the full lifecycle, short-circuits, and escape hatches.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

🤖 Review regenerated on @CamSoper's request.

@github-actions github-actions Bot added review:outstanding-issues Claude review completed; outstanding has author-actionable findings and removed review:in-progress Claude review is currently running labels Aug 6, 2026
@github-actions github-actions Bot added review:stale New commits since last Claude review; refresh on next ready-transition or @claude mention and removed review:outstanding-issues Claude review completed; outstanding has author-actionable findings labels Aug 7, 2026
@jkodroff

jkodroff commented Aug 7, 2026

Copy link
Copy Markdown
Member Author

@claude #update-review

@workprentice workprentice Bot mentioned this pull request Aug 7, 2026
Open
@github-actions github-actions Bot added review:in-progress Claude review is currently running and removed review:stale New commits since last Claude review; refresh on next ready-transition or @claude mention labels Aug 7, 2026
@pulumi pulumi deleted a comment from github-actions Bot Aug 7, 2026
@pulumi pulumi deleted a comment from github-actions Bot Aug 7, 2026
@pulumi-bot

pulumi-bot commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

⚠️ Low-confidence

Review each and resolve as appropriate — these don't block the PR.

Most of what follows is one situation rather than many: the rewritten Azure DevOps page documents setup behavior — what Pulumi creates in your Entra tenant, the service principal and its scoped group, federated credentials, the service hook event names, commit-status naming, injected environment variables — that isn't described on any public page yet, so there was nothing to check it against. That isn't evidence of error, and none of it blocks the PR. It does mean this page's accuracy rests entirely on the author's knowledge of the implementation, so a careful self-check against the shipped feature is worth more here than usual.

  • [L25] content/docs/idp/concepts/no-code-stacks.md — "No-code stacks work with any configured version control provider, widening the previously-documented GitHub-and-Azure-DevOps-only support." — verdict: unverifiable — Author confirmation needed: the two edits agree with each other, so nothing here is internally inconsistent, but this quietly doubles the set of providers no-code stacks support. Worth confirming that GitLab and Bitbucket no-code stacks actually work before the restriction disappears from the docs.

  • [L126] content/docs/ai/neo/permissions/_index.md — "When a Neo task needs individual GitHub access that the user hasn't connected, Neo posts a nudge prompting the user to grant it." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L34-37] content/docs/integrations/version-control/azure-devops-integration.md — "After authorizing, users select the Azure DevOps organization and project they want to integrate with, then configure integration settings." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L36] content/docs/integrations/version-control/azure-devops-integration.md — "Setting up the integration requires selecting Authorize Azure DevOps and completing the Microsoft Entra ID sign-in." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L39] content/docs/integrations/version-control/azure-devops-integration.md — "Pulumi registers service hooks in the Azure DevOps project for the git.push, git.pullrequest.created, git.pullrequest.updated, and git.pullrequest.merged events." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L41] content/docs/integrations/version-control/azure-devops-integration.md — "Each Azure DevOps integration maps one Pulumi organization to one Azure DevOps organization and project pair." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L45-48] content/docs/integrations/version-control/azure-devops-integration.md — "Authorization for the Azure DevOps integration is a single Microsoft Entra ID flow that grants Pulumi two sets of scopes: Microsoft Graph and Azure DevOps." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L47] content/docs/integrations/version-control/azure-devops-integration.md — "The Microsoft Graph scope lets Pulumi discover the user's Entra tenant and manage the application registration it uses for the integration." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L51] content/docs/integrations/version-control/azure-devops-integration.md — "If a tenant restricts user consent, an Entra ID administrator must approve the Application.ReadWrite.All consent before the integration can be created." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L54] content/docs/integrations/version-control/azure-devops-integration.md — "Once the Azure DevOps integration exists, Pulumi authenticates to Azure DevOps for deployments using Microsoft Entra ID federated credentials rather than a stored access token." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L54] content/docs/integrations/version-control/azure-devops-integration.md — "Federated credential configuration for the Azure DevOps integration is done automatically when the integration is connected, requiring no issuer, subject, or audience." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L58-62] content/docs/integrations/version-control/azure-devops-integration.md — "The application registration's description and notes record the Azure DevOps organization, project, and integration ID, so tenants with several integrations can tell them apart." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L60] content/docs/integrations/version-control/azure-devops-integration.md — "Pulumi creates an application registration named Pulumi in Microsoft Entra ID, registered as single tenant (accounts in the user's directory only)." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L60-61] content/docs/integrations/version-control/azure-devops-integration.md — "Pulumi creates a service principal for the application, added to the Azure DevOps organization with the Basic access level." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L62] content/docs/integrations/version-control/azure-devops-integration.md — "Pulumi creates a federated identity credential on the application, and no client secret or certificate is created." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L64-70] content/docs/integrations/version-control/azure-devops-integration.md — "The Pulumi Service group in Azure DevOps carries only the permissions the integration needs." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L68-69] content/docs/integrations/version-control/azure-devops-integration.md — "The Pulumi Service group has Git repositories permissions to read, contribute, create branch, create repository, and contribute to pull requests." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L70] content/docs/integrations/version-control/azure-devops-integration.md — "The Pulumi Service group has Project permissions to view project-level information and test results." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L72] content/docs/integrations/version-control/azure-devops-integration.md — "During setup, the service principal is temporarily added to Project Administrators in Azure DevOps so it can create the Pulumi Service group and assign its permissions." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L74] content/docs/integrations/version-control/azure-devops-integration.md — "Deleting the integration removes the service hooks, the service principal and its group, and the Entra ID application registration." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L85] content/docs/integrations/version-control/azure-devops-integration.md — "To disconnect the Azure DevOps account entirely, the user must delete all Azure DevOps integrations first, then select Disconnect from the identity dropdown." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L94-95] content/docs/integrations/version-control/azure-devops-integration.md — "The Detailed diff for pull request comments setting is Enabled by default." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L97] content/docs/integrations/version-control/azure-devops-integration.md — "The Detailed diff setting requires pull request comments to be enabled." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L99] content/docs/integrations/version-control/azure-devops-integration.md — "Deleting an Azure DevOps integration removes the service hooks from Azure DevOps and disconnects all stacks using that integration." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L105] content/docs/integrations/version-control/azure-devops-integration.md — "Pulumi automatically posts comments on Azure DevOps pull requests with the results of any stack changes." — verdict: unverifiable; evidence: the cited URL is the Pulumi Cloud sign-in page, not documentation — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L117] content/docs/integrations/version-control/azure-devops-integration.md — "Statuses are posted under the genre pulumi and named after the project, stack, and operation — for example my-project/dev - preview deployment. Each one links back to the full deployment in Pulumi Cloud." — verdict: unverifiable — Author confirmation needed: this is internal commit-status naming behavior not documented publicly; not a blocker, please confirm it matches the shipped implementation before merging.

  • [L119] content/docs/integrations/version-control/azure-devops-integration.md — "Azure DevOps commit statuses are additive: posting a new status with the same genre and name supersedes the previous one, so the latest result is what appears on the commit." — verdict: unverifiable; evidence: Microsoft Learn's Azure DevOps Git Statuses API docs are consistent with an additive, context-keyed model but don't state supersession explicitly — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L135] content/docs/integrations/version-control/azure-devops-integration.md — "Choosing the Azure DevOps integration is required if multiple integrations are configured." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L145] content/docs/integrations/version-control/azure-devops-integration.md — "Review stacks follow the naming convention pr-{organization}-{repository}-{pullRequestId}." — verdict: unverifiable; evidence: the GitLab sibling doc states the identical convention and example for its own provider — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L145] content/docs/integrations/version-control/azure-devops-integration.md — "Review stack stacks are automatically deleted after the destroy completes." — verdict: framing-drift; framing: the review-stacks concept page says stacks are "destroyed when the pull request is merged or closed," not "deleted," and Pulumi elsewhere keeps destroy (resources removed) distinct from stack rm (record removed) — not promoted, because the sentence is copied verbatim from the established sibling wording (gitlab.md L98 already says the same thing); fix on both pages at once if it's inaccurate.

  • [L172] content/docs/integrations/version-control/azure-devops-integration.md — "If only one VCS provider is configured, it's selected automatically during project setup." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L178-182] content/docs/integrations/version-control/azure-devops-integration.md — "An Azure DevOps repository is registered as a template source using its web URL in the form https://dev.azure.com/<organization>/<project>/_git/<repository>." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L186] content/docs/integrations/version-control/azure-devops-integration.md — "The Pulumi Azure DevOps integration posts results back to Azure DevOps regardless of which CI/CD system triggers the run." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L194] content/docs/integrations/version-control/azure-devops-integration.md — "In the Pulumi Cloud console, users can verify the Azure DevOps integration is connected and shows a valid status under Management > Version control." — verdict: unverifiable; evidence: the cited console URL is login-gated — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L202] content/docs/integrations/version-control/azure-devops-integration.md — "If the integration card shows an invalid or disconnected status, deleting the integration and re-creating it resolves the issue." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L219] content/docs/integrations/version-control/azure-devops-integration.md — "The 'Remove all integrations before disconnecting' error is resolved by deleting all Azure DevOps integrations before disconnecting the Azure DevOps identity." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L63-64] content/docs/integrations/version-control/bitbucket.md — "The Bitbucket integration's 'Detailed diff for pull request comments' setting is enabled by default." — verdict: unverifiable — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L72] content/docs/integrations/version-control/bitbucket.md — "Pulumi automatically posts comments on Bitbucket pull requests with the results of stack changes." — verdict: unverifiable; evidence: the cited URL is the Pulumi Cloud sign-in page — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

  • [L148] content/docs/integrations/version-control/github-app.md — "Changes to the GitHub app integration settings save automatically." — verdict: unverifiable; evidence: this claim describes Pulumi's own product UI behavior with no independent public source — Verification incomplete: it's also unchanged behavior — "Changes save automatically" was already in the current page before this PR — so it's low-stakes.

  • [L71] content/docs/integrations/version-control/gitlab.md — "Pulumi's merge request comments include a summary of how many resources were created, updated, or deleted." — verdict: unverifiable; evidence: the cited URL is a sign-in page — Author confirmation needed: not a blocker; please confirm it matches the shipped implementation before merging.

Style findings

Found by pattern-based linting; Findings may be false positives.

Click each filename to expand.

content/docs/ai/neo/settings/_index.md (1 issue: 1 punctuation)
  • line 38: [style] punctuation — Commas and periods go inside quotation marks.
content/docs/idp/concepts/new-project-wizard.md (2 issues: 2 wordiness)
  • line 66: [style] wordiness — 'authorize' is too wordy.
  • line 72: [style] wordiness — 'authorize' is too wordy.
content/docs/integrations/version-control/azure-devops-integration.md (3 issues: 1 wordiness, 1 weasel word, 1 style)
  • line 36: [style] wordiness — 'Authorize' is too wordy.
  • line 60: [style] weasel word — 'several' is a weasel word!
  • line 64: [style] style — 'Pulumi Service' is the former name of Pulumi Cloud. Use the specific current product for this context -- e.g. Pulumi Cloud, Pulumi ESC, or Pulumi Deployments (STYLE-GUIDE.md §Product Names). (False positive: this is the literal name of the Azure DevOps group Pulumi creates — Pulumi Service (<your-pulumi-org>) — not a product reference. It has to match what the reader sees in Azure DevOps.)

💡 Pre-existing issues in touched files (optional)

No pre-existing issues in touched files.

✅ Resolved since last review

  • [L146→L150] content/docs/integrations/version-control/github-app.md and the Neo docs — "Neo code reviews are configured separately, under Settings > Neo > Code reviews, not in the integration settings described on this page." (resolved in fa58775) — the mismatch is gone: all four touched pages (github-app.md L150, code-reviews/_index.md L44/L49, settings/_index.md L38) now consistently point to Settings > Neo settings > Code reviews, and no page still tells readers to look in the GitHub App integration settings.

  • [L165→L172] content/docs/integrations/version-control/azure-devops-integration.md — "When using the VCS-backed deployment method in the New Project Wizard, the wizard configures deploy-on-push, pull request previews, and review stacks automatically." (resolved in fa58775) — the sentence now reads "the wizard turns on deploy-on-push and pull request previews for you. Review stacks stay off — enable them afterward on the stack you want to use as a template," which matches the page's own manual review-stacks toggle instructions instead of contradicting them.

  • [L108→L109] content/docs/integrations/version-control/azure-devops-integration.md — "Draft pull requests do not trigger deployments." (resolved in fa58775) — no longer reads as an unexplained divergence from every sibling provider. The sentence now explicitly names Azure DevOps as the exception and cross-references GitLab, Bitbucket, and GitHub's differing behavior, which also covers the identical troubleshooting-checklist repeat (now L198).

  • [L150-153→L156-160] content/docs/integrations/version-control/azure-devops-integration.md — "PULUMI_CI_BRANCH_NAME is set on pull request and tag push events and contains the source branch name, or refs/tags/<tag> for a tag push." (resolved in fa58775) — a new paragraph explains the mechanism (deployment runner clones by commit SHA, leaving a detached HEAD, so the engine falls back to PULUMI_CI_BRANCH_NAME), which is an internally consistent rationale for the previously-odd-looking fallback.

📜 Review history

  • 2026-08-06T23:49:01Z — Azure DevOps page rewrite reviewed: 5 blockers (Neo code-reviews configuration path contradicted by two other docs pages, review stacks described as both automatic and manual, draft-PR behavior diverging from every sibling provider, and the commit-status narrowing on GitLab/Bitbucket/Azure DevOps needing confirmation); the bulk of the new Entra/service-principal setup detail is not publicly documented and needs author confirmation. (2901745)
  • 2026-08-07T15:57:33Z — re-reviewed after fix push (1 new commit, fa58775): 3 of 5 blockers resolved (Neo code-reviews configuration path, wizard/review-stacks claim, draft-PR divergence now explained as an intentional Azure DevOps exception), plus the PULUMI_CI_BRANCH_NAME tag-push oddity flagged as low-confidence; 2 blockers remain open (commit-status narrowing on GitLab/Bitbucket/Azure DevOps still needs deployments-team confirmation). (fa58775)
  • 2026-08-07T16:19:49Z — re-verified on request (pulumi-bot): no new commits since fa58775; the 2 outstanding commit-status-narrowing findings and the low-confidence items are unchanged.
  • Refresh this review — comment @claude #update-review. Say what you fixed, or which finding you dispute and why; both work in the same mention.
  • Ask for anything else — comment @claude with no hashtag (questions, one-off fixes). Leaves this review untouched.

Important

Please don't hide, resolve, or delete this comment! It breaks things!

📖 How pre-merge review works — the full lifecycle, short-circuits, and escape hatches.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🤖 Review updated on @pulumi-bot's request.

@github-actions github-actions Bot added review:outstanding-issues Claude review completed; outstanding has author-actionable findings and removed review:in-progress Claude review is currently running labels Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CamSoper CamSoper CamSoper approved these changes

Assignees

No one assigned

Labels

review:outstanding-issues Claude review completed; outstanding has author-actionable findings

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Refresh ADO VCS Integration Page

3 participants

@jkodroff @pulumi-bot @CamSoper