docs: Rewrite Azure Batch tutorial for native Batch Forge support - #1674
docs: Rewrite Azure Batch tutorial for native Batch Forge support#1674adamrtalbot wants to merge 6 commits into
Conversation
Batch Forge now provisions most advanced Azure Batch topologies natively (separate head/worker pools, dedicated head job resources, autoscaling, Entra authentication, and private networking). The manual-setup tutorials taught Nextflow-config workarounds that are now redundant, and the Nextflow autopool option was misleading since Platform disables allowPoolCreation. - Restructure both tutorials as a native-first Batch Forge walkthrough - Keep genuinely manual content: low-priority (spot) worker nodes (Forge is dedicated-only) and pre-existing/custom pools - Remove deprecated autopool options; replace with a deprecation note - Add reference cross-links from compute-envs/azure-batch.md to the tutorial Scope: cloud + enterprise latest only; versioned snapshots untouched. Generated by Claude Code
✅ Deploy Preview for seqera-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
@claude please port this to the current versioned docs (26.1) as well |
|
Claude finished @justinegeffen's task in 2m 50s —— View job Porting Azure Batch tutorial to versioned docs (26.1)
I ported both changes to
Net: Warning I could not push the commit. Every push attempt (both The |
Applies the same native-Batch-Forge rewrite to the versioned 26.1 snapshot so the current release surface matches master. Preserves the versioned-tree convention of using an absolute URL for the Seqera CLI installation link. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
justinegeffen
added
1. Editor review
| 1. For **Managed Identity Client ID**, enter the client ID of the managed identity created earlier. | ||
|
|
||
| Duplicate the `hello-world-autopool-low-priority` pipeline and save it as `hello-world-entra-mi`. | ||
| 1. Under the managed identity fields, enter the client ID of the managed identity created earlier. |
There was a problem hiding this comment.
The MI step here is now incomplete for Batch Forge. The compute-environment form actually exposes four MI fields (a client ID + a resource ID for both head and worker pool), and resource IDs are required when using Batch Forge because Forge passes them to BatchManager at pool-create time to attach the UAMIs. Worth mirroring what the reference doc landed on (via #1315 / #1396) so readers pick up the right fields.
| 1. Under the managed identity fields, enter the client ID of the managed identity created earlier. | |
| 1. Under the managed identity fields, enter the **client ID** and **resource ID** of the managed identity. When using Batch Forge you can also specify a separate pool managed identity by populating the corresponding pool fields — see [Entra service principal and managed identity][azure-batch-entra] in the Azure Batch reference for the four fields and when each is required. |
(Same fix needed in platform-cloud/docs/enterprise/advanced-topics/manual-azure-batch-setup.md line 129 and the versioned 26.1 copy line 152.)
| - Exchange of data can be faster and cheaper than routing over the public internet. | ||
|
|
||
| In practice, you are more likely to connect an Azure Batch Node pool to an existing virtual network that is connected to other resources, such as Seqera Platform or the Azure Storage Account. In this instance, connecting it to a VNet with public internet access will route the network traffic via the virtual network while still allowing you to perform every action. | ||
| To connect the Batch pool nodes to a private Azure VNet, enter a **Subnet ID** when creating the Batch Forge compute environment. Batch Forge attaches the subnet to both the head and worker pools, so worker nodes inherit the same networking automatically. |
There was a problem hiding this comment.
This is accurate for Batch Forge specifically — but a reader who later follows the "Advanced. Use a pre-existing Azure Batch pool" section won't get that automatic subnet inheritance and has to attach the VNet to each pool themselves. Worth adding a one-liner in the Advanced section (line ~189) so readers know the Forge behaviour doesn't carry over.
| Select **Launch** next to the `hello-world-entra-mi` pipeline in your workspace Launchpad to complete the launch form and launch the workflow. The pipeline runs as before, but uses the managed identity to authenticate to Azure Batch and Storage. No keys are exchanged. | ||
|
|
||
| :::note | ||
| You can also use User Subscription mode instead of Batch Managed here, but this is beyond the scope of this tutorial. |
There was a problem hiding this comment.
This note is preserved from the old doc but reads oddly under a Batch Forge-first flow: Forge always creates pools in Batch Managed allocation mode (via BatchManager), not User Subscription. If we want to keep the pointer, it belongs in the "Advanced. Use a pre-existing Azure Batch pool" section rather than immediately after a Forge walkthrough — otherwise readers may think Forge supports both modes. Suggested: drop the callout here, or move it down.
(Same in versioned 26.1 line 160.)
| [nextflow-working-directory]: https://docs.seqera.io/nextflow/cache-and-resume#work-directory | ||
| [create-org-workspace]: ../../getting-started/workspace-setup | ||
| [add-pipeline]: ../../getting-started/quickstart-demo/add-pipelines#add-from-the-launchpad | ||
| [azure-batch-reference]: ../../compute-envs/azure-batch |
There was a problem hiding this comment.
Nit — link definitions in the three variants of this tutorial disagree on the file-extension convention:
platform-cloud/docs/enterprise/advanced-topics/manual-azure-batch-setup.mduses../../compute-envs/azure-batch.md(with.md)- This file (Enterprise) uses
../../compute-envs/azure-batch(no.md) platform-enterprise_versioned_docs/version-26.1/...also uses no.md
Docusaurus tolerates both, but worth aligning across the three copies (dropping .md from the Cloud one seems cleanest since two files already do it that way).
| 1. For **Managed Identity Client ID**, enter the client ID of the managed identity created earlier. | ||
|
|
||
| Duplicate the `hello-world-autopool-low-priority` pipeline and save it as `hello-world-entra-mi`. | ||
| 1. Under the managed identity fields, enter the client ID of the managed identity created earlier. |
There was a problem hiding this comment.
One more thing in this Entra section — the SP roles link a few lines up ("Assign roles to the service principal") points at a generic Microsoft page and doesn't tell the reader which roles to pick. We spent a while in #1315 / #1396 nailing this down (there is no built-in Azure Batch Contributor; the correct one is Azure Batch Data Contributor). Worth adding a pointer to the reference so readers don't re-derive it:
See the Service principal section of the Azure Batch reference for the specific roles the SP needs —
Azure Batch Data Contributoron the Batch account,Storage Blob Data Contributoron the storage account,Managed Identity Operatoron each UAMI, and — situationally —Network Contributoron the VNet.
(Same addition useful in the Cloud and versioned 26.1 copies.)
|
Small self-correction on my review — I retracted my earlier comment about the head-pool Autoscale tip (it's now deleted). Turns out the compute-env form does expose a per-pool |
Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
3 participants
Context
The
manual-azure-batch-setup.mdtutorials were written when advanced Azure Batch topologies required manual Nextflow config and manual Azure Portal pool creation. Seqera Platform now provisions most of these natively via Batch Forge and the compute-environment form.Verified against
seqera/platform:process.queueallowPoolCreation=false)Batch Forge provisions dedicated nodes only, so low-priority (spot) nodes remain a genuinely manual step and are preserved.
Changes
compute-envs/azure-batch.mdto the tutorial (previously only linked one way).Versioned snapshots (24.2–26.1) left untouched.
🤖 Generated with Claude Code