docs(resource-labels): document AWS resource labels for cost observability

platform-cloud/docs/monitoring/cloud-costs.md

@@ -11,12 +11,18 @@ Monitor cloud costs to manage resources effectively and prevent unexpected expen
 
 Use [Resource labels](../resource-labels/overview) in your compute environments to annotate and track the actual cloud resources consumed by a pipeline run. Resource labels are applied to the resources spawned during a run and sent to your cloud provider in `key=value` format.
 
+For full cost accounting — including storage and networking — combine resource labels with your cloud provider's native cost tools rather than custom wrapper scripts that dedicate whole instances to single jobs. See [Include Seqera resource labels in AWS billing reports](../resource-labels/overview#include-seqera-resource-labels-in-aws-billing-reports).
+
 ## Seqera cost estimate
 
-Run details include an **Estimated cost** display. This is the total estimated compute cost of all tasks in the pipeline run.
+The [run details](../monitoring/run-details) page includes an **Estimated cost** display on the **Metrics** tab. This is the total estimated compute cost of all tasks in the pipeline run. Per-task cost — along with the machine type, price model, and requested CPU and memory used to derive it — is shown in each task's **Metrics** details.
 
 The Seqera cost estimator should only be used for at-a-glance heuristic purposes. For accounting and legal cost reporting, use resource labels and leverage your compute platform's native cost reporting tools.
 
+:::tip
+Per-task metrics, including estimated cost, are also available programmatically through the Platform API for building custom cost dashboards across runs. See the [describe workflow task](https://docs.seqera.io/platform-api/describe-workflow-task) and [list workflow tasks](https://docs.seqera.io/platform-api/list-workflow-tasks) API endpoints.
+:::
+
 The compute cost of a task is computed as follows:
 
 $$

platform-cloud/docs/resource-labels/overview.md

@@ -27,6 +27,10 @@ Resource labels can be applied to compute environments, pipelines, actions, runs
 Seqera applies resource labels to cloud resources in one direction only. Any tags changed or deleted directly in your cloud environment will not be reflected in Seqera.
 :::
 
+:::note
+Resource labels are normally created and managed in Seqera at the workspace, compute environment, pipeline, action, run, and Studio levels. Advanced users can also define resource labels directly in Nextflow configuration using the [`resourceLabels`](https://docs.seqera.io/nextflow/reference/process#resourcelabels) process directive, set per process or globally with `process.resourceLabels`. Labels defined this way are applied by Nextflow at task submission and execution time.
+:::
+
 ### Resource labels applied to compute environments
 
 Resource labels can be applied to all cloud compute environments. Cloud resources are tagged when a pipeline run is launched or a Studio is started with those resource labels applied.
@@ -165,9 +169,17 @@ The following resources are tagged using the resource labels associated with the
 
 At execution time, when jobs are submitted to Batch, the requests are set up to propagate tags to all the instances and volumes created by the head job.
 
+:::caution
+Only compute environments and their associated queues **created by Batch Forge** are tagged with your resource labels automatically. AWS Batch compute environments, job queues, or other resources you create **manually outside of Seqera** don't inherit these tags, so their costs aren't attributable in AWS Cost Explorer or your data exports until you tag them yourself. If you run a mix of Forge-created and manually created queues, add the relevant cost-allocation tag (for example, `project=<value>`) to the manually created resources in the AWS console.
+:::
+
 The [IAM permissions](../compute-envs/aws-batch#required-platform-iam-permissions) contain the roles needed for Batch Forge-created AWS Batch compute environments to tag AWS resources. Specifically, the required roles are `iam:TagRole`, `iam:TagInstanceProfile`, and `batch:TagResource`.
 
-To view and manage the resource labels applied to AWS resources by Seqera and Nextflow, go to the [AWS Tag Editor](https://docs.aws.amazon.com/tag-editor/latest/userguide/find-resources-to-tag.html) (as an administrative user) and follow these steps:
+#### Verify resource label propagation to AWS
+
+Resource labels applied in Seqera surface as AWS tags with the same `key=value` on the resources listed above. For example, a resource label `project=rnaseq` on a Batch Forge compute environment is applied as the AWS tag `project=rnaseq` on the Batch compute environment, job queues, and EC2 instances at creation time, and on the jobs and job definitions submitted for each run. A dynamic resource label such as `platformRun=${workflowId}` is applied as a tag like `platformRun=12345abcde` on the jobs and job definitions spawned by that run.
+
+To view, manage, and verify the resource labels applied to AWS resources by Seqera and Nextflow, go to the [AWS Tag Editor](https://docs.aws.amazon.com/tag-editor/latest/userguide/find-resources-to-tag.html) (as an administrative user) and follow these steps:
 
 1. Under **Find resources to tag**, search for the resource label key and value in the relevant search fields under **Tags**. Your search can be further refined by AWS region and resource type.
 1. Select **Search resources**. **Resource search results** display all the resources tagged with your given resource label key and/or value.
@@ -176,6 +188,10 @@ To view and manage the resource labels applied to AWS resources by Seqera and Ne
 
 To include the cost information associated with your resource labels in your AWS billing reports, you need to activate cost allocation tags. The method for viewing costs differs between static and dynamic resource labels:
 
+:::tip
+Resource labels combined with your cloud provider's native cost tools are the recommended way to achieve full cost accounting — including compute, storage, and networking — for your pipeline runs. Avoid custom wrapper scripts that dedicate an entire EC2 instance to a single job to attribute cost: this pattern is incompatible with AWS Batch's shared-instance scheduling model and typically increases cost. Tag your resources with resource labels and report on them in AWS Cost Explorer or your data exports instead.
+:::
+
 **For static resource labels**: Because static resource labels have fixed values at compute environment creation time or workflow submission time, they are applied to static resources including Batch compute environments and EC2 instances. Static resource label costs can be viewed in AWS Cost Explorer, [Data Exports](https://docs.aws.amazon.com/cur/latest/userguide/what-is-data-exports.html), and QuickSight dashboards.
 
 **For dynamic resource labels**: Dynamic resource labels are only propagated at workflow submission and execution time. This means only jobs and job definitions (for AWS Batch compute environments), and EC2 instances (for AWS Cloud compute environments) spawned at runtime are tagged with the unique workflow identifiers. You must [enable split cost allocation data](https://docs.aws.amazon.com/cur/latest/userguide/enabling-split-cost-allocation-data.html) and view costs in [AWS Data Exports](https://docs.aws.amazon.com/cur/latest/userguide/what-is-data-exports.html) and Cost and Usage Reports (CUR). Dynamic resource label costs are not visible in AWS Cost Explorer, which does not support split cost allocation data.
@@ -201,6 +217,14 @@ To include the cost information associated with your resource labels in your AWS
    - Query reports using Amazon Athena or visualize in Amazon QuickSight dashboards (requires a QuickSight subscription)
    - For a complete walkthrough, see our [guide to AWS cost tracking with resource labels](https://seqera.io/blog/aws-labels-cost-tracking/)
 
+#### Verify cost data in your AWS billing reports
+
+After you activate cost-allocation tags, cost data for your labeled resources typically appears in Cost Explorer and your data exports (CUR/Parquet) only after a **24–48 hour delay**. To confirm that your labels and their costs landed, inspect the data export directly — for example, query the Parquet files with Amazon Athena, or download and open them — and check that your resource-label tag keys are present and associated with non-zero costs.
+
+:::caution
+AWS Cost and Usage Reports normalize tag characters. In CUR (version 2), colons (`:`) are rewritten as underscores (`_`), and mixed- or upper-case characters are lowercased and separated with underscores. For example, a tag key `costCenter` can appear as `cost_center`, and `team:genomics` as `team_genomics`, in the export. Design your resource-label keys and values so they remain unambiguous after this normalization, and account for it in downstream Athena or QuickSight queries.
+:::
+
 #### AWS limitations
 
 - Resource label keys and values must contain a minimum of 2 and a maximum of 39 alphanumeric characters (each), separated by dashes or underscores.

platform-cloud/docs/troubleshooting_and_faqs/resource-labels.md

@@ -32,3 +32,25 @@ Resources launch without the expected tags, or dynamic label values are empty. T
 - Verify that resource labels are applied to the correct compute environment.
 - Check that workflows use the tagged compute environment.
 - For dynamic resource labels, ensure variables use the correct syntax: `${sessionId}`, `${userName}`, or `${workflowId}`.
+
+#### Costs missing for manually created AWS Batch queues
+
+Costs for some AWS Batch runs never appear in Cost Explorer or your data exports, even though resource labels are applied. This happens when the compute environment or job queue was created manually, outside of Batch Forge, and so doesn't inherit Seqera's cost-allocation tags. To resolve:
+
+- Add the relevant cost-allocation tag (for example, `project=<value>`) to the manually created compute environments, job queues, and related resources in the AWS console.
+- Prefer Batch Forge-created compute environments where possible, so tags propagate automatically.
+
+#### Cost data missing from the AWS data export
+
+Resource labels are applied and cost-allocation tags are activated, but split or unblended cost fields are missing or show zero in your data export. To resolve:
+
+- Confirm that the cost-allocation tag keys are activated in the **AWS Billing and Cost Management console** of the payer (billing) account.
+- Enable [split cost allocation data](https://docs.aws.amazon.com/cur/latest/userguide/enabling-split-cost-allocation-data.html) in your Cost and Usage Report preferences — without it, downstream reporting returns blended-only or zero values.
+- Allow a 24–48 hour delay for cost data to appear, then inspect the export (for example, query the Parquet files with Amazon Athena) to confirm the tag keys and their costs are present.
+
+#### Resource label tag keys look different in the AWS Cost and Usage Report
+
+Tag keys or values in the AWS Cost and Usage Report (CUR) don't match the resource labels you applied, breaking Athena or QuickSight queries. This is expected CUR normalization: in CUR (version 2), colons (`:`) are rewritten as underscores (`_`), and mixed- or upper-case characters are lowercased and separated with underscores (for example, `costCenter` becomes `cost_center`). To resolve:
+
+- Design resource-label keys and values that remain unambiguous after normalization.
+- Reference the normalized key names in your downstream Athena or QuickSight queries.

platform-enterprise_docs/monitoring/cloud-costs.md

@@ -12,12 +12,18 @@ Monitor cloud costs to manage resources effectively and prevent unexpected expen
 
 Use [Resource labels](../resource-labels/overview) in your compute environments to annotate and track the actual cloud resources consumed by a pipeline run. Resource labels are applied to the resources spawned during a run and sent to your cloud provider in `key=value` format.
 
+For full cost accounting — including storage and networking — combine resource labels with your cloud provider's native cost tools rather than custom wrapper scripts that dedicate whole instances to single jobs. See [Include Seqera resource labels in AWS billing reports](../resource-labels/overview#include-seqera-resource-labels-in-aws-billing-reports).
+
 ## Seqera cost estimate
 
-Run details include an **Estimated cost** display. This is the total estimated compute cost of all tasks in the pipeline run.
+The [run details](../monitoring/run-details) page includes an **Estimated cost** display on the **Metrics** tab. This is the total estimated compute cost of all tasks in the pipeline run. Per-task cost — along with the machine type, price model, and requested CPU and memory used to derive it — is shown in each task's **Metrics** details.
 
 The Seqera cost estimator should only be used for at-a-glance heuristic purposes. For accounting and legal cost reporting, use resource labels and leverage your compute platform's native cost reporting tools.
 
+:::tip
+Per-task metrics, including estimated cost, are also available programmatically through the Platform API for building custom cost dashboards across runs. See the [describe workflow task](https://docs.seqera.io/platform-api/describe-workflow-task) and [list workflow tasks](https://docs.seqera.io/platform-api/list-workflow-tasks) API endpoints.
+:::
+
 The compute cost of a task is computed as follows:
 
 $$