Skip to content

Preserve URI scheme when resolving Xenium bundle child paths#190

Merged
heylf merged 1 commit into
nf-core:devfrom
rnaidu-seqera:fix/preserve-uri-scheme-bundle-paths
Jul 22, 2026
Merged

Preserve URI scheme when resolving Xenium bundle child paths#190
heylf merged 1 commit into
nf-core:devfrom
rnaidu-seqera:fix/preserve-uri-scheme-bundle-paths

Conversation

@rnaidu-seqera

@rnaidu-seqera rnaidu-seqera commented Jul 20, 2026

Copy link
Copy Markdown

Problem

Running the pipeline on an object-storage work directory (S3 on AWS Batch) aborts during Xenium bundle validation, even though the UNTAR step succeeds:

❌ Xenium bundle does not exist: /<bucket>/work/7a/a1c7ff.../test_run
java.nio.file.NoSuchFileException: /<bucket>/work/.../test_run/gene_panel.json

Note the paths have no s3:// scheme — the bucket name became the first segment of a local path.

Root cause

Bundle child paths were built from Path.toString(). For remote filesystems, nextflow.cloud.aws.nio.S3Path.toString() returns /<bucket>/<key> and drops the s3:// scheme. The scheme-less string was then re-parsed with file(...), which resolved it on the local filesystem of the head node, where the files don't exist.

This is invisible on local / shared-filesystem executors (e.g. nf-core CI), because there toString() already yields a valid local path. It only manifests on object storage.

Affected sites in workflows/spatialaxe.nf:

  • UNTAR output staging (-profile test)
  • required/optional bundle-file checks (affects production runs too)
  • transcripts.parquet, experiment.xenium, gene_panel.json, and the morphology-focus image resolution

Fix

Use scheme-preserving APIs instead of toString():

  • Path.resolve(child) for the required/optional file existence checks (keeps the filesystem provider);
  • bundle.toUriString() / file(bundle).toUriString() for the UNTAR staging value and the string-concatenation sites.

No behaviour change on local / shared-filesystem executors.

Validation

  • Each rebuilt path resolves to the correct provider: S3Path for s3:// inputs (including the trailing-slash morphology_focus/ directory) and UnixPath for local inputs.
  • nextflow lint passes with no new errors.
  • A minimal reproducer confirms the failure on an S3 work dir before the change and success after.

PR checklist

  • CHANGELOG.md is updated.
  • Documentation update (n/a — no user-facing parameter/behaviour change).
  • nf-test / CI green (please run in nf-core CI).

Bundle validation built child paths from `Path.toString()`, which drops
the URI scheme (e.g. `s3://`) for remote filesystems. On object-storage
work dirs (S3 on AWS Batch) the scheme-less path was resolved on the
local filesystem, so every child-file check failed with
`Xenium bundle does not exist` / `NoSuchFileException`, even though the
UNTAR step succeeded.

Use scheme-preserving `Path.resolve()` / `toUriString()` instead of
`toString()` at all bundle-path construction sites. This fixes both the
`-profile test` staging path (UNTAR output) and production runs where the
samplesheet supplies an object-storage bundle. No change on local /
shared-filesystem executors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@heylf heylf added this to the 1.1.0 milestone Jul 22, 2026

@heylf heylf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you so much @rnaidu-seqera for your fix and work ❤️

@heylf
heylf merged commit f5c2d46 into nf-core:dev Jul 22, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants