Skip to content

genome: --genomeTransformOutput SAM reports original coordinates - #160

Open
BenjaminDEMAILLE wants to merge 5 commits into
scverse:mainfrom
BenjaminDEMAILLE:bd/genome-transform-output
Open

genome: --genomeTransformOutput SAM reports original coordinates#160
BenjaminDEMAILLE wants to merge 5 commits into
scverse:mainfrom
BenjaminDEMAILLE:bd/genome-transform-output

Conversation

@BenjaminDEMAILLE

@BenjaminDEMAILLE BenjaminDEMAILLE commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

--genomeTransformOutput SAM: report alignments in the original genome's coordinates.

What changed

--genomeTransformType Haploid bakes a VCF's variants into the genome sequence, so reads carrying those alleles align without mismatches. The price is that every reported coordinate then refers to a genome nobody else has. This pays it back.

Three steps, STAR's order (Transcript_transformGenome.cpp):

  1. Remap. Each exon goes through the conversion blocks written at build time, split wherever it straddles a boundary — the two halves land at unrelated original coordinates.
  2. Merge. Splits that are seamless in the original genome collapse again, and an uneven gap has its shared part folded into the neighbouring exon. What survives as a gap is a real indel, so an indel baked into the sequence reappears as an I/D CIGAR operation.
  3. Reclassify. Junction motifs and annotation flags are recomputed against the original genome. A junction canonical in the transformed genome need not be canonical in the original, and STAR reports what the original says.

GenomeIndex gains transform_out, loaded only when the flag is set: the untransformed genome from OriginalGenome/, its annotated junctions, and the parsed block map. A missing OriginalGenome/ or transformGenomeBlocks.tsv is an error, not a fallback — falling back would report transformed coordinates while claiming they are original.

Where the conversion happens, and why it matters

Transcripts are converted immediately after alignment, before anything reads them, so records, junction counts and statistics share one coordinate space. Converting at the writer instead would leave SJ.out.tab and the counters describing a genome the SAM file does not.

The SAM header comes from the original genome as well. Original coordinates under transformed reference lengths would produce a file no downstream tool could read correctly.

A paired result converts as a unit: if one mate fails to convert the pair is dropped whole, rather than reported as a half-mapped read that never existed. Insert size is recomputed from the converted mates, since the transform can change the distance between them.

What is refused rather than half-done

--genomeTransformOutput SJ and Quant are unimplemented and rejected. So are the combinations whose other outputs would remain in transformed space: --quantMode, --soloType, --outWigType. A file mixing the two coordinate spaces is worse than no file.

Verification

End to end (test_genome_transform_output_sam_reports_original_coordinates): a genome with a 5-base deletion at position 5001, one read at 6000. Without the flag the reported POS is 5996; with it, 6001 — the original coordinate — and the CIGAR is unchanged, since the read spans no variant.

Unit tests in src/genome/transform_align.rs:

  • an exon inside one block is shifted by that block
  • an exon straddling a boundary comes back out as 10M5D10M
  • a seamless split is merged back
  • soft clips are carried through
  • a junction motif is read from the original genome
  • a junction canonical only in the transformed genome becomes non-canonical
  • a gap shorter than --alignIntronMin stays a deletion
  • an alignment outside the block map is dropped rather than guessed at

Plus block_map_round_trips_through_the_tsv and a malformed-header rejection.

Gate: 571 lib + 22 integration tests, cargo clippy --all-targets -- -D warnings, cargo fmt --check, MSRV 1.89 — all green.

Output-neutral unless the flag is passed: output_genome() returns the searched genome when there is no transform, so every existing run takes the same path it did.

Output neutrality, measured

10k yeast SE reads (CONTRIBUTING's dataset) through this branch and through origin/main, both at default flags: the SAM records are byte-identical (9724 each; the two files differ only in the @PG CL: line, which records the binary path).

Split out of #109 following the one-theme rule in CONTRIBUTING.md; that PR keeps the libsais suffix-array builder.

BenjaminDEMAILLE and others added 5 commits July 29, 2026 11:35
…x, sjdbInsertSave

Four genome-side STAR parameters, each honest about what it does.

`--genomeType` accepts `Full` and refuses `Transcriptome` and
`SuperTranscriptome`. Those need a different index layout entirely; accepting
them would build an ordinary genome under another name.

`--genomeTransformOutput` accepts `None` and refuses `SAM` and `SJ`. Mapping
alignments back to the original coordinate space is not implemented, and a
silent no-op here is worse than a failure: it would report transformed
coordinates as though they were original ones, and nothing downstream could
tell.

`--sjdbInsertSave` accepts `Basic` and `None`. The inserted-junction files are
not retained either way.

`--genomeSuffixLengthMax` is accepted and inert, with the reason recorded on the
flag: the suffix-array builder exposes no comparison bound, and STAR uses this
only to cap work on highly repetitive genomes. The index is identical either
way, so ignoring it changes no output byte — which is the standard this
codebase already applies to the other inert knobs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`--genomeTransformType Haploid` bakes a VCF's variants into the genome sequence,
so reads carrying those alleles align without mismatches. The price is that
every coordinate in the output then refers to a genome nobody else has. This is
the machinery that pays it back, STAR's `Transcript_transformGenome.cpp`.

Three steps, in STAR's order. Remap each exon through the conversion blocks,
splitting wherever it straddles a boundary because the two halves land at
unrelated original coordinates. Merge back the splits that turn out to be
seamless, and fold the shared part of an uneven gap into the neighbouring exon
so what survives as a gap is a real indel. Then reclassify: junction motifs and
annotation flags are read off the *original* genome, because a junction that was
canonical in the transformed genome need not be canonical in the original one,
and STAR reports what the original says.

`blocks_from_tsv` reads `transformGenomeBlocks.tsv` back. The file already
stores the map keyed on transformed coordinates, which is the order the walk
wants and the reverse of what the build side keeps in memory, so parsing is not
symmetric with writing and the test pins the round trip rather than assuming it.
A sentinel entry terminates the map so the forward walk stops on a comparison
instead of a bounds test, as STAR's does.

Tests cover an exon inside one block, an exon straddling a boundary coming back
out as `10M5D10M`, a seamless split collapsing again, soft clips surviving, a
motif read from the original genome, a junction canonical only in the
transformed genome turning non-canonical, a sub-`alignIntronMin` gap staying a
deletion, and an alignment outside the map being dropped rather than guessed at.

Not yet wired to the SAM writer: that needs the original genome loaded at align
time and is the next commit.
The back-transform now runs. Transcripts are converted immediately after
alignment, before anything reads them, so a single coordinate space holds below
that line: records, junction counts and statistics are all in the original
genome. The alternative — converting at the writer — would leave SJ.out.tab and
the counters describing a genome the SAM file does not.

The SAM header comes from the original genome as well, via `output_genome()`.
Original coordinates under transformed reference lengths would produce a file no
downstream tool could read correctly.

`GenomeIndex` gains `transform_out`, loaded only when the flag is set: the
untransformed genome from `OriginalGenome/`, its annotated junctions, and the
parsed block map. A missing `OriginalGenome/` or `transformGenomeBlocks.tsv` is
an error rather than a fallback — falling back would report transformed
coordinates while claiming they are original.

A paired result converts as a unit. If one mate fails to convert the pair is
dropped whole, since reporting the other alone would invent a half-mapped read.
The insert size is recomputed from the converted mates, because the transform
can change the distance between them.

Combinations whose other outputs would stay in transformed space are refused:
`--quantMode`, `--soloType`, `--outWigType`. `--genomeTransformOutput SJ` and
`Quant` remain unimplemented and are still rejected.

Verified end to end: a genome with a 5-base deletion at 5001, one read at 6000.
Without the flag the reported POS is 5996; with it, 6001 — the original
coordinate — and the CIGAR is unchanged, since the read spans no variant.
The other three flags from the original commit belong to the index and
SuperTranscriptome themes; CONTRIBUTING.md asks for one theme per PR and for
the description to match the diff.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant