Skip to content

solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry - #150

Closed
BenjaminDEMAILLE wants to merge 6 commits into
scverse:mainfrom
BenjaminDEMAILLE:bd/solo-barcode-chemistry
Closed

solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry#150
BenjaminDEMAILLE wants to merge 6 commits into
scverse:mainfrom
BenjaminDEMAILLE:bd/solo-barcode-chemistry

Conversation

@BenjaminDEMAILLE

@BenjaminDEMAILLE BenjaminDEMAILLE commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

STARsolo barcode handling: cbMinP posterior threshold, the oneExact guard, adapter-anchored geometry, featuresGeneField3.

What changed

cbMinP and oneExact. resolve_multi_cb computed the count+quality posterior for a 1MM_multi barcode, computed the normalising total, then ignored it: any candidate with positive weight won by argmax. STAR requires two more conditions, both now enforced:

  • the winner must hold at least 97.5% of the total posterior mass (cbMinP), so a read whose barcode cannot be singled out is discarded rather than guessed;
  • unless pseudocounts are in use, the winner must have been observed as an exact match at least once (oneExact). A candidate supported only by a pseudocount is eligible, not evidence.

Adapter-anchored geometry for CB_UMI_Complex: --soloAdapterSequence and --soloAdapterMismatchesNmax, with the four STAR anchor codes (read start, read end, adapter start, adapter end).

The adapter search follows STAR's localAlignHammingDist (SequenceFuns.cpp:341): every offset is scored and the one with the fewest mismatches wins, ties go to the leftmost of the best, and an N in the adapter is a wildcard that costs nothing. Since the barcode positions are measured from the adapter, the chosen offset decides what the read is counted as.

--soloOutFormatFeaturesGeneField3 controls the third features.tsv column; - suppresses it.

Why

Without cbMinP a barcode is assigned whenever any candidate has positive weight, including when two candidates are nearly tied — exactly the case the threshold exists to reject.

This changes default counting behaviour

An existing unit test asserted that priors of 10 against 3 resolve to the higher one: a 77% posterior share, which now correctly resolves to nothing. The test encoded the missing threshold rather than checking it. It now covers a decisive 1000-against-3 prior (accepted), a tie under pseudocounts (rejected), and a winner never seen exactly with pseudocounts off (rejected).

Fixed after review

The adapter search took the leftmost acceptable offset, not the best-scoring one — the opposite of STAR's rule, so an early approximate match beat a later exact one and shifted the whole barcode. Reported by @Psy-Fer with the upstream source. The original tests used single-candidate or exact fixtures, where the two rules agree; there is now a test where they differ, one for the tie-break, and one for N in the adapter.

--soloChemistry is removed. It is not a STAR parameter — it came from STAR-rs — and widening the CLI beyond STAR's is a positioning decision for this project, not something to arrive inside a PR about the barcode posterior. The geometry it set remains reachable through --soloCBstart, --soloCBlen, --soloUMIstart and --soloUMIlen. --soloCBwhitelistDir was described in an earlier version of this text but was never in the diff; that claim is gone too.

--soloCBtype String stays refused: it is a real STAR parameter, and refusing an unimplemented value is the honest answer.

Verification

Gate green: 568 lib + 26 integration, clippy -D warnings, cargo fmt --check, MSRV 1.89. Tests cover the anchored geometry, the three adapter-search rules, the field-3 suppression, and umi_valid_rejects_every_homopolymer (STAR's umiL = 0 quirk lets poly-C/G/T through; documented in docs-old/dev/divergences.md).

The solo differential against real STARsolo passes (test/solo_diff_docker.sh, one run, Linux container with STAR 2.7.11b):

--- 3' SE (CellRanger-style) ---
PASS: rustar-aligner matrix matches real STARsolo exactly.
--- 5' PE (--soloBarcodeMate 1) ---
PASS: Gene raw matrix byte-identical to STARsolo (3 entries)
PASS: GeneFull raw matrix byte-identical to STARsolo (3 entries)

Both runs use --soloCBmatchWLtype 1MM_multi_Nbase_pseudocounts, which is the path cbMinP and oneExact gate, so the thresholds are exercised rather than bypassed.

What this does not establish: the harness fixtures are small and hand-computed, so they show the changed path agrees with STAR on cases where the answer is known, not that counts are unchanged on a real dataset. Reads whose barcode cannot be singled out now land nowhere, so raw matrix counts will move on real data — that is the intended effect of the threshold.

BenjaminDEMAILLE and others added 3 commits July 28, 2026 23:58
…uard

`resolve_multi_cb` computed the count+quality posterior correctly, computed the
normalising total, and then ignored it: any candidate with positive weight won
by argmax. Two of STAR's conditions were missing.

`cbMinP`. STAR requires the winner to hold at least 97.5% of the total
posterior mass. Without it a barcode is assigned whenever any candidate has
positive weight, including when two candidates are nearly tied — which is
precisely the case the threshold exists to reject. A read whose barcode cannot
be singled out should be discarded, not guessed.

`oneExact`. Unless pseudocounts are in use, the winner must have been observed
as an exact match at least once. A candidate whose only support is a
pseudocount is eligible, not evidence.

**This changes default counting behaviour**, and the existing unit test had to
change with it: it asserted that priors of 10 against 3 resolve to the higher
one, which is a 77% posterior share and now correctly resolves to nothing. The
test was encoding the missing threshold. It now covers both sides — a decisive
1000-against-3 prior is accepted, a tie under pseudocounts is not.

Verification I could not do here, and would want before this merges: the solo
differential against real STARsolo (`test/solo_diff_docker.sh`). Reads that
previously landed on an ambiguous barcode now land nowhere, so raw matrix
counts will move, and the direction should be confirmed against the oracle
rather than argued from the source. Everything in-tree is green — 560 lib + 26
integration — but that is not the same thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…er-UMI rule

`--soloOutFormatFeaturesGeneField3` sets the third column of `features.tsv`,
STAR's default being `Gene Expression`. The sentinel `-` drops the column,
giving the two-column form some downstream tools expect. Previously the string
was hard-coded.

Also adds `umi_valid_rejects_every_homopolymer`. No code change was needed —
`is_homopolymer` already rejects a UMI of any single repeated base — but the
rule deserves a test that says why it is deliberate.

STAR has a quirk (`umiL = 0`) under which a non-A homopolymer UMI slips through
its filter, so poly-C, poly-G and poly-T survive there and are counted. That is
a defect rather than a rule: a UMI of one repeated base carries no information
whichever base it happens to be. This diverges from 2.7.11b knowingly, and the
test now records that instead of leaving it to be rediscovered as a difference
in a differential run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… String

`--soloChemistry` sets the CB/UMI geometry from a 10x chemistry name so it does
not have to be spelled out four flags at a time: SC3Pv1 (14-base CB, 10-base
UMI), SC3Pv2 (16/10), SC3Pv3, SC3Pv4 and SC5P (16/12), plus the CellRanger
aliases. Unknown names are refused.

The preset wins over an explicit `--soloCBstart` and friends, and warns when it
overrode one. Half-applying a preset — taking its CB length but the flag's UMI
start, say — would produce a geometry neither the user nor the preset asked
for, and would fail silently in the counts rather than loudly at startup.

`--soloCBtype` is declared and accepts `Sequence`. `String` is refused rather
than accepted and ignored: barcodes here are 2-bit packed ACGT, and a non-ACGT
barcode packed that way is nonsense rather than an approximation. Supporting it
means a different whitelist representation throughout, which is its own change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@BenjaminDEMAILLE BenjaminDEMAILLE changed the title solo: apply STAR's cbMinP posterior threshold and the oneExact guard solo: cbMinP posterior threshold, oneExact guard, chemistry presets, featuresGeneField3 Jul 28, 2026
`--soloCBposition` and `--soloUMIposition` take four fields,
`startAnchor_startDist_endAnchor_endDist`, where the anchor codes are 0 = read
start, 1 = read end, 2 = adapter start, 3 = adapter end. Only code 0 was
accepted; the rest were a parse error.

Codes 2 and 3 are the reason this needs a new layout rather than a wider
parser: the adapter moves from read to read, so the offsets cannot be resolved
once at startup the way every other layout's are. `SoloBarcodeLayout::Anchored`
carries the specs with their anchors and the adapter, and resolves per read.

Adds `--soloAdapterSequence` and `--soloAdapterMismatchesNmax`. The adapter
search takes the leftmost position within the mismatch budget, which is what
STAR takes — the first acceptable match, not the best-scoring one.

A read where the adapter is not found, or where an anchor resolves outside the
read, yields no barcode. Guessing would be worse than declining: a barcode read
at an unknown offset is not salvageable, and a wrong barcode is a read
attributed to the wrong cell rather than a read lost.

Tests cover the adapter moving (both fields follow it), one mismatch inside the
budget, no adapter at all, an anchor running off the front of the read, and the
leftmost-match rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Psy-Fer

Psy-Fer commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

The --soloChemistry arg doesn't exist in STAR, only in STAR-rs. This isn't made clear at all in the PR

@Psy-Fer

Psy-Fer commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

--soloCBwhitelistDir also doesn't exist in the PR???

@Psy-Fer

Psy-Fer commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Also the adapter comment claims STAR takes leftmost when STAR actually takes best-scoring

STAR (SequenceFuns.cpp:341, localAlignHammingDist):

uint32 distBest = query.size();
for (ii = 0 .. text.size()-query.size()) {
    dist1 = <mismatches at offset ii>;
    if (dist1 < distBest) { distBest = dist1; pos = ii; }
}
return distBest; 

This PR (find_adapter)

for start in 0..=(seq.len() - adapter.len()) {
    ... count mm ...
    if mm <= max_mm { return Some(...); }
}

These are opposites. The tests written use exact (0-mismatch) or single-match fixtures, so leftmost and best-scoring coincide, which is why it was missed.

…eftmost

STAR's `localAlignHammingDist` (`SequenceFuns.cpp:341`) scores every offset and
keeps the one with the fewest mismatches. This took the first offset that fit
the mismatch budget, which is the opposite rule: an early approximate match beat
a later exact one. Since `CB_UMI_Complex` measures its barcode positions from
the adapter, choosing the wrong offset shifts the whole barcode, so this decides
what the read is counted as, not merely where a match is reported.

Reported by @Psy-Fer on scverse#150, with the upstream source. The tests did not catch
it because their fixtures had a single candidate or an exact one, where leftmost
and best-scoring give the same answer. There is now a test where they differ.

Two further details from the same function, both of which were missing:

Ties go to the leftmost of the best, because STAR replaces its incumbent only on
a strictly smaller distance.

An `N` in the adapter never counts as a mismatch. It is a wildcard in the query,
not a base that fails to match, and treating it as a mismatch spends budget that
STAR does not spend.

STAR also seeds its best distance with the adapter length, so an offset that
mismatches everywhere can never win. That only shows when the budget is as large
as the adapter, where it keeps the answer `None` instead of an arbitrary offset.
`--soloChemistry` does not exist in STAR 2.7.11b; it was carried over from
STAR-rs. Adding it here widens the CLI beyond STAR's, which is a positioning
decision for this project rather than something to arrive inside a PR about the
barcode posterior. Raised by @Psy-Fer on scverse#150.

The geometry it set is still reachable through the STAR flags it was a shorthand
for: --soloCBstart, --soloCBlen, --soloUMIstart, --soloUMIlen.

`--soloCBtype String` stays refused, as before: it is a real STAR parameter and
refusing an unimplemented value is the honest answer.
@BenjaminDEMAILLE BenjaminDEMAILLE changed the title solo: cbMinP posterior threshold, oneExact guard, chemistry presets, featuresGeneField3 solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry Jul 29, 2026
@BenjaminDEMAILLE

Copy link
Copy Markdown
Contributor Author

All three fixed in 20494db and c8a5d7d.

Adapter search. You are right, and it was a real bug, not just a wrong comment: find_adapter now scores every offset and keeps the fewest mismatches, ties to the leftmost, with N in the adapter as a wildcard that costs nothing — localAlignHammingDist lines 347-358. New test find_adapter_takes_the_best_scoring_match_not_the_leftmost uses a fixture where the rules disagree (approximate match at 0, exact at 5), plus one for the tie-break and one for N.

--soloChemistry. Removed. Not a STAR parameter, so it does not belong in a PR about the barcode posterior. The geometry is still reachable through --soloCBstart, --soloCBlen, --soloUMIstart, --soloUMIlen.

--soloCBwhitelistDir. It was never in the diff — the PR text described it and the code did not have it. The text is corrected.

PR body rewritten. 568 lib + 26 integration green, clippy -D warnings, fmt, MSRV 1.89.

BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
STAR writes the condition as `roE > segmentMin + roS + chimOverlap`
(`chimericAlignScore`). On inclusive read-order coordinates that is
`length > segmentMin + overlap + 1`, not `length > segmentMin + overlap`, so
both detection paths were admitting a segment one base shorter than STAR does.

Found while re-deriving the predicates against upstream after the adapter-search
error on scverse#150, which came from the same habit of reading a condition rather than
re-deriving it.

Both paths carried the same expression, so both are corrected together.
@Psy-Fer

Psy-Fer commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Please see #154 and the updated CONTRIBUTING.md

BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 29, 2026
STAR writes the condition as `roE > segmentMin + roS + chimOverlap`
(`chimericAlignScore`). On inclusive read-order coordinates that is
`length > segmentMin + overlap + 1`, not `length > segmentMin + overlap`, so
both detection paths were admitting a segment one base shorter than STAR does.

Found while re-deriving the predicates against upstream after the adapter-search
error on scverse#150, which came from the same habit of reading a condition rather than
re-deriving it.

Both paths carried the same expression, so both are corrected together.
@BenjaminDEMAILLE

Copy link
Copy Markdown
Contributor Author

All three findings were right, and the adapter one was a genuine correctness bug rather than a documentation problem. Rather than force-push over the review history here, the corrected work is #165. What changed:

Adapter search. Your reading of localAlignHammingDist is what the code now does: score every offset, keep the fewest-mismatch one. The old version returned the first offset inside the mismatch budget, the opposite rule. It mattered more than "where a match is reported", because CB_UMI_Complex measures barcode positions from the adapter, so the wrong offset shifts the whole barcode and changes what the read is counted as. Your point about the fixtures was also correct: single-candidate or exact-match fixtures cannot distinguish the two rules. find_adapter_takes_the_best_scoring_match_not_the_leftmost is the discriminating case, one mismatch at offset 0 against an exact match at offset 5.

Going back to that function properly turned up two more things the old code had wrong: ties go to the leftmost of the best, since STAR replaces its incumbent only on a strictly smaller distance, and an N in the adapter is a wildcard rather than a mismatch, so charging it spends budget STAR does not spend.

--soloChemistry and --soloCBwhitelistDir. Both gone, and they were one mistake rather than two. Neither exists in STAR; they came from the STAR-rs donor and were presented here as parity work without saying so. CONTRIBUTING.md is clear that a non-STAR flag needs sign-off and must not be presented as faithfulness, so reverting was the right call rather than relabelling. If chemistry presets are worth having, they should be proposed as an addition on their own merits.

#154 / CONTRIBUTING.md. Read. Two consequences beyond the above: #165 is rebased onto current main, and it adds the DIVERGENCE.md entry that was missing here. The divergence is that rustar-aligner rejects all four homopolymer UMIs while STAR under CB_UMI_Complex rejects only poly-A, and I checked the source rather than asserting it: the four packed constants are built in the SoloReadBarcode constructor from pSolo.umiL (SoloReadBarcode.cpp:16-21), but that path only learns its UMI length after extracting one (SoloReadBarcode_getCBandUMI.cpp:353-354), so all four stay 0. Poly-A packs to 0 and is caught; the other three do not and slip through.

#165 supersedes this one. Happy to close this as soon as you confirm that is how you want it tracked.

BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 30, 2026
…eftmost

STAR's `localAlignHammingDist` (`SequenceFuns.cpp:341`) scores every offset and
keeps the one with the fewest mismatches. This took the first offset that fit
the mismatch budget, which is the opposite rule: an early approximate match beat
a later exact one. Since `CB_UMI_Complex` measures its barcode positions from
the adapter, choosing the wrong offset shifts the whole barcode, so this decides
what the read is counted as, not merely where a match is reported.

Reported by @Psy-Fer on scverse#150, with the upstream source. The tests did not catch
it because their fixtures had a single candidate or an exact one, where leftmost
and best-scoring give the same answer. There is now a test where they differ.

Two further details from the same function, both of which were missing:

Ties go to the leftmost of the best, because STAR replaces its incumbent only on
a strictly smaller distance.

An `N` in the adapter never counts as a mismatch. It is a wildcard in the query,
not a base that fails to match, and treating it as a mismatch spends budget that
STAR does not spend.

STAR also seeds its best distance with the adapter length, so an offset that
mismatches everywhere can never win. That only shows when the budget is as large
as the adapter, where it keeps the answer `None` instead of an arbitrary offset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE added a commit to BenjaminDEMAILLE/rustar-aligner that referenced this pull request Jul 30, 2026
`--soloChemistry` does not exist in STAR 2.7.11b; it was carried over from
STAR-rs. Adding it here widens the CLI beyond STAR's, which is a positioning
decision for this project rather than something to arrive inside a PR about the
barcode posterior. Raised by @Psy-Fer on scverse#150.

The geometry it set is still reachable through the STAR flags it was a shorthand
for: --soloCBstart, --soloCBlen, --soloUMIstart, --soloUMIlen.

`--soloCBtype String` stays refused, as before: it is a real STAR parameter and
refusing an unimplemented value is the honest answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants