solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry - #150
solo: cbMinP posterior threshold, oneExact guard, adapter-anchored geometry#150BenjaminDEMAILLE wants to merge 6 commits into
Conversation
…uard `resolve_multi_cb` computed the count+quality posterior correctly, computed the normalising total, and then ignored it: any candidate with positive weight won by argmax. Two of STAR's conditions were missing. `cbMinP`. STAR requires the winner to hold at least 97.5% of the total posterior mass. Without it a barcode is assigned whenever any candidate has positive weight, including when two candidates are nearly tied — which is precisely the case the threshold exists to reject. A read whose barcode cannot be singled out should be discarded, not guessed. `oneExact`. Unless pseudocounts are in use, the winner must have been observed as an exact match at least once. A candidate whose only support is a pseudocount is eligible, not evidence. **This changes default counting behaviour**, and the existing unit test had to change with it: it asserted that priors of 10 against 3 resolve to the higher one, which is a 77% posterior share and now correctly resolves to nothing. The test was encoding the missing threshold. It now covers both sides — a decisive 1000-against-3 prior is accepted, a tie under pseudocounts is not. Verification I could not do here, and would want before this merges: the solo differential against real STARsolo (`test/solo_diff_docker.sh`). Reads that previously landed on an ambiguous barcode now land nowhere, so raw matrix counts will move, and the direction should be confirmed against the oracle rather than argued from the source. Everything in-tree is green — 560 lib + 26 integration — but that is not the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…er-UMI rule `--soloOutFormatFeaturesGeneField3` sets the third column of `features.tsv`, STAR's default being `Gene Expression`. The sentinel `-` drops the column, giving the two-column form some downstream tools expect. Previously the string was hard-coded. Also adds `umi_valid_rejects_every_homopolymer`. No code change was needed — `is_homopolymer` already rejects a UMI of any single repeated base — but the rule deserves a test that says why it is deliberate. STAR has a quirk (`umiL = 0`) under which a non-A homopolymer UMI slips through its filter, so poly-C, poly-G and poly-T survive there and are counted. That is a defect rather than a rule: a UMI of one repeated base carries no information whichever base it happens to be. This diverges from 2.7.11b knowingly, and the test now records that instead of leaving it to be rediscovered as a difference in a differential run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… String `--soloChemistry` sets the CB/UMI geometry from a 10x chemistry name so it does not have to be spelled out four flags at a time: SC3Pv1 (14-base CB, 10-base UMI), SC3Pv2 (16/10), SC3Pv3, SC3Pv4 and SC5P (16/12), plus the CellRanger aliases. Unknown names are refused. The preset wins over an explicit `--soloCBstart` and friends, and warns when it overrode one. Half-applying a preset — taking its CB length but the flag's UMI start, say — would produce a geometry neither the user nor the preset asked for, and would fail silently in the counts rather than loudly at startup. `--soloCBtype` is declared and accepts `Sequence`. `String` is refused rather than accepted and ignored: barcodes here are 2-bit packed ACGT, and a non-ACGT barcode packed that way is nonsense rather than an approximation. Supporting it means a different whitelist representation throughout, which is its own change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`--soloCBposition` and `--soloUMIposition` take four fields, `startAnchor_startDist_endAnchor_endDist`, where the anchor codes are 0 = read start, 1 = read end, 2 = adapter start, 3 = adapter end. Only code 0 was accepted; the rest were a parse error. Codes 2 and 3 are the reason this needs a new layout rather than a wider parser: the adapter moves from read to read, so the offsets cannot be resolved once at startup the way every other layout's are. `SoloBarcodeLayout::Anchored` carries the specs with their anchors and the adapter, and resolves per read. Adds `--soloAdapterSequence` and `--soloAdapterMismatchesNmax`. The adapter search takes the leftmost position within the mismatch budget, which is what STAR takes — the first acceptable match, not the best-scoring one. A read where the adapter is not found, or where an anchor resolves outside the read, yields no barcode. Guessing would be worse than declining: a barcode read at an unknown offset is not salvageable, and a wrong barcode is a read attributed to the wrong cell rather than a read lost. Tests cover the adapter moving (both fields follow it), one mismatch inside the budget, no adapter at all, an anchor running off the front of the read, and the leftmost-match rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The |
|
|
|
Also the adapter comment claims STAR takes STAR (SequenceFuns.cpp:341, localAlignHammingDist): This PR (find_adapter) These are opposites. The tests written use exact (0-mismatch) or single-match fixtures, so leftmost and best-scoring coincide, which is why it was missed. |
…eftmost STAR's `localAlignHammingDist` (`SequenceFuns.cpp:341`) scores every offset and keeps the one with the fewest mismatches. This took the first offset that fit the mismatch budget, which is the opposite rule: an early approximate match beat a later exact one. Since `CB_UMI_Complex` measures its barcode positions from the adapter, choosing the wrong offset shifts the whole barcode, so this decides what the read is counted as, not merely where a match is reported. Reported by @Psy-Fer on scverse#150, with the upstream source. The tests did not catch it because their fixtures had a single candidate or an exact one, where leftmost and best-scoring give the same answer. There is now a test where they differ. Two further details from the same function, both of which were missing: Ties go to the leftmost of the best, because STAR replaces its incumbent only on a strictly smaller distance. An `N` in the adapter never counts as a mismatch. It is a wildcard in the query, not a base that fails to match, and treating it as a mismatch spends budget that STAR does not spend. STAR also seeds its best distance with the adapter length, so an offset that mismatches everywhere can never win. That only shows when the budget is as large as the adapter, where it keeps the answer `None` instead of an arbitrary offset.
`--soloChemistry` does not exist in STAR 2.7.11b; it was carried over from STAR-rs. Adding it here widens the CLI beyond STAR's, which is a positioning decision for this project rather than something to arrive inside a PR about the barcode posterior. Raised by @Psy-Fer on scverse#150. The geometry it set is still reachable through the STAR flags it was a shorthand for: --soloCBstart, --soloCBlen, --soloUMIstart, --soloUMIlen. `--soloCBtype String` stays refused, as before: it is a real STAR parameter and refusing an unimplemented value is the honest answer.
|
All three fixed in Adapter search. You are right, and it was a real bug, not just a wrong comment:
PR body rewritten. 568 lib + 26 integration green, clippy |
STAR writes the condition as `roE > segmentMin + roS + chimOverlap` (`chimericAlignScore`). On inclusive read-order coordinates that is `length > segmentMin + overlap + 1`, not `length > segmentMin + overlap`, so both detection paths were admitting a segment one base shorter than STAR does. Found while re-deriving the predicates against upstream after the adapter-search error on scverse#150, which came from the same habit of reading a condition rather than re-deriving it. Both paths carried the same expression, so both are corrected together.
|
Please see #154 and the updated CONTRIBUTING.md |
STAR writes the condition as `roE > segmentMin + roS + chimOverlap` (`chimericAlignScore`). On inclusive read-order coordinates that is `length > segmentMin + overlap + 1`, not `length > segmentMin + overlap`, so both detection paths were admitting a segment one base shorter than STAR does. Found while re-deriving the predicates against upstream after the adapter-search error on scverse#150, which came from the same habit of reading a condition rather than re-deriving it. Both paths carried the same expression, so both are corrected together.
|
All three findings were right, and the adapter one was a genuine correctness bug rather than a documentation problem. Rather than force-push over the review history here, the corrected work is #165. What changed: Adapter search. Your reading of Going back to that function properly turned up two more things the old code had wrong: ties go to the leftmost of the best, since STAR replaces its incumbent only on a strictly smaller distance, and an
#154 / CONTRIBUTING.md. Read. Two consequences beyond the above: #165 is rebased onto current #165 supersedes this one. Happy to close this as soon as you confirm that is how you want it tracked. |
…eftmost STAR's `localAlignHammingDist` (`SequenceFuns.cpp:341`) scores every offset and keeps the one with the fewest mismatches. This took the first offset that fit the mismatch budget, which is the opposite rule: an early approximate match beat a later exact one. Since `CB_UMI_Complex` measures its barcode positions from the adapter, choosing the wrong offset shifts the whole barcode, so this decides what the read is counted as, not merely where a match is reported. Reported by @Psy-Fer on scverse#150, with the upstream source. The tests did not catch it because their fixtures had a single candidate or an exact one, where leftmost and best-scoring give the same answer. There is now a test where they differ. Two further details from the same function, both of which were missing: Ties go to the leftmost of the best, because STAR replaces its incumbent only on a strictly smaller distance. An `N` in the adapter never counts as a mismatch. It is a wildcard in the query, not a base that fails to match, and treating it as a mismatch spends budget that STAR does not spend. STAR also seeds its best distance with the adapter length, so an offset that mismatches everywhere can never win. That only shows when the budget is as large as the adapter, where it keeps the answer `None` instead of an arbitrary offset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`--soloChemistry` does not exist in STAR 2.7.11b; it was carried over from STAR-rs. Adding it here widens the CLI beyond STAR's, which is a positioning decision for this project rather than something to arrive inside a PR about the barcode posterior. Raised by @Psy-Fer on scverse#150. The geometry it set is still reachable through the STAR flags it was a shorthand for: --soloCBstart, --soloCBlen, --soloUMIstart, --soloUMIlen. `--soloCBtype String` stays refused, as before: it is a real STAR parameter and refusing an unimplemented value is the honest answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
STARsolo barcode handling:
cbMinPposterior threshold, theoneExactguard, adapter-anchored geometry,featuresGeneField3.What changed
cbMinPandoneExact.resolve_multi_cbcomputed the count+quality posterior for a 1MM_multi barcode, computed the normalising total, then ignored it: any candidate with positive weight won by argmax. STAR requires two more conditions, both now enforced:cbMinP), so a read whose barcode cannot be singled out is discarded rather than guessed;oneExact). A candidate supported only by a pseudocount is eligible, not evidence.Adapter-anchored geometry for
CB_UMI_Complex:--soloAdapterSequenceand--soloAdapterMismatchesNmax, with the four STAR anchor codes (read start, read end, adapter start, adapter end).The adapter search follows STAR's
localAlignHammingDist(SequenceFuns.cpp:341): every offset is scored and the one with the fewest mismatches wins, ties go to the leftmost of the best, and anNin the adapter is a wildcard that costs nothing. Since the barcode positions are measured from the adapter, the chosen offset decides what the read is counted as.--soloOutFormatFeaturesGeneField3controls the thirdfeatures.tsvcolumn;-suppresses it.Why
Without
cbMinPa barcode is assigned whenever any candidate has positive weight, including when two candidates are nearly tied — exactly the case the threshold exists to reject.This changes default counting behaviour
An existing unit test asserted that priors of 10 against 3 resolve to the higher one: a 77% posterior share, which now correctly resolves to nothing. The test encoded the missing threshold rather than checking it. It now covers a decisive 1000-against-3 prior (accepted), a tie under pseudocounts (rejected), and a winner never seen exactly with pseudocounts off (rejected).
Fixed after review
The adapter search took the leftmost acceptable offset, not the best-scoring one — the opposite of STAR's rule, so an early approximate match beat a later exact one and shifted the whole barcode. Reported by @Psy-Fer with the upstream source. The original tests used single-candidate or exact fixtures, where the two rules agree; there is now a test where they differ, one for the tie-break, and one for
Nin the adapter.--soloChemistryis removed. It is not a STAR parameter — it came from STAR-rs — and widening the CLI beyond STAR's is a positioning decision for this project, not something to arrive inside a PR about the barcode posterior. The geometry it set remains reachable through--soloCBstart,--soloCBlen,--soloUMIstartand--soloUMIlen.--soloCBwhitelistDirwas described in an earlier version of this text but was never in the diff; that claim is gone too.--soloCBtype Stringstays refused: it is a real STAR parameter, and refusing an unimplemented value is the honest answer.Verification
Gate green: 568 lib + 26 integration, clippy
-D warnings,cargo fmt --check, MSRV 1.89. Tests cover the anchored geometry, the three adapter-search rules, the field-3 suppression, andumi_valid_rejects_every_homopolymer(STAR'sumiL = 0quirk lets poly-C/G/T through; documented indocs-old/dev/divergences.md).The solo differential against real STARsolo passes (
test/solo_diff_docker.sh, one run, Linux container with STAR 2.7.11b):Both runs use
--soloCBmatchWLtype 1MM_multi_Nbase_pseudocounts, which is the pathcbMinPandoneExactgate, so the thresholds are exercised rather than bypassed.What this does not establish: the harness fixtures are small and hand-computed, so they show the changed path agrees with STAR on cases where the answer is known, not that counts are unchanged on a real dataset. Reads whose barcode cannot be singled out now land nowhere, so raw matrix counts will move on real data — that is the intended effect of the threshold.