Skip to content

perf(align): reuse cluster_seeds' window-bin map across reads (-7% median, output-neutral) - #185

Open
BenjaminDEMAILLE wants to merge 14 commits into
scverse:mainfrom
BenjaminDEMAILLE:bd/perf-cluster-seeds
Open

perf(align): reuse cluster_seeds' window-bin map across reads (-7% median, output-neutral)#185
BenjaminDEMAILLE wants to merge 14 commits into
scverse:mainfrom
BenjaminDEMAILLE:bd/perf-cluster-seeds

Conversation

@BenjaminDEMAILLE

@BenjaminDEMAILLE BenjaminDEMAILLE commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Stacked on #184#182#179#178#176#175#174#173. Merge in that order. It touches only src/align/stitch.rs and rebases off the solo stack cleanly if you would rather take it alone.

First item of roadmap P3, and not the item P3 led with — see the profile below for why.

The defects

Two, both per-read allocations in cluster_seeds.

1. A rehashing map

cluster_seeds built a fresh FxHashMap per read, pre-sized to anchor_indices.len() * 2. That pre-sizing is only a floor. When two windows merge, the code re-keys every bin in the merged span:

for bin in left_window.bin_start..=left_window.bin_end {
    win_bin.insert((strand, bin), left_idx);
}

so a read with wide windows inserts far more entries than it has anchors, and the map rehashes — inside the function the code itself labels "the #1 align hotspot".

2. A clone thrown away one statement later

clusters.push(SeedCluster {
    alignments: window.alignments.clone(),   // full Vec<WindowAlignment> copy
    ...
});

windows is dropped at the end of the function. The copy existed only to be discarded.

Profile first, since P3 assumed otherwise

Sampled a real human run: 10x pbmc_1k_v3, 20 M read pairs, GRCh38, 14 threads.

self time
cluster_seeds + its hashbrown::reserve_rehash 19.2%, of which 2.1% is the rehash alone
seed search (compare_seq_to_genome, search_direction_sparse, PackedArray::read) 16.4%
stitch DP (extend_alignment, stitch_recurse, finalize_transcript) 21%
allocator (mimalloc + RawVec::finish_grow) 11.2%
idle / synchronisation 12.5%
inflate (gzip input) 0.5%

P3 led with the batched prefetching seed search, on the strength of a 2.09× the donor measured for that engine. At 16.4% of on-CPU time here, 2.09× on that engine is worth about 8% of wall clock — real, but it is 500 lines of state machine, and find_mult_range probes in ways the donor's batch does not cover. These changes are ten lines against a bigger share.

(That 0.5% for inflate is also a second, independent confirmation of the measurement that closed #183.)

What it does

The map is kept per thread and cleared per read, so its capacity settles at what the workload needs and growth is paid once per thread rather than once per read.

It is taken out of the thread-local with mem::take and handed back at both exit points, rather than borrowed across the body. There is no nested call today; if one appeared it would get a fresh map instead of panicking on an already-active RefCell borrow.

Output-neutral, checked

On the same 20 M read pairs, old binary against new:

matrix.mtx.gz     identical
barcodes.tsv.gz   identical
features.tsv.gz   identical
SJ.out.tab        identical

Compared decompressed, so a change in gzip framing could not be mistaken for a change in output.

Timing, and what I got wrong measuring it

Two changes, both in cluster_seeds: the map reuse above, and moving each window's Vec<WindowAlignment> into its cluster instead of cloning it (windows is dropped one statement later, so every read paid a full copy per window to discard the original).

The honest answer is "a few percent, direction confirmed, magnitude not pinned down." Getting there took two attempts and the first one was biased.

My first harness ran ABAB — old, new, old, new. Under that, new won 8 of 8 pairs, median −7.0%. Then a second run of the same shape had new losing 2 of 3, and the reason was visible in the raw times: the same binary on the same input went 97.5 s → 111.7 s → 122.2 s. The machine was drifting about 20% while I measured a 5% effect, and ABAB is biased under monotonic drift, because within each pair the second binary always runs on the hotter machine. That penalises "new" systematically.

So I rewrote it as ABBA — old, new, new, old — which cancels a linear drift because each binary gets one early and one late slot per block, let the machine cool, and re-ran:

block old (drift-cancelled mean) new
1 104.2 s 104.3 s +0.1%
2 109.3 s 99.0 s −9.4%
3 103.8 s 97.2 s −6.3%
4 107.2 s 102.2 s −4.7%

New faster in 3 of 4 blocks, median −5.5%. Drift across the run, measuring the old binary alone, was still 96.9 s to 116.9 s — a 21% spread against a 5% effect.

Taking both runs together, new is ahead in 11 of 12 comparisons, and the ABAB 8-of-8 counts for more than I first thought since the bias ran against it. What I will not claim is a specific figure: on a machine with a 21% spread, "−5.5%" is a direction with an order of magnitude, not a measurement.

This is exactly what #171 exists to prevent. That PR's harness samples CPU idle before and after every run and drops the rounds it could not measure honestly. It has been open since 29 July. I got caught by precisely the failure mode it was written for, which is the best argument for it I can offer.

What is not in this PR

The rest of P3 — SmallVec in the stitch recursion, Arc<[u8]> for Transcript::read_seq, and the batched seed search — is separate work. The profile above says the allocator is worth 11.2% and the seed search 16.4%, so both are worth doing; neither belongs in a four-line PR.

Gate: 577 lib + 27 integration tests, cargo clippy --all-targets -- -D warnings, cargo fmt --check, all green.

BenjaminDEMAILLE and others added 14 commits July 31, 2026 10:06
`--soloUMIfiltering MultiGeneUMI_CR` kept every gene tied at the highest
read count. CellRanger's rule is the opposite on exactly that case: the
gene with the *strictly* highest count takes the UMI, and a tie means no
gene counts it.

STAR walks the genes keeping a running maximum and clears its winner
whenever it meets an equal count
(`SoloFeature_collapseUMIall.cpp:212-224`):

    if (ig.second>maxu) { maxu=ig.second; maxg=ig.first; }
    else if (ig.second==maxu) { maxg=-1; };
    ...
    if ( maxg+1==0 ) continue; // not counted for any gene

One read per gene is the ordinary shape of a multi-gene UMI, and it is
always a tie, so the old rule made the flag inert in practice rather
than merely inaccurate. Measured on a 20 000-read 10x fixture (200 cells
from the real v3 whitelist, 400 genes, 720 UMIs deliberately shared
between two genes), against STAR 2.7.11b with the same flags:

                       identical entries    STAR counts   rustar counts
    before             13 749 / 14 806          15 423          16 465
    after              13 902 / 13 967          15 423          15 414

The flag removed nothing at all before; STAR removes 1 030 counts. The
gap goes from +1 042 to -9.

The outcome does not depend on the order the genes are visited — a
strict maximum always ends as the winner, a tie always ends with none —
so iterating a `HashMap` here stays deterministic.

`multi_gene_umi_cr_drops_a_tie_entirely` pins the case the old tests
missed: they only covered 3 reads against 1, where both rules agree.

Not yet implemented, and stated so rather than left to be discovered:
STAR applies a second condition, that the winning gene must also hold
the top count among *uncorrected* UMIs (`umiGeneMapCount0`, same file,
lines 226-232). That needs the pre-correction counts, which this code
does not keep. The 65 entries still differing out of 13 967 are the
place to look for its effect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…w matrix

STARsolo's raw matrix has a column per whitelist barcode. For 10x v3 that
is 3 686 400 columns and a 62 MB `barcodes.tsv`, nearly all zeros.
CellRanger's `raw_feature_bc_matrix` has a column per *observed* barcode.

The two files therefore share no keys, which is not a rounding difference
in a comparison, it is zero overlap: comparing our raw output against a
real `cellranger count` run gave 0 identical entries out of 27 396 until
the columns were reconciled.

`--soloOutRawBarcodes Observed` narrows the raw matrix to the barcodes
that carry a count. Default `Whitelist` keeps what STARsolo writes, so
nothing changes for anyone not asking.

Measured on the 20 000-read fixture:

    Whitelist  3 686 400 barcodes   barcodes.tsv 62 668 800 bytes
    Observed         200 barcodes   barcodes.tsv      3 400 bytes

with identical counts on both sides: 13 937 entries, 15 414 counts.

`finalize_matrix` already took a column remap for the filtered matrix, so
this reuses it rather than adding a second path. The observed set is read
back from the streamed body, which costs one pass and only when the flag
is on.

This is a **non-STAR flag** and needs sign-off; recorded in
`DIVERGENCE.md` §3.2 rather than presented as parity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
STAR corrects UMIs within each gene *before* deciding which gene owns a
UMI, and applies two conditions, not one
(`SoloFeature_collapseUMIall.cpp:134-148` and `:203-235`):

1. one gene must hold a strictly higher read count than every other, on
   the **corrected** UMI map — that is scverse#173, already landed;
2. and that winner must not be beaten in the **uncorrected** map at the
   same key.

The second condition exists because correction moves reads between UMIs:
a gene can win only because correction folded a neighbouring UMI onto it,
and STAR rejects that win rather than counting it.

Reproducing it needs the order STAR uses. The generic path here filters
multi-gene UMIs first and corrects afterwards, which cannot express either
condition: by the time correction happens the ownership decision is
already made. `MultiGeneUMI_CR` therefore takes its own path, which is
also what STAR does — the flag is only valid with `--soloUMIdedup 1MM_CR`,
so there is no combination this bypasses.

`cellranger_1mm_map` exposes the correction mapping that
`cellranger_1mm` already computed and threw away.

Measured against **CellRanger 10.0.0** on the 20 000-read fixture from
scverse#172, with scverse#165 and scverse#173 also applied:

                              identical entries   CellRanger   rustar
    scverse#165 + scverse#173               13 651 / 13 709        15 111     15 091
    plus this change          13 676 / 13 709        15 111     15 116

Entries CellRanger has and we do not go from 29 to 7, and the count gap
from -20 to +5, which is 0.03%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Aligning 10x data and comparing the count matrix against CellRanger gave
a successful run and different numbers, with nothing in the output
pointing at the five flags that explain the difference. Measured against
CellRanger 10.0.0 on a 20 000-read fixture, those flags are the whole
gap: 8.9% away without them, 0.03% with them.

When the geometry is unambiguously 10x — `CB_UMI_Simple`, a whitelist, a
16-base cell barcode, a 10- or 12-base UMI — the five now default to
their CellRanger values:

    --clipAdapterType      CellRanger4
    --outFilterScoreMin    30
    --soloCBmatchWLtype    1MM_multi_Nbase_pseudocounts
    --soloUMIfiltering     MultiGeneUMI_CR
    --soloUMIdedup         1MM_CR

A flag given on the command line always wins, including when the value
asked for is STARsolo's own default: `value_source` distinguishes an
explicit flag from a default, so the divergence is escapable by naming
what you want. Every substitution is logged at INFO with the geometry
that triggered it.

**This changes default output behaviour on 10x runs and diverges from
STARsolo**, which is why it is confined to a geometry nothing else in
common use shares, why it is announced on every run it touches, and why
it is in `DIVERGENCE.md` §1.3 as the largest entry in that file. It needs
sign-off.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Re-derived both sides from one clean state: the five flags move the matrix
from 8.96% above CellRanger to 2.17% above it, not to 0.03%. The earlier
figure compared a rustar run against a CellRanger run built from a different
state of the fixture. STAR 2.7.11b with the same flags is at +0.09%, so the
remaining gap is open rather than closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The figure I replaced this with was measured without scverse#165, whose cbMinP
posterior threshold is a precondition for it. With scverse#165 the five flags move
the matrix from 8.96% above CellRanger to 0.03% above it, which is what the
original text said.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Writes the same numbers where `cellranger count` writes them:
outs/{raw,filtered}_feature_bc_matrix/, gzipped, a -1 GEM-well suffix on
every barcode, one raw column per observed barcode, and no per-feature
subdirectory when a single feature is requested.

The layout implies --soloOutGzip yes, --soloOutRawBarcodes Observed and an
outs/ output directory; each is still overridable on the command line. On
10x geometry it is the default, alongside the five flags from the previous
commit, so it changes where output files are written on those runs.

Counts are untouched: 13 959 entries and 15 439 counts in both layouts on
the 20 000-read fixture, compared entry by entry. Against a real
cellranger count 10.0.0 run on the same fixture the raw barcode sets match
exactly, 200 of 200.

Recorded in DIVERGENCE.md 3.3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Writes CellRanger 10.0.0's 20 metrics, in its order and value formats
(thousands separators, quoted when that adds a comma; percentages to one
decimal). STARsolo's Summary.csv is unchanged and still written alongside.

Twelve of the 20 match a real cellranger count 10.0.0 run exactly on the
20 000-read fixture. The other eight rest on an interpretation of a
denominator CellRanger does not document, and each interpretation is stated
in DIVERGENCE.md 3.4 rather than left implicit.

Three metrics needed new counters: Q30Stats tallies Phred >= 30 bases over
the barcode, the UMI and the cDNA read, populated only when the layout asks
for the metrics. The exonic/intronic split needs the gene-body overlap
query, so a Gene-only run under this layout now performs it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
10x define it as 1 - n_deduped_reads / n_reads over unique (barcode, UMI,
gene) combinations among confidently mapped reads. Implementing that
literally, counting distinct triples before UMI correction, gives 0.0% on
the fixture: no two reads there share an exact triple. So the literal
reading is wrong and their numerator is the corrected molecule count, like
ours. What differs is the read denominator, which 3.4 now says instead of
leaving the disagreement unexplained.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The same quantile-over-ratio rule STARsolo calls CellRanger2.2, but with
the expected cell count searched for instead of fixed at 3 000: minimise
(OrdMag(x) - x)^2 / x over x from 2 to maxExpectedCells, where OrdMag(x) is
the number of cells the rule calls when told to expect x of them. The loss
is small where the rule predicts itself.

EmptyDrops_CR now uses it for its initial cell set, which is the order
CellRanger runs the two steps in. CellRanger2.2 is untouched and stays the
default.

Two things 10x's page leaves unspecified are decided here and documented:
every integer in range is evaluated rather than a geometric grid, so the
result is the true minimum of the stated loss and not a nearby grid point;
and ties go to the smaller x, so nothing depends on iteration order.

On the 20 000-read fixture this changes nothing: CellRanger2.2, OrdMag and
EmptyDrops_CR all call 200 cells, as does CellRanger 10.0.0. That fixture
has a clean plateau where any of these rules works. The two part company on
a graded distribution, which is what the unit tests cover.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CellRanger has included intronic reads in its gene counts since v7.0.
STARsolo's Gene feature is exonic-only, so a 10x run compared against
CellRanger came out 30.5% low. --soloFeatures now defaults to GeneFull on
10x geometry, alongside the other CellRanger defaults; naming the flag
explicitly still wins.

Measured on 10x's own pbmc_1k_v3, 20 M read pairs, refdata-gex-GRCh38-2024-A
on both sides, against cellranger count 10.0.0:

  Gene     6 382 961 counts, -30.5%, 52.9% of entries identical
  GeneFull 9 039 161 counts,  -1.6%, 91.8% of entries identical

The yeast fixture could not show this: yeast has almost no introns, and
there GeneFull counted fewer than Gene because its dense overlapping genes
turn unambiguous exonic reads into ambiguous gene-body ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cluster_seeds built a fresh FxHashMap per read. The pre-sizing to
anchor_indices.len() * 2 was only a floor: merging two windows re-keys every
bin in the merged span, so a read with wide windows inserts far more entries
than it has anchors and the map rehashes. Sampling a human 10x run put
hashbrown::reserve_rehash at 2.1% of on-CPU time, all of it here.

The map is now kept per thread and cleared per read, so its capacity settles
at what the workload needs and the growth is paid once per thread. It is
taken out of the thread-local rather than borrowed across the body, so a
nested call would get a fresh map instead of panicking on an active borrow.

Output-neutral: matrix.mtx, barcodes.tsv, features.tsv and SJ.out.tab are
byte-identical before and after on 20 M read pairs of 10x pbmc_1k_v3 against
GRCh38, compared decompressed.

Interleaved A/B, 8 pairs: new faster in 8 of 8, median -7.0%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The per-window Vec<WindowAlignment> was cloned into the SeedCluster while
`windows` was dropped one statement later, so every read paid a full copy
per window to throw the original away. std::mem::take moves it instead.

Output-neutral, verified by an empty diff on 20 M read pairs of 10x
pbmc_1k_v3 against GRCh38.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dependency discussion: rapidgzip-rust for parallel gzip input (pure Rust)

1 participant