solo: metrics_summary.csv under the CellRanger layout (stacked on #178)

[BenjaminDEMAILLE]

Stacked on #178 → #176 → #175 → #174 → #173. Merge in that order.

Part of #172. #178 got CellRanger's matrix files right; this adds the file a 10x pipeline reads to judge whether the run worked.

What this does

Under --soloOutLayout CellRanger, writes metrics_summary.csv with CellRanger 10.0.0's 20 metrics, in its order and its value formats: thousands separators on integers, quoted once that puts a comma in the field, and percentages to one decimal.

STARsolo's Summary.csv is untouched and still written alongside. The older CellRanger.summary.csv is not written under this layout, because its four rows are a subset of the new file.

Measured against a real cellranger count 10.0.0 run

Same 20 000-read fixture as the rest of the stack, with #165 applied — that is a precondition for every number in this stack, not a footnote, and I got it wrong once already (see the retraction on #175). The header row is byte-identical. Thirteen of the 20 values match exactly:

metric	CellRanger	rustar
Estimated Number of Cells	200	200
Mean Reads per Cell	100	100
Number of Reads	"20,000"	"20,000"
Valid UMI Sequences	100.0%	100.0%
Q30 Bases in Barcode	100.0%	100.0%
Q30 Bases in RNA Read	100.0%	100.0%
Q30 Bases in UMI	100.0%	100.0%
Reads Mapped to Genome	98.6%	98.6%
Reads Mapped Confidently to Intronic Regions	0.0%	0.0%
Reads Mapped Confidently to Transcriptome	88.5%	88.5%
Reads Mapped Antisense to Gene	0.0%	0.0%
Fraction Reads in Cells	100.0%	100.0%
Median UMI Counts per Cell	75	75

And the seven that do not:

metric	CellRanger	rustar	why
Sequencing Saturation	7.4%	12.7%	different denominator; the largest gap and the one I am least sure of
Valid Barcodes	97.8%	100.0%	every barcode in this fixture is drawn from the whitelist, so 100% is what our definition gives
Reads Mapped Confidently to Genome	90.6%	90.3%	follows the alignment difference, not a definition difference
Reads Mapped Confidently to Exonic Regions	90.4%	90.3%	same
Reads Mapped Confidently to Intergenic Regions	0.2%	0.0%	same
Median Genes per Cell	68	69	follows the residual count difference
Total Genes Detected	371	375	same

Without #165 it is 12 of 20, and Median UMI Counts per Cell is the one that moves.

What it costs

Three metrics needed counters that did not exist: Q30Stats tallies Phred ≥ 30 bases over the barcode, the UMI and the cDNA read, on the unclipped read, populated only when the layout asks for the metrics.

The exonic/intronic split needs the gene-body overlap query, which a Gene-only run does not otherwise do. Under this layout it now performs it: one extra interval query per read. Named here because it is a change to a hot path, and it is why --soloFeatures Gene alone can now report the funnel at all.

No count changes: matrix.mtx is byte-identical with and without this commit.

Verification

• test_solo_out_layout_cellranger — extended: the header row is compared against the literal string from a real CellRanger run, then the 20 values, including saturation (8 reads collapsing to 2 molecules = 75.0%) and the three Q30 metrics
• metric_values_are_formatted_the_way_cellranger_formats_them — both value formats, including the quoting threshold at 1 000 and division by zero
• q30_counts_bases_at_phred_30_and_above — the boundary at '>' (Phred 29) and '?' (Phred 30), and a read with no barcode

Gate: 572 lib + 27 integration tests, cargo clippy --all-targets -- -D warnings, cargo fmt --check, cargo +1.89 check --all-targets, all green.

Not in this PR

raw_feature_bc_matrix.h5 and filtered_feature_bc_matrix.h5 need an HDF5 dependency; that is #177, raised before writing any of it. molecule_info.h5, web_summary.html, cloupe.cloupe and analysis/ I am not proposing: the last two are an undocumented proprietary format and a clustering pipeline that does not belong in an aligner.

[BenjaminDEMAILLE]

Human data: the +0.03% is a yeast artefact, and the real gap is introns

The 20 000-read yeast fixture is the only dataset these PRs were measured on, and I flagged more than once that a second, differently-shaped one was needed. I ran it: 10x's own pbmc_1k_v3, 20 M read pairs, refdata-gex-GRCh38-2024-A on both sides, cellranger count 10.0.0 against this stack plus #165.

Raw matrix, against CellRanger

build	entries	counts	vs CellRanger	entries identical
CellRanger 10.0.0	3 547 245	9 187 379	—	—
rustar, --soloFeatures Gene (our default)	2 279 624	6 382 961	−30.5%	52.9%
rustar, --soloFeatures GeneFull	3 544 768	9 039 161	−1.6%	91.8%

Our default is 30.5% below CellRanger on real human data. The +0.03% I measured on yeast is correct for that fixture and does not generalise at all — yeast has almost no introns, so exonic-only counting happened to agree. On human, CellRanger has included intronic reads by default since v7.0 and we do not.

That is one flag: --soloFeatures GeneFull closes it from −30.5% to −1.6%, with 91.8% of the 3.6 M entries identical cell-for-cell and gene-for-gene.

metrics_summary.csv (#179), against CellRanger

The three quality metrics match to the decimal, which is the part of that PR I was most confident about and it holds:

metric	CellRanger	rustar
Q30 Bases in Barcode	94.2%	94.2%
Q30 Bases in RNA Read	90.0%	90.0%
Q30 Bases in UMI	92.8%	92.8%
Reads Mapped Confidently to Intergenic Regions	3.7%	3.7%
Sequencing Saturation	42.0%	42.1%
Valid Barcodes	97.3%	97.5%
Reads Mapped Confidently to Intronic Regions	31.0%	30.6%
Reads Mapped Antisense to Gene	7.9%	7.7%

Sequencing Saturation was the metric I said I was least sure of. On yeast it read 12.7% against CellRanger's 7.4% and I could not explain it. On human it is 42.1% against 42.0%. The yeast disagreement was a property of a fixture where no two reads share a UMI, not a defect in the formula. DIVERGENCE.md §3.4 needs updating to say that, and I will.

The metrics that remain far apart — Reads Mapped Confidently to Transcriptome 81.3% vs 55.2%, Total Genes Detected 23 893 vs 19 531, Median UMI Counts per Cell 6 353 vs 4 555 — are all downstream of the same intron difference, not separate problems.

What I think this means for the stack

Nothing here invalidates #173–#179: the rules they implement are still the rules, and the yeast numbers in their descriptions are still what that fixture produces. What changes is the headline. "Within 0.03% of CellRanger" is true of one small yeast fixture and false of real human data, and I would rather say so here than have a reviewer find it.

The concrete consequence is that --soloFeatures GeneFull belongs in CELLRANGER_DEFAULTS in #176, alongside the five flags already there. It is a one-line change and I have not made it, because it changes default output again and #176 is already asking for sign-off on that. Your call; the measurement is above.