Falco v2.0

[andrewdavidsmith]

Discussion related to falco2, aimed to release in July 2026.

Example topics here:

-- Any questions
-- Feature requests
-- Tell me what kinds of changes would be deal-breakers
-- Suggestions on test data (both for accuracy and performance) and hardware configurations for benchmarks
-- Use cases that should be considered for testing and performance tuning

Note:
-- Motivation for changes: open up opportunities for future performance increases and incorporate preseq stats
-- This post will be edited soon with a summary of anticipated changes to the user interface and behaviors
-- Before Falco v2.0 is released, it will be available in a separate branch so you can try it
-- Falco v1.X will be maintained for some time alongside v2.0, and I will accept bug reports on v1.X in case you can't migrate

[andrewdavidsmith]

Data files for some performance numbers

(Performance numbers to be posted below)

These were taken from: https://github.com/genome-in-a-bottle/giab_data_indexes

File formats below were identified using falco-2.0.0-7809fc0-Linux (artifact links only work for logged in users) which was built from commit 7809fc0.

Data sets

BAM files

$ ls -sh1 bam
total 474G
122G HG002.GRCh38.2x250.bam
122G HG002.hs37d5.2x250.bam
110G HG003.GRCh38.2x250.bam
121G HG004.GRCh38.2x250.bam

FASTQ files (plain)

$ ls -sh1 plain
total 261G
39G hg002_gm24385.lncrna.R1.fastq
39G hg002_gm24385.lncrna.R2.fastq
47G hg002_gm26105.lncrna.R1.fastq
47G hg002_gm26105.lncrna.R2.fastq
46G hg002_gm27730.lncrna.R1.fastq
46G hg002_gm27730.lncrna.R2.fastq

FASTQ files (gz)

$ ls -sh1 gz
total 50G
7.1G hg002_gm24385.lncrna.R1.fastq.gz
7.6G hg002_gm24385.lncrna.R2.fastq.gz
8.5G hg002_gm26105.lncrna.R1.fastq.gz
9.1G hg002_gm26105.lncrna.R2.fastq.gz
8.5G hg002_gm27730.lncrna.R1.fastq.gz
9.2G hg002_gm27730.lncrna.R2.fastq.gz
$ ./falco-2.0.0-7809fc0-Linux -t 16 -v -o ${TMPDIR} gz/hg002_gm24385.lncrna.R1.fastq.gz | grep "input file format"
input file format: FASTQ gzip-compressed sequence data

$ ls -sh1 gz2
total 425G
54G BGISEQ500_PCRfree_NA24149_CL100076246_L01_read_1.fq.gz
56G BGISEQ500_PCRfree_NA24149_CL100076246_L01_read_2.fq.gz
53G BGISEQ500_PCRfree_NA24149_CL100076246_L02_read_1.fq.gz
55G BGISEQ500_PCRfree_NA24149_CL100076246_L02_read_2.fq.gz
53G BGISEQ500_PCRfree_NA24385_CL100076190_L01_read_1.fq.gz
54G BGISEQ500_PCRfree_NA24385_CL100076190_L01_read_2.fq.gz
51G BGISEQ500_PCRfree_NA24385_CL100076190_L02_read_1.fq.gz
52G BGISEQ500_PCRfree_NA24385_CL100076190_L02_read_2.fq.gz
$ ./falco-2.0.0-7809fc0-Linux -t 16 -v -o ${TMPDIR} gz2/BGISEQ500_PCRfree_NA24149_CL100076246_L01_read_1.fq.gz | grep "input file format"
input file format: FASTQ gzip-compressed sequence data

FASTQ files (bgzf)

$ ls -sh1 bgzf
total 51G
7.2G hg002_gm24385.lncrna.R1.fastq.gz
7.7G hg002_gm24385.lncrna.R2.fastq.gz
8.6G hg002_gm26105.lncrna.R1.fastq.gz
9.1G hg002_gm26105.lncrna.R2.fastq.gz
8.6G hg002_gm27730.lncrna.R1.fastq.gz
9.2G hg002_gm27730.lncrna.R2.fastq.gz
$ ./falco-2.0.0-7809fc0-Linux -t 16 -v -o ${TMPDIR} bgzf/hg002_gm24385.lncrna.R1.fastq.gz | grep "input file format"
input file format: FASTQ BGZF-compressed sequence data

$ ls -sh1 bgzf2
total 376G
48G BGISEQ500_PCRfree_NA24149_CL100076246_L01_read_1.fq.gz
50G BGISEQ500_PCRfree_NA24149_CL100076246_L01_read_2.fq.gz
47G BGISEQ500_PCRfree_NA24149_CL100076246_L02_read_1.fq.gz
49G BGISEQ500_PCRfree_NA24149_CL100076246_L02_read_2.fq.gz
46G BGISEQ500_PCRfree_NA24385_CL100076190_L01_read_1.fq.gz
48G BGISEQ500_PCRfree_NA24385_CL100076190_L01_read_2.fq.gz
45G BGISEQ500_PCRfree_NA24385_CL100076190_L02_read_1.fq.gz
46G BGISEQ500_PCRfree_NA24385_CL100076190_L02_read_2.fq.gz
$ ./falco-2.0.0-7809fc0-Linux -t 16 -v -o ${TMPDIR} bgzf2/BGISEQ500_PCRfree_NA24149_CL100076246_L01_read_1.fq.gz | grep "input file format"
input file format: FASTQ BGZF-compressed sequence data

[andrewdavidsmith]

Falco v2

Numbers are seconds, percent of cores used, and memory used.

Build of falco used for this profiling All analyses will be done with a build of falco that's available through the Actions tab from the falco repo, identifying the Actions tab from the main falco repository page, finding Release (Linux) in the list of actions, and clicking on it, and then a specific run. You will arrive at a page with a section called Artifacts at the bottom, and a download link for a universal Linux binary that should run on any Linux machine from the past 15+ years. For now I will only keep the runs of the Release (Linux) action with artifacts used to produce numbers here.

Hardware: AMD EPYC 9554 64-Core Processor with network-attached storage (hence the variance between runs below).

The two builds that are relevant here:

• falco-2.0.0-7809fc0-Linux
• falco-2.0.0-b9a3139-Linux (Has updates to how threads are assigned to tasks when analyzing BAM/BGZF files.)

I specified to use 16 threads for uncompressed files and regular gz files, but 32 for BAM and BGZF, as those can be decompressed in parallel using threads.

Timing for FASTQ files (plain)

FASTQ files without compression (6 files, total size 261G).

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-7809fc0-Linux -t 16 -o ${TMPDIR} plain/hg002_gm2*fastq; done
0:42.19 1109% 1570092k
0:41.13 1133% 1552272k
0:41.91 1113% 1564212k
0:41.68 1121% 1576328k
0:41.84 1115% 1552276k
0:42.05 1113% 1558268k
0:41.15 1132% 1588320k
0:41.30 1130% 1546380k
0:41.54 1121% 1564336k
0:41.39 1124% 1576212k

Timing for FASTQ files with gzip compression

First set, 6 files, total size 50G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-7809fc0-Linux -t 16 -o ${TMPDIR} gz/hg002_gm2*gz; done
1:07.27 1124% 2834060k
1:05.94 1135% 2773328k
0:59.78 1207% 2822096k
1:00.72 1194% 2816040k
0:59.98 1202% 2815880k
0:59.84 1213% 2776056k
1:01.51 1179% 2827332k
0:58.53 1208% 2798140k
1:01.02 1198% 2797952k
0:59.89 1208% 2822984k

Second set, 8 files, total size 425G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-7809fc0-Linux -t 16 -o ${TMPDIR} gz2/BGISEQ500_PCRfree_NA24*gz; done
4:59.94 1307% 3406280k
5:08.17 1250% 3391604k
5:00.26 1294% 3404028k
5:07.00 1259% 3419708k
4:57.31 1303% 3367092k
5:10.78 1252% 3445280k
5:04.94 1269% 3437116k
5:00.33 1291% 3426028k
5:07.27 1273% 3441076k
5:11.20 1254% 3375832k

Timing for FASTQ files with BGZF compression

First set, 6 files, total size 51G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-b9a3139-Linux -t 32 -o ${TMPDIR} bgzf/*.gz; done
0:42.63 2831% 2830880k
0:41.58 2797% 2788752k
0:36.59 3050% 2837432k
0:34.88 3133% 2806804k
0:40.00 2935% 2838356k
0:38.51 2962% 2824916k
0:36.31 3086% 2842800k
0:40.12 2916% 2817992k
0:40.66 2892% 2830572k
0:40.63 2917% 2837608k

Second set, 8 files, total size 425G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-b9a3139-Linux -t 32 -o ${TMPDIR} bgzf2/*.gz; done
3:33.07 2840% 3452084k
3:25.99 2893% 3463548k
3:37.18 2750% 3489048k
3:38.07 2701% 3505196k
3:29.89 2756% 3477688k
3:08.00 2902% 3475684k
3:20.48 2804% 3439816k
3:36.38 2711% 3483976k
3:20.18 2813% 3466600k
3:30.97 2736% 3460804k

Timing for BAM files

4 files, total size 474G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-b9a3139-Linux -t 32 -o ${TMPDIR} bam/HG00*bam; done
5:10.75 2848% 1810640k
5:11.31 2864% 1816616k
5:10.35 2827% 1816548k
5:11.81 2829% 1822404k
5:17.07 2841% 1822712k
5:07.30 2922% 1812164k
5:21.32 2840% 1810252k
5:23.68 2719% 1814628k
5:26.97 2657% 1804548k
5:21.68 2772% 1798840k

Just one among those files:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ./falco-2.0.0-b9a3139-Linux -t 16 -o ${TMPDIR} bam/HG002.GRCh38.2x250.bam; done
4:45.92 763% 424700k
5:02.45 785% 424184k
4:39.79 759% 424324k
4:46.05 758% 423076k
4:45.85 770% 425160k
4:41.05 740% 425024k
4:58.66 761% 425592k
4:51.26 741% 423812k
4:47.14 751% 425412k
4:46.44 723% 425168k

These numbers will be updated with performance from falco v1.3.1 and FastQC.

[andrewdavidsmith]

FastQC v0.13.0.devel

Commit 54b336e

Equivalent runs to those above, with the same operating modes on by default (all except k-mers).

Regarding threads and cores
FastQC only uses up to 4 threads per input file, but 32 total are specified here. Although Falco v2 can use any number of threads, eventually as with any software, too many threads results in contention for queues and hardware. Understanding this is a machine-specific issue due to NUMA. If you have 64 separate physical cores, Falco v2 should be able to use all of them with some benefit, but this might be masked by performance of your storage device.

Timing for FASTQ files (plain)

FASTQ files without compression (6 files, total size 261G).

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FASTQC_DIR}/bin/fastqc -q --outdir ${TMPDIR} -t 32 plain/hg002_gm2*.fastq > /dev/null; done
5:37.04 1233% 3365460k
5:24.61 1269% 3356716k
5:26.28 1263% 3331364k
5:34.65 1255% 3352520k
5:35.48 1249% 3334628k
5:28.94 1269% 3342040k
5:41.17 1230% 3342904k
5:37.41 1229% 3334336k
5:39.72 1241% 3359032k
5:31.83 1252% 3337836k

Timing for FASTQ files with BGZF compression

First set, 6 files, total size 51G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FASTQC_DIR}/bin/fastqc -q --outdir ${TMPDIR} -t 32 bgzf/*.gz > /dev/null; done
5:03.25 1489% 3340628k
5:37.76 1364% 3360864k
5:24.76 1404% 3353396k
5:19.78 1429% 3346608k
5:38.32 1390% 3375092k
5:11.94 1434% 3355536k
5:47.79 1352% 3370912k
5:34.58 1398% 3369664k
5:36.44 1373% 3358600k
5:23.14 1419% 3361808k

Second set, 8 files, total size 425G:

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FASTQC_DIR}/bin/fastqc -q --outdir ${TMPDIR} -t 32 bgzf2/BGISEQ500_PCRfree_NA24*gz > /dev/null; done
19:22.66 1907% 3207504k
18:39.90 1951% 3253444k
18:10.41 1981% 3269572k
18:37.17 1971% 3275096k
18:17.97 1988% 3299652k
17:42.74 2058% 3192620k
18:03.01 1994% 3272152k
18:51.44 1958% 3276488k
18:52.04 1962% 3257204k
19:07.84 1951% 3252076k

Timing for FASTQ files with gzip compression

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FASTQC_DIR}/bin/fastqc -q --outdir ${TMPDIR} -t 32 gz/hg002_gm2*gz > /dev/null; done
5:34.29 1373% 3337032k
5:22.85 1389% 3361648k
5:26.51 1376% 3367840k
5:31.73 1362% 3335804k
5:11.26 1418% 3358416k
5:18.02 1383% 3347620k
5:33.72 1329% 3335612k
5:17.26 1395% 3343492k
5:24.30 1379% 3349240k
5:35.77 1334% 3354300k

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FASTQC_DIR}/bin/fastqc -q --outdir ${TMPDIR} -t 32 gz2/BGISEQ500_PCRfree_NA24*gz > /dev/null; done
17:51.07 2045% 3131016k
16:14.55 2194% 3208552k
17:25.09 2105% 3053264k
18:10.50 2014% 3109832k
18:57.01 1956% 3201648k
18:08.35 1983% 3202416k
17:26.69 2084% 3155280k
18:18.03 2013% 3042864k
18:20.53 2004% 3141148k
16:58.41 2129% 3068592k

Timing for BAM files

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FASTQC_DIR}/bin/fastqc -q --outdir ${TMPDIR} -t 32 bam/HG00*bam > /dev/null; done
41:28.72 1148% 2074624k
41:33.51 1146% 2079588k
43:08.09 1128% 2070912k
43:38.49 1111% 2032916k
43:00.87 1123% 2005492k
42:46.66 1114% 2041360k
43:54.64 1100% 2072532k
41:03.04 1167% 2076704k
42:55.49 1125% 2116312k
44:30.01 1079% 2052356k

[andrewdavidsmith]

Falco v1.3.2

Numbers for falco-1.3.2, which does not use threads at all:

Timing for FASTA files (plain)

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FALCO132}/falco -q -o ${TMPDIR} plain/*.fastq; done
36:09.49 98% 101916k
34:53.64 99% 101244k
34:09.78 99% 101144k
34:16.18 99% 101588k
35:14.87 99% 101220k
34:29.57 99% 102072k
34:22.23 99% 100944k
34:41.54 99% 101276k
34:31.45 99% 101660k
35:53.24 99% 101924k

Timing for one BAM file

$ for i in `seq 1 10`; do gtime -f "%E %P %Mk" ${FALCO132}/falco -q -o ${TMPDIR} bam/HG002.GRCh38.2x250.bam; done
1:06:08 99% 93816k
1:05:39 99% 93844k
1:05:36 99% 93940k
1:05:41 99% 93800k
1:09:12 99% 93896k
1:06:55 99% 93980k
1:05:46 99% 93912k
1:05:42 99% 93536k
1:05:59 99% 93732k
1:06:35 99% 93716k