distributedCombat_central crashes with “setting an array element with a sequence” when computing new-site harmonisation parameters (--harmo_only)

[martin-ef]

Summary
When computing harmonisation parameters for a new scanner site (--harmo_only), the pipeline crashes inside distributedCombat_central at the first feature (.label-avg.curvature.sm1). No AIDHS_H1_combat_parameters.hdf5 is produced. The crash is reproducible and independent of the cohort — it occurs identically with n=3, n=29 and n=30 healthy controls, and no subject is auto-excluded (INFO: exclude subjects []).

Environment
• AID-HS version: 1.1.0
• Run via Docker (docker compose run --rm aidhs)
• numpy: 1.22.4

Command

python scripts/new_patient_pipeline/new_patient_pipeline.py \
  -harmo_code H1 -ids subjects_list.txt -demos demographics_harmo.csv --harmo_only

• subjects_list.txt: 29 controls (single new scanner site)
• demographics_harmo.csv: columns ID, Harmo code, Group, Age at preoperative, Sex, 29 rows, all Harmo code = H1, Group = control, real ages (spread present, no zero-variance), real sex.

Log excerpt (leading to the crash)

INFO: exclude subjects []
PROCESS : COMPUTE HARMONISATION PARAMETERS
STEP: Compute combat harmonisation parameters for new site
.label-avg.curvature.sm1
['sub-LUM02', 'sub-LUM03', ..., 'sub-LUM30']
step1
step2
Must specify filename to output results as a file. Currently saving output to current workspace only.
TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/scripts/new_patient_pipeline/new_patient_pipeline.py", line 147, in <module>
    run_pipeline_preprocessing(
  File "/app/scripts/new_patient_pipeline/run_pipeline_preprocessing.py", line 285, in run_pipeline_preprocessing
    run_data_processing_new_subjects(subject_ids,
  File "/app/scripts/new_patient_pipeline/run_pipeline_preprocessing.py", line 189, in run_data_processing_new_subjects
    combat.get_combat_new_site_parameters(feature, demographic_file)
  File "/app/aidhs/data_preprocessing.py", line 600, in get_combat_new_site_parameters
    dc_out = dc.distributedCombat_central(
  File "/app/aidhs/distributedCombat.py", line 250, in distributedCombat_central
    ls1 = np.array([x["ls_site"][0] for x in site_outs]).astype(np.float64)
ValueError: setting an array element with a sequence.

Observations
• Crash is at distributedCombat.py line 250: ls1 = np.array([x["ls_site"][0] for x in site_outs]).astype(np.float64).
• The chained TypeError: only size-1 arrays can be converted to Python scalars indicates the per-site ls_site[0] matrices assembled across site_outs are not of matching shape (inhomogeneous), so the array cannot be cast to float64.
• Same crash location and message at n=3, n=29 and n=30 → independent of the specific subjects and of cohort size.
• No subjects were flagged/excluded by the pipeline’s own QC (exclude subjects []).

Root cause (measured)
In get_combat_new_site_parameters (aidhs/data_preprocessing.py), distributedCombat_central is called with two site pickles:

dc_out = dc.distributedCombat_central(
    [os.path.join(aidhs_combat_path, f'combat_{feature}.pickle'),          # shipped reference (H0)
     os.path.join(site_combat_path, f"{site_code}_{feature}_summary.pickle")],  # our new site, just created
    ref_batch='H0')

ls_site[0] = design.T @ design, shape (n_design_cols, n_design_cols) with
n_design_cols = n_batch + n_covariates.
We inspected both pickles for feature .label-avg.curvature.sm1:

REF (combat_.label-avg.curvature.sm1.pickle):        ls_site[0] = (5, 5) | n_batch = 2 | n_covariates = 3
NEW (H1_.label-avg.curvature.sm1_summary.pickle):    ls_site[0] = (4, 4) | n_batch = 2 | n_covariates = 2

Both have n_batch = 2, but the shipped reference pickle has 3 covariates while the new-site pickle has only 2. The two ls_site[0] matrices are therefore (5,5) vs (4,4) — inhomogeneous — so np.array([x["ls_site"][0] for x in site_outs]).astype(np.float64) at line 250 fails.
Our harmonisation cohort is all controls, so the group covariate has no variation. This is consistent with the new-site design ending up with one fewer covariate (2) than the shipped reference (3, built from a mixed patient/control cohort). This matches a “5 vs 4 design columns, controls-only cohort” behaviour that appears not to be fully resolved on the new-site harmonisation path.

Questions
1. Is a controls-only cohort supported for computing new-site harmonisation parameters? The reference pickle has 3 covariates but a controls-only new site yields 2, which makes distributedCombat_central crash.
2. Is there a controls-only reference pickle, a flag, or a documented workaround — or is this a bug in the new-site path that needs a fix (e.g. reference pickles regenerated with a matching covariate set)?

Happy to provide any further details.