MIMIC: Multiple instance learning for Identification of Mycosis fungoides In Cutaneous biopsies

Inference with the released T1 and T2 models

infer_mimic.py is the standalone entry point for scoring slides with either MIMIC model. Its input is a WSI, WSI directory, or CSV of WSI paths. Using LazySlide, it performs the complete preprocessing internally: tissue-tile selection at the training resolution (256 px / 128 µm, no stain normalization), H-optimus-1 extraction, and MIL prediction. Extracted feature bags are cached for repeat runs.

A CUDA GPU is recommended for H-optimus-1 extraction.

T2 is the recommended model for MIMIC inference. In addition to LUMC/UMCU WSIs, T2 was trained with WSIs from the broader CLIDIPA registry and therefore has the preferred training coverage for deployment on new slides. Use T1 when reproducing the original T1 external-validation workflow or when that experiment is specifically required.

End-to-end LazySlide inference with uv

Create a dedicated inference environment from this repository root:

cd /exports/path-cutane-lymfomen-hpc/siemen/MF_BID_STUDY/MF_BID_Classification
uv venv .venv-inference --python 3.12
uv pip install --python .venv-inference/bin/python \
  -r requirements-inference.txt

The inference requirements pin the training-era PyTorch 2.6 / torchvision 0.21 stack, which supports the GPU architectures used for this study. Verify the installation and GPU before processing a WSI:

.venv-inference/bin/python -c \
  "import torch, lazyslide; print('CUDA:', torch.cuda.is_available())"
.venv-inference/bin/python infer_mimic.py --help

End-to-end T1 inference from one WSI, including attention:

.venv-inference/bin/python infer_mimic.py \
  --task t1 \
  --weights t1/T1/training/T1_cross_center/cv \
  --slides /path/to/slide.svs \
  --output predictions/t1_predictions.json \
  --attention-dir predictions/t1_attention \
  --cache-dir inference_cache

The same command supports T2 by changing --task and --weights:

.venv-inference/bin/python infer_mimic.py \
  --task t2 \
  --weights t2/T2/training/T2_clinical_simulation/cv \
  --slides /path/to/slides.csv \
  --output predictions/t2_predictions.json \
  --attention-dir predictions/t2_attention \
  --cache-dir inference_cache

LazySlide discovers tissue, creates 256 px tiles at 0.5 MPP (128 µm field of view), applies the H-optimus-1 preprocessing transform, and extracts the 1536-D features consumed by MIMIC. Cached bags and coordinates are reused on subsequent runs unless --force-extract is supplied.

Run all commands from this repository root:

cd /exports/path-cutane-lymfomen-hpc/siemen/MF_BID_STUDY/MF_BID_Classification
.venv-inference/bin/python infer_mimic.py --help

The --weights argument accepts any of:

• one best.ckpt file (an adjacent config.json is required);
• one fold_N directory;
• a cv directory containing fold_* directories; or
• the parent training directory containing cv/fold_*.

When a CV directory is supplied, all discovered folds are loaded and ensembled. Use the checked-in model weights as follows.

T1 inference

.venv-inference/bin/python infer_mimic.py \
  --task t1 \
  --weights t1/T1/training/T1_cross_center/cv \
  --slides /path/to/slide.svs \
  --output predictions/t1_predictions.json

T2 inference

.venv-inference/bin/python infer_mimic.py \
  --task t2 \
  --weights t2/T2/training/T2_clinical_simulation/cv \
  --slides /path/to/wsi_directory \
  --output predictions/t2_predictions.json

--slides accepts multiple values. Each value may be a WSI, a directory of WSIs, or a CSV with path and optional slide columns:

slide,path
slide_001,/data/slides/slide_001.svs
slide_002,/data/slides/slide_002.mrxs

Request tile-level attention scores with --attention-dir:

.venv-inference/bin/python infer_mimic.py \
  --task t1 \
  --weights t1/T1/training/T1_cross_center/cv \
  --slides /path/to/slides.csv \
  --output predictions/t1_predictions.json \
  --attention-dir predictions/attention

Exact inference output

Use a .json output path for the recommended output format. The file is a JSON dictionary keyed by slide ID. Each value preserves the input metadata and contains these primary prediction fields:

• platt_prob: Platt-scaled probability of MF (class 1). The released MIMIC folds store temperature scalers, which are zero-intercept Platt scalers on the binary logit margin. This is null if a supplied model has no scaler.
• raw_prob: uncalibrated probability of MF, equal to sigmoid(logit).
• logit: uncalibrated binary logit margin, class_1_logit - class_0_logit.
• attention_path: absolute path to the slide's compressed attention store. This key is present only when --attention-dir was requested and the model returned attention.

For example:

{
  "slide_001": {
    "path": "/data/slides/slide_001.svs",
    "platt_prob": 0.8123,
    "raw_prob": 0.7741,
    "logit": 1.232,
    "attention_path": "/results/attention/slide_001.attention.npz"
  }
}

The full dictionary also records predicted_label, predicted_label_argmax, decision_threshold, n_tiles, n_models, task, and class-wise probability aliases. Class 0 is BID and class 1 is MF. Each attention NPZ contains aligned attention and coords arrays; coordinates are level-0 WSI pixel coordinates. A .csv output path remains supported for tabular compatibility and contains the same per-slide fields as columns.

Add --recursive for nested WSI directories, --force-extract to ignore cached features, or change --cache-dir to place the cache elsewhere.

Container

Build from this repository root:

cd /exports/path-cutane-lymfomen-hpc/siemen/MF_BID_STUDY/MF_BID_Classification
docker build -f Dockerfile.inference -t mimic-inference .

Run with NVIDIA Container Toolkit, mounting WSIs, outputs, model weights, and a persistent feature cache:

docker run --rm --gpus all \
  -v /path/to/slides:/slides:ro \
  -v /path/to/results:/results \
  -v "$PWD/t1:/weights/t1:ro" \
  -v /path/to/mimic_cache:/cache \
  mimic-inference \
  --task t1 \
  --weights /weights/t1/T1/training/T1_cross_center/cv \
  --slides /slides \
  --output /results/t1_predictions.json \
  --attention-dir /results/attention \
  --cache-dir /cache

Run the Docker smoke suite on a Docker-enabled host:

bash scripts/test_docker_inference.sh

This builds the image from scratch, inspects it, checks the entry point, compiles the inference modules, and imports the pinned runtime stack. To also run a GPU end-to-end T2 inference and validate the JSON plus attention store:

MIMIC_WSI=/absolute/path/to/slide.svs \
MIMIC_T2_WEIGHTS="$PWD/t2/T2/training/T2_clinical_simulation/cv" \
MIMIC_RESULT_DIR=/absolute/path/to/docker-smoke-results \
bash scripts/test_docker_inference.sh

Apptainer (HPC)

Build and test the native Apptainer image:

module load container/apptainer/1.5.3/gcc-8.5.0
apptainer build mimic-inference.sif Apptainer.inference.def
apptainer test mimic-inference.sif

Validate the supplied image with:

MIMIC_APPTAINER_SKIP_BUILD=1 \
MIMIC_APPTAINER_IMAGE=/path/to/mimic-inference.sif \
bash scripts/test_apptainer_inference.sh

Run T2 inference with GPU passthrough:

apptainer run --nv \
  --bind /path/to/slides:/slides:ro \
  --bind "$PWD/t2:/weights/t2:ro" \
  --bind /path/to/results:/results \
  --bind /path/to/mimic_cache:/cache \
  mimic-inference.sif \
  --task t2 \
  --weights /weights/t2/T2/training/T2_clinical_simulation/cv \
  --slides /slides \
  --output /results/t2_predictions.json \
  --attention-dir /results/attention \
  --cache-dir /cache

The automated Apptainer suite performs the build, embedded %test, inspection, entry-point smoke test, and—when paths are supplied—GPU T2 inference plus JSON and attention-store validation:

MIMIC_WSI=/absolute/path/to/slide.svs \
MIMIC_T2_WEIGHTS="$PWD/t2/T2/training/T2_clinical_simulation/cv" \
MIMIC_RESULT_DIR=/absolute/path/to/apptainer-smoke-results \
bash scripts/test_apptainer_inference.sh

flowchart TD
    A["CLIDIPA cohort"] --> B["PathBench-MIL<br/>feature extraction"]
    B --> C["Slide bags<br/>SLIDE.pt + .index.npz"]
    C --> D["T1 training<br/>10-fold CV<br/>(LUMC + UMCU)"]
    C --> E["T2 training<br/>10-fold CV<br/>(all except UMCU-CS)"]
    D --> F["T1 inference<br/>(slide-level)"]
    E --> G["T2 inference<br/>(slide-level)"]
    G --> H["Case-level<br/>aggregation"]
    F --> I["Calibration +<br/>threshold decisions"]
    G --> I
    H --> I
    I --> J["Clinical decision<br/>analysis"]
    I --> K["Robustness<br/>analysis"]
    I --> L["Heatmaps /<br/>top tiles"]
    I --> M["Rater-study +<br/>GRM comparison"]

Loading

1) What MIMIC is and what this repository contains

This repository contains the codebase for MIMIC (Multiple instance learning for Identification of Mycosis fungoides In Cutaneous biopsies):

• model training for MIL classifiers,
• slide/case-level inference,
• calibration and clinical analysis,
• explainability/heatmap generation,
• robustness and rater-study analyses.

Study context and data governance

• The study uses data from the CLIDIPA registry: https://clidipa.org/the-registry/
• Weights and patient data are not publicly available due to patient privacy constraints and CLIDIPA data-sharing rules.

This repo therefore provides the full processing logic and expected I/O contracts, but not protected clinical data artifacts.

---

2) Installation

Option A — pip (existing workflow)

pip install -r requirements.txt

Option B — uv project workflow

pyproject.toml is included to support uv-managed environments.

uv sync

If you want to mirror legacy requirements installation exactly:

uv pip install -r requirements.txt

Note: the old requirements included a machine-local editable MIL-Lab path. The pyproject.toml uses a portable MIL-Lab source dependency instead.

---

3) Feature extraction with PathBench-MIL (required upstream)

MIMIC expects pre-extracted tile feature bags. It does not tile WSIs itself.

Use PathBench-MIL for feature extraction:

• https://github.com/Sbrussee/PathBench-MIL

Expected bag format in this repo

For each slide, MIMIC expects:

• SLIDE_ID.pt → tile feature matrix
• index.npz or SLIDE_ID.index.npz → tile coordinate array (arr_0) with shape (N_tiles, 2)

In practice:

• *.pt files contain tile feature vectors.
• *.index.npz sidecars contain corresponding tile (x, y) locations.

Example folder

/features/
  UMCU_T24-00126_1A.pt
  UMCU_T24-00126_1A.index.npz
  UMCU_T24-00318_2A.pt
  UMCU_T24-00318_2A.index.npz

Example slide metadata CSV (minimum)

slide,patient,category,dataset,case
UMCU_T24-00126_1A,P001,MF,UMCU,CASE_001
UMCU_T24-00318_2A,P002,BID,UMCU,CASE_002

WSI annotations format (recommended, next to feature bags)

Use a slide-annotation CSV alongside the extracted PathBench-MIL features. At minimum include slide; commonly used columns are shown below:

patient,category,dataset,slide
PATIENT_A,MF,CENTER_A,SLIDE_A
PATIENT_B,BID,CENTER_A,SLIDE_B
PATIENT_X,MF,CENTER_B,SLIDE_X
PATIENT_X,MF,CENTER_B,SLIDE_Y

---

4) MIL-LAB integration (model architectures)

MIMIC uses MIL model architectures through the MIL-LAB ecosystem (ABMIL/TransMIL/CLAM-style families depending on selected variant).

How this repo treats MIL-LAB:

• training/inference wrappers live in this repository (src/train_mil.py, src/mil_module.py, src/inference_engine.py),
• core architecture construction is delegated to MIL-LAB-backed builders,
• architecture variants are selected via model/variant arguments in experiment and training commands.

This separation keeps study logic (splits, calibration, analysis) in MIMIC while model backbones remain modular.

---

5) T1 workflow: train + infer + output interpretation

T1 is the slide-level external validation workflow.

5.1 How T1 was trained

T1 uses 10-fold cross-validation with training/validation on T1-configured centers (typically LUMC + UMCU setup in the pipeline configuration).

python src/pipelines.py t1 \
  --csv /path/to/annotations.csv \
  --feature-root /path/to/features \
  --tfrecord-root /path/to/tfrecords \
  --train-centers LUMC,UMCU \
  --val-centers UMCU \
  --test-centers MINDEN,TURIN,UMCU-rater,ZURICH,WUERZBURG \
  --out-dir ./outputs

5.2 Inference (standalone)

python src/inference_cli.py \
  --run-dir ./outputs/T1/training/T1_cross_center \
  --slides-csv /path/to/annotations.csv \
  --features-dir /path/to/features \
  --out-csv ./outputs/T1/inference/t1_full_inference.csv

5.3 How the decision threshold was determined

In the orchestrated training flow, threshold optimization is enabled via --opt-threshold (in experiment commands).

Operationally:

• each CV fold produces validation predictions,
• an optimal threshold is selected on validation outputs,
• inference CSV exports threshold-based decisions using this value (predicted_label_threshold, decision_threshold).

Argmax decisions (predicted_label_argmax) remain separate and do not require an optimized threshold.

5.4 Calibration: validation-based and prevalence-based

MIMIC inference reports two calibration stages:

1. Validation-set calibration (temperature scaling)

   • confidence calibration learned from validation logits,
   • exported as columns such as prob1_validation_calibrated.

2. Prevalence-based calibration (prior-shift correction)

   • adjusts probabilities for deployment prevalence mismatch,
   • exported as columns such as prob1_prior_calibrated.

For T2 clinical simulation, target prevalence 0.203 is used, and this prevalence was determined from measured prevalence in UMCU skin biopsies with MF suspicion.

5.5 What inference outputs contain

Main output is one row per slide with prediction and calibration provenance.

Common columns:

• identity/meta: slide, optional passthrough columns like patient, dataset, case
• decisions: predicted_label, predicted_label_argmax, predicted_label_threshold, decision_threshold
• probabilities:
  • prob1_uncalibrated
  • prob1_validation_calibrated (temperature scaling)
  • prob1_prior_calibrated (target prevalence correction)
  • compatibility aliases (e.g., prob_class1)
• diagnostics: inference_time_sec, optional CPU/RAM/GPU usage columns

Example output row (illustrative):

slide,predicted_label,predicted_label_argmax,predicted_label_threshold,decision_threshold,prob1_uncalibrated,prob1_validation_calibrated,prob1_prior_calibrated,prob_class1,dataset
UMCU_T24-00126_1A,1,1,1,0.62,0.71,0.68,0.74,0.68,UMCU

For detailed field definitions, see INFERENCE.MD.

5.6 Run T2 slides with the T1-trained model

If you want to score the T2 clinical-simulation slide set with the T1 trained model, point inference_cli.py at the T1 model and pass the T2 slides CSV. If you also want T2-style grouped outputs, request case/patient aggregation at inference time:

python src/inference_cli.py \
  --model-package ./outputs/model_package \
  --package-model t1 \
  --slides-csv /path/to/t2_slides.csv \
  --features-dir /path/to/features \
  --target-prevalence 0.203 \
  --grouped-inference-cols case,patient \
  --out-csv ./outputs/t2_slides_scored_with_t1.csv

This writes:

• slide-level predictions to t2_slides_scored_with_t1.csv
• optional grouped outputs to t2_slides_scored_with_t1_case_level.csv and t2_slides_scored_with_t1_patient_level.csv

5.7 Monte Carlo inference

To run stochastic inference with dropout enabled, use --mc-dropout-passes. This performs repeated stochastic forward passes per bag and exports the probability distribution plus confidence intervals:

python src/inference_cli.py \
  --model-package ./outputs/model_package \
  --package-model t1 \
  --slides-csv /path/to/t2_slides.csv \
  --features-dir /path/to/features \
  --mc-dropout-passes 100 \
  --mc-ci-level 0.95 \
  --out-csv ./outputs/t2_slides_t1_mc100.csv

Key extra output columns:

• prob1_mc_mean, prob1_mc_std
• prob1_mc_ci_lower, prob1_mc_ci_upper
• prob1_mc_samples

---

Mathematical notes (calibration, inference, AUC, GRM, explainability)

Inference probabilities and thresholding

For binary logits $z=(z_0,z_1)$, class probabilities are computed via softmax:

$$p(y=1\mid x)=\frac{e^{z_1}}{e^{z_0}+e^{z_1}}.$$

Threshold-based decision uses a cutoff $\tau$:

$$\hat y_\tau=\mathbb{1}[p(y=1\mid x)\ge \tau].$$

Argmax decision is:

$$\hat y_{\mathrm{argmax}}=\arg\max_{c\in\{0,1\}} z_c.$$

Validation calibration (temperature scaling)

Given logits $z$ and learned temperature $T>0$, calibrated probabilities are:

$$p_T(y=c\mid x)=\frac{\exp(z_c/T)}{\sum_k \exp(z_k/T)}.$$

Prevalence/prior-shift calibration

Given training prevalence $\pi_0$, deployment prevalence $\pi_1$, and model probability $\hat p$:

$$\hat p' = \frac{\frac{\pi_1}{\pi_0}\hat p} {\frac{\pi_1}{\pi_0}\hat p + \frac{1-\pi_1}{1-\pi_0}(1-\hat p)}.$$

AUC (ROC area)

With ROC parameterized by threshold $t$, $\mathrm{TPR}(t)$ and $\mathrm{FPR}(t)$:

$$\mathrm{AUC}=\int_0^1 \mathrm{TPR}(u)\, d u, \quad u=\mathrm{FPR}.$$

Equivalent rank interpretation:

$$\mathrm{AUC}=P(s^+ > s^-)+\tfrac{1}{2}P(s^+=s^-),$$

where $s^+$ and $s^-$ are positive/negative scores.

GRM (Samejima-style graded response model)

For ordered category $k$, discrimination $a_i$, ability $\theta_r$, threshold $b_{ik}$:

$$P(Y_i \ge k \mid \theta_r) = \sigma\!\big(a_i(\theta_r-b_{ik})\big),$$

with $\sigma(\cdot)$ logistic sigmoid. Category probability:

$$P(Y_i = k \mid \theta_r)=P(Y_i\ge k\mid\theta_r)-P(Y_i\ge k+1\mid\theta_r).$$

Attention / IG / LRP heatmap scoring (high-level)

• Attention heatmaps visualize normalized attention weights $\alpha_j$ over tiles:

$$\sum_j \alpha_j = 1,\quad \alpha_j\ge 0.$$

• Integrated Gradients tile attribution for feature $x_j$:

$$\mathrm{IG}_j(x)=(x_j-\tilde x_j)\int_0^1 \frac{\partial F(\tilde x+\beta(x-\tilde x))}{\partial x_j}\, d\beta.$$

• LRP relevance approximately conserves output score through layers:

$$\sum_j R_j \approx F(x).$$

These values are mapped back to tile coordinates from .index.npz to produce spatial heatmaps and top-tile rankings.

---

6) Rater-study processing and model-vs-pathologist comparison

Use the rater-study pipeline:

python src/pipelines.py rater_study \
  --rater-csv /path/to/rater_study.csv \
  --model-csv /path/to/t1_or_t2_inference.csv \
  --out-dir ./outputs/rater_study

Expected rater input

• delimiters: ; or ,
• required slide identifier column: Image or Slide
• remaining non-metadata columns are treated as individual raters

Example:

Diagnosis;Center;Image;rater1;rater2;rater3
MF;CENTER_A;SLIDE_A;Mycosis fungoides (uncertain);Mycosis fungoides (probable);Completely uncertain diagnosis
BID;CENTER_A;SLIDE_B;Inflammatory dermatosis (probable);Inflammatory dermatosis (uncertain);Completely uncertain diagnosis

Linear mapping used for AUC-style comparisons

Rater answers are linearly mapped to ordinal scores in {-2,-1,0,+1,+2}:

• Inflammatory dermatosis (probable) → -2
• Inflammatory dermatosis (uncertain) → -1
• Completely uncertain diagnosis → 0
• Mycosis fungoides (uncertain) → +1
• Mycosis fungoides (probable) → +2

These mapped scores are used for reader/model comparison analyses.

If scores are linearly mapped to $[0,1]$, one common mapping is:

$$s_{01}=\frac{s+2}{4}, \quad s\in\{-2,-1,0,+1,+2\}.$$

This can be compared directly with model probabilities $P(\mathrm{MF})$ for probability-style discrimination analyses (e.g., ROC/AUC). This is a reasonable and commonly used pragmatic comparison when treating ordinal rater confidence as a monotonic surrogate probability.

GRM (graded response model) fit

src/grm_model.py fits a Samejima-style GRM to compare:

• pathologist/model propensity/ability behavior,
• slide-level discriminative signal,
• decision thresholds (ordered category boundaries),
• agreement/correlation patterns between MIMIC and pathologist signals.

Standalone GRM example:

python src/grm_model.py \
  --csv /path/to/rater_study.csv \
  --model-csv /path/to/inference.csv \
  --prob-col prob_class1 \
  --out-dir ./outputs/grm

Typical GRM outputs include per-rater ability estimates, item/slide threshold summaries, and comparative visual/statistical artifacts.

---

7) T2 workflow: train + infer + case aggregation + clinical analysis

T2 is the clinical simulation workflow with case-level evaluation.

7.1 How T2 was trained

T2 uses 10-fold cross-validation on all datasets except UMCU-CS (UMCU Clinical Simulation Set), with evaluation on the held-out UMCU-CS center.

python src/pipelines.py t2 \
  --csv /path/to/annotations.csv \
  --feature-root /path/to/features \
  --tfrecord-root /path/to/tfrecords \
  --train-centers LUMC,UMCU,MINDEN,TURIN,UMCU_rater,ZURICH,WUERZBURG \
  --clinical-sim-center UMCU_CS \
  --target-prevalence 0.203 \
  --t2-case-col case \
  --out-dir ./outputs

T2 process summary:

• train/validate by 10-fold CV on all datasets except UMCU_CS,
• infer on --clinical-sim-center,
• apply prevalence correction with --target-prevalence 0.203 (UMCU cohort-derived),
• aggregate slide predictions to case level.

7.2 What is produced

• slide-level inference CSV (one row per slide)
• grouped/case-level inference CSV (one row per case)
• case-level clinical decision analysis artifacts

Case-level aggregation step (standalone):

python src/aggregate_case_inference.py \
  --in-csv ./outputs/T2/inference/t2_slide_level_inference.csv \
  --out-csv ./outputs/T2/inference/t2_case_level_inference.csv \
  --case-col case

Example case-level output row (illustrative):

case,n_slides,prob_class1,predicted_label_threshold,predicted_label_argmax,classification_cutoff
CASE_001,3,0.81,1,1,0.62

---

8) Heatmaps and top-tile generation

Heatmaps/attribution maps are generated through src/visualization.py (also wired into T1/T2 pipelines).

Standalone example:

python src/visualization.py \
  --run-dir /path/to/run \
  --slides-csv /path/to/slides.csv \
  --slides-dir /path/to/wsi \
  --bags-root /path/to/features \
  --do-attention \
  --do-ig \
  --do-lrp \
  --out-dir ./outputs/visualization

Expected outputs include per-slide interpretability artifacts (attention/IG/LRP maps) and tile-ranking products used for qualitative review of top informative regions.

---

9) Robustness analysis

Run robustness analysis on an inference CSV:

python src/robustness_analysis.py ./outputs/T1/inference/t1_full_inference.csv \
  --out-dir ./outputs/robustness \
  --positive-category MF \
  --prob-col prob_class1 \
  --center-col dataset \
  --annotations-csv /path/to/annotations.csv

Optional extra inputs:

• --reader-csv for pathologist reader summaries,
• --slide-metrics-csv for slide-level metric augmentation.

Input expectation summary:

• required inference CSV with prediction probabilities,
• expected label/category context via inference columns and/or annotations CSV.

Output summary:

• center- and subgroup-oriented robustness metrics,
• plots/CSV summaries under --out-dir.

---

10) Acknowledgements

• CLIDIPA registry and participating centers/pathologists.
• PathBench-MIL for feature extraction tooling.
• MIL-LAB for MIL architecture ecosystem support.

11) Citation

If you use this repository, please cite the corresponding MIMIC study publication.