Single model training with all good ground truth

[rohanbanerjee]

The goal of this issue is to track the training and performance of a single model(s) (instead of fine-tuning).

Result comparisons and observations: report

Steps involved:

1. Concatenate all the subjects from Systematic review of binary ground truth quality #25, Training and inference discussion for active learning round 1 #35, Training and inference discussion for active learning round 2 #38 and Training and inference discussion for active learning round 3 #40 into one dataset (will be called data_superset from now on)
2. Take the default 3d_fullres nnUNetv2 generated plan using nnunetv2_preprocess (here) and train a model. The plan configured by nnUNetv2 is attached below:
   nnUNetPlans.json
   To reproduce the experiment, run the following command:

nnUNetv2_preprocess -d <DATASET_ID> -c 3d_fullres

After the training was completed inference was run on the held-out_test set (#33) using this script

Note: This process is repeated for all the following comments also:

Attaching the QC for qualitative results:
held-out_test_common_bids_qc.zip

[rohanbanerjee]

Experiment 1: To check if changing the default spacing configured by the nnunetv2_preprocess affects the model performace

How
Customize the spacing hyperparameter (instructions). Attaching the nnUnetv2 plan for this below:
nnUNetPlans.json
To reproduce the experiment, run the following command:

nnUNetv2_preprocess -d <DATASET_ID> -c 3d_fullres_1_spacing -np 1

Attaching the QC for qualitative results:
held-out_test_common_1_bids_qc.zip

[rohanbanerjee]

Experiment 2: To check if changing the default patch_size configured by the nnunetv2_preprocess affects the model performace

How
Customize the spacing hyperparameter (instructions). Attaching the nnUnetv2 plan for this below:
nnUNetPlans.json
To reproduce the experiment, run the following command:

nnUNetv2_preprocess -d <DATASET_ID> -c 3d_fullres_mod_patch_size -np 1

Attaching the QC for qualitative results:
held-out_test_common_mod_patch_size_bids_qc.zip