MS lesion Ground Truth segmentation on MP2RAGE images

[Nilser3]

Description

Data

• nih-ms-mp2rage

As discussed here, the manual segmentations from algo-2 (3D nnUnet, for details see. #75) will be used to start creating MS GT

Here in the QC for nih-ms-mp2rage

Legend of QC masks

• label-lesion_algo1.nii.gz -> prediction of "algo 1" with binarization at 0.2
• label-lesion_algo2.nii.gz -> prediction of "algo 2"

Related Issues

#267 #56 #80 #75

[Nilser3]

Here in the QC for GT MS lesions for 104 subjects from nih-ms-mp2rage dataset

Legend of QC masks

• label-lesion_algo2.nii.gz -> prediction of algo 2 (3D nnUnet, for details see. Train model with nnUnet or MONAI ivadomed/model_seg_ms_mp2rage#75 (comment))

• label-lesion_desc-rater1.nii.gz -> Mask segmented by algo 2 corrected by rater 1 (Nilser)

[jcohenadad]

great job! these are the subjects where I think algo2 is better than the correction (and should therefore be kept as the 'GT'):

• sub-nih003_UNIT1.nii.gz: your seg is leaking outside the cord, and some of the 'diffuse' lesions are undersegmented
• sub-nih004_UNIT1.nii.gz: your seg is too big
• sub-nih005_UNIT1.nii.gz: i'm not sure there is actually a 2nd lesion caudally-- to be verified by neuroradio
• sub-nih006_UNIT1.nii.gz: your seg too big
• sub-nih007_UNIT1.nii.gz: your seg too big
• sub-nih008_UNIT1.nii.gz: your seg captures better pathology, but still too big. Eg you go outside the cord:
  
• sub-nih009_UNIT1.nii.gz: not sure this is pathology-- to validate
• sub-nih010_UNIT1.nii.gz: too big
• sub-nih011_UNIT1.nii.gz: too big
• sub-nih012_UNIT1.nii.gz: too big

I stopped at this subject, anticipating that the same trends goes on for the other subjects. I suggest you revisit your method for manual correction. That will also be much less work for you!

[Nilser3]

Thank you for your comments @jcohenadad
following the suggestions, here the QC for 45 patients.

TODO:

1. Create a list of subjects with lesions in doubt to be validated by a neuroradio.

[Nilser3]

Here in the QC for GT MS lesions for all nih-ms-mp2rage dataset

Legend of QC masks

• label-lesion_algo2.nii.gz -> prediction of algo 2 (3D nnUnet, from Train model with nnUnet or MONAI ivadomed/model_seg_ms_mp2rage#75 (comment))

• label-lesion_desc-rater1.nii.gz -> Mask segmented by algo 2 corrected by rater 1 (Nilser)

I have also indicated with the Artifact flag to some subjects.

[jcohenadad]

Excellent work @Nilser3!

My review 👉 qc_flags.json

Legend:

• ✅: Good to use as new ground truth (can be pushed to git-annex)
• ❌: Not a good GT (see below for specific changes).
• ⚠️: Excessive artefact that should be added to the exclude.yaml file so the image is not considered for training/testing.

In general, I find that you oversegment compared to the automatic segmentation.

Specific suggested changes:

• sub-nih061_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih062_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih063_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih090_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih141_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih149_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih165_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih166_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih173_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih175_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih179_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih181_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih182_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih196_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.

[Nilser3]

Thanks for your feedback @jcohenadad

I have push to the git-annex 163 MS maks (159 masks corrected by rater_1 ✅ + 14 made by algo-2 ❌)

• Masks from algo-2 accepted as GT: sub-nih061 sub-nih062 sub-nih063 sub-nih090 sub-nih141 sub-nih149 sub-nih165 sub-nih166 sub-nih173 sub-nih175 sub-nih179 sub-nih181 sub-nih182 sub-nih196)

branch: nlm/add_ms_lesion_gt
commit: 8187361e4f5143ffc6c8d93750a34a57e424a3a8

Redy for PR @mguaypaq

• TO DO : Subjects to be manually corrected again: sub-nih008 sub-nih009 sub-nih013 sub-nih017 sub-nih020 sub-nih021 sub-nih034 sub-nih037 sub-nih052 sub-nih054 sub-nih078 sub-nih079 sub-nih086 sub-nih095 sub-nih098 sub-nih134 sub-nih152 sub-nih159 sub-nih174 sub-nih177 sub-nih189 sub-nih192

[mguaypaq]

@Nilser3 I'm confused about the status of this issue. Is it ready for review, or are there still TODO items?

[Nilser3]

@mguaypaq
It's ready for your review,

The TODO list will be for a next PR

Thanks you,

[mguaypaq]

Ok, sorry for the delay. All the files are properly git-annexed. Merged into master.

[Nilser3]

QC of subjects with lesions to be reviewed with a neuroradiologist

[maxradx]

QC of subjects with lesions to be reviewed with a neuroradiologist

Hi @Nilser3 I see 22 subjects to be revised. I am not sure we will have time to go through all cases since we have 9 subjects for tSCI and dcm (other projects) (except if you have very specific questions). Is it possible to prioritize some?

[Nilser3]

Data

I would like to generate the MS ground truth from the recently git-annexed data: ms-rennes-mp2rage

• This data has no manual annotations
• commit: 4a6b2d32bf7dca400d96a17d71eb749b0db4b4eb

Automatic segmentations

• Preprocessing steps: Based on this tutorial
• SC mask: Applying seg_sc_contrast_agnostic from SCT v6.4
• MS mask : Applying seg_ms_lesion_mp2rage release r20240610 from SCT v6.4

Here the QC for ms-rennes-mp2rage

Legend of QC masks

1. label-lesion_seg.nii.gz -> prediction of seg_ms_lesion_mp2rage

@plbenveniste it would be good to see the multi-contrast model segmentations to decide on which mask to proceed with manual corrections.

Related issue:

#33

[plbenveniste]

Great work @Nilser3 !
I'll run my model this week and will add the QC here as well ! I'll let you know when it's done !

[plbenveniste]

I ran the following code which uses SCT branch plb/4682_add_ms_lesion_seg_model :

Code used

"""
This script is used to predict the MS lesion segmentation of the ms-rennes-mp2rage dataset using the model sct_deepseg -task ms_lesion_seg.
"""

import os
import sys
from pathlib import Path
from tqdm import tqdm

def main():

    # Define the path to the dataset
    path_data = '/home/GRAMES.POLYMTL.CA/p119007/label_ms_rennes/ms-rennes-mp2rage'
    path_output = '/home/GRAMES.POLYMTL.CA/p119007/label_ms_rennes/predictions'
    path_qc = '/home/GRAMES.POLYMTL.CA/p119007/label_ms_rennes/qc'
    path_output_sc = '/home/GRAMES.POLYMTL.CA/p119007/label_ms_rennes/predictions_sc'

    # get all files to segment using rglob
    files_to_segment = list(Path(path_data).rglob('*UNIT1.nii.gz'))

    # loop across all files
    for file in tqdm(files_to_segment):

        # build the output path for the lesion segmentation
        path_output_file_sc = os.path.join(path_output_sc, file.name)
        path_output_file_sc = path_output_file_sc.replace('.nii.gz', '_sc_seg.nii.gz')

        # build the output path for the lesion segmentation
        path_output_file_lesion = os.path.join(path_output, file.name)
        path_output_file_lesion  = path_output_file_lesion.replace('.nii.gz', '_seg.nii.gz')

        if os.path.exists(path_output_file_lesion) and os.path.exists(path_output_file_sc):
            print(f'File {file.name} already processed')
            continue

        # segment the spinal cord 
        if os.system(f'sct_deepseg -i {str(file)} -task seg_sc_contrast_agnostic -o {path_output_file_sc}') != 0:
            print(f'Error processing {str(file)}')
            return None
        
        # # segment the file using sct_deepseg
        if os.system(f'sct_deepseg -i {str(file)} -task seg_ms_lesion -o {path_output_file_lesion}') != 0:
            print(f'Error processing {str(file)}')
            return None

        # Generate the QC report
        if os.system(f'sct_qc -i {str(file)} -p sct_deepseg_lesion -d {path_output_file_lesion} -s {path_output_file_sc} -qc {path_qc} -plane sagittal') != 0:
            print(f'Error processing {str(file)}')
            return None
    
    print('All files processed successfully')


if __name__ == '__main__':
    main()

Here is the QC of the output: qc.zip

Also, I saw that some lesions were segmented in the brain in some cases which can easily be corrected using the spinal cord segmentation

[plbenveniste]

@Nilser3 I moved my predictions to this duke folder: ~/duke/temp/plben/label_ms_rennes

[Nilser3]

Thank you @plbenveniste
Here is the QC of both models in using the same background,

Legend of QC masks

1. label_ms_rennes_PL/predictions/sub-001_ses-20210225_UNIT1_seg.nii.gz -> multi-contrast model from @plbenveniste
2. label-lesion_seg.nii.gz -> prediction of seg_ms_lesion_mp2rage