WholeBody CT Segmentation (104 classes) | SegResNet | TotalSegmentator

[tangy5]

Add whole body 104 tissues segmentation bundle.

• Inference pipeline
• Training pipeline
• Benchmarking
• Performance Figures
• Add Hardware requirements after benchmarking, this is a huge model, a minimum 32G GPU is required. Add more details.
• Other checks
• Utilities

[tangy5]

Thanks, most basic config files are done, I'm benchmarking and training using A100 GPUs.
I need to get performance comparison, train loss, validation dice figures.
Hopefully we will get a finalized model trained with A100 GPUs.
Will notice for review.

[tangy5]

Hi @yiheng-wang-nv , @Nic-Ma , this bundle is finished. All components are developed and evaluated. Please see details in the doc.
It would be great that you could give a look and see whether any part needs modification, especially formats, styles.

In additon, the CI checks failed on "code-format-check", caused by other files.

Thank you so much.

[tangy5]

Hi @SachidanandAlle, this bundle is done, it would be great if you could give a review. I tested it with MONAI Label.

I will create a PR in MONAI Label repo to support "options" for models versions and inference options such as "sw_batch_size", "sw_overlap".

Thanks

[yiheng-wang-nv]

Thanks @tangy5 , the code format check issue will be resolved soon.

[tangy5]

Hi @yiheng-wang-nv @Nic-Ma ,

All above suggestions are uploaded and addressed. I'm ok to merge it.
Let me know if there are further comments. Thanks!

[yiheng-wang-nv]

/build

[yiheng-wang-nv]

/build

[ibro45]

Hi,

I have two questions:

1. How was the configuration selected? Did you use Auto3DSeg, and if so, with what constraints?

2. There seems to be no test set DICE scores. I compared your 1.5mm model against the original fast (3mm, upsampled to 1.5mm) and slow (1.5mm) nnUNet Totalsegmentator models:

Did you also observe a DICE of 0.74 in the case of your network or am I getting something wrong?

[diazandr3s]

Hi @ibro45,

Very good questions.

1. How was the configuration selected? Did you use Auto3DSeg, and if so, with what constraints?

The whole-body CT segmentation model uses a single network (SegResNet). Both Auto3D and nnUNET train an ensemble of networks with different hyperparameters. This means more computational resources might be needed to train and perform inference when using those two algorithms

2. There seems to be no test set DICE scores. I compared your 1.5mm model against the original fast (3mm, upsampled to 1.5mm) and slow (1.5mm) nnUNet Totalsegmentator models:

I wouldn't be surprised if the performance of the whole-body CT segmentation model is slightly worse when compared to nnUNET or Auto3D. Essentially because this model uses a single network instead of an ensemble.

A good exercise would be to compare the Auto3D against the nnUNET results on this dataset.

[ibro45]

Thanks @diazandr3s!

I see, I haven't worked with Auto3D yet, but I assumed that it, like nnUNet, configures the network, patch and batch size, etc. based on the dataset. So I was just wondering if that was done first to figure out the "ideal" configuration before training it.

Is there any particular reason why the test scores aren't reported? I can't find any in MONAI VISTA either, where TotalSegmentator was used too.

I'm actually developing a single SegResNet (same architecture as wholebody_ct) for all 104 labels and am getting close to the original 5-model nnUNet TotalSegmentator performance. I was just wondering about the design choices as I'm debating whether I should also adjust the current SegResNet architecture and see if it improves the performance.

[diazandr3s]

Hi @ibro45,

I see, I haven't worked with Auto3D yet, but I assumed that it, like nnUNet, configures the network, patch and batch size, etc. based on the dataset. So I was just wondering if that was done first to figure out the "ideal" configuration before training it.

That's correct, the Auto3D algorithm configures those hyperparameters. You can learn more about this model here: https://github.com/Project-MONAI/tutorials/tree/main/auto3dseg#performance-benchmarking

For the whole-body model, we used the vanilla SegResNet. No major network hyperparameter search was performed.

Is there any particular reason why the test scores aren't reported? I can't find any in MONAI VISTA either, where TotalSegmentator was used too.

The initial idea of proposing the whole-body CT segmentation was not to show a better performance than the nnUNET or Auto3D. It was more to demonstrate how the whole body can be segmented using a single network instead of an ensemble. Easier to train and faster inference for downstream tasks.

With regard to the MONAI VISTA, you should hear more details about this model soon. Stay tuned!

I'm actually developing a single SegResNet (same architecture as wholebody_ct) for all 104 labels and am getting close to the original 5-model nnUNet TotalSegmentator performance. I was just wondering about the design choices as I'm debating whether I should also adjust the current SegResNet architecture and see if it improves the performance.

This is great! Please let us know how that goes. Looking forward to hearing more.