DeepEdit is a method that combines an automatic and a semi-automatic approach for 3D medical images into a single deep learning-based model. DeepEdit has three working modes: first, it can be used in click-free inference mode (similar to a regular segmentation network), providing fully-automatic segmentation predictions which can be used as a form of initialisation; second, it allows users to provide clicks to initialise and guide a semi-automatic segmentation model; lastly, given an initial segmentation, DeepEdit can be used to refine and improve the initial prediction by providing editing clicks. DeepEdit training process is similar to the algorithm proposed by Sakinis et al. DeepGrow - Gaussian-smoothed clicks for all labels and background are generated and added as input to the backbone CNN, but removes the minimum-click limitation of DeepGrow. Contrary to DeepGrow, DeepEdit model allows the prediction of an automatic segmentation-based initialisation without user-provided clicks, which can then be further edited by providing clicks. Additionally, DeepEdit can also be used for multi-label segmentation problems, allowing the user to generate/segment all labels simultaneously instead of one label at a time.
This tutorial contains an example to train a DeepEdit model and a notebook to run inference over a pre-trained model. The train file reads images and labels folders (imagesTr, labelsTr) as they come in the Medical Segmentation Decathlon.
More information about the transforms used in DeepEdit are also in the Notebook.
Important note:
This tutorial is intended to show how to train and test a DeepEdit model in MONAI Core library. Users may also find interesting DeepEdit model working in MONAI Label platform. There you can find how DeepEdit works along with 3D Slicer and/or OHIF.
Sakinis et al., Interactive segmentation of medical images through fully convolutional neural networks. (2019) https://arxiv.org/abs/1903.08205
A DeepEdit model could be trained on any 3D medical image dataset.
For this tutorial we used the public available dataset (Task09_Spleen) that can be downloaded from Medical Segmentation Decathlon
-
For questions relating to the use of MONAI, please us our Discussions tab on the main repository of MONAI.
-
For bugs relating to MONAI functionality, please create an issue on the main repository.
-
For bugs relating to the running of a tutorial, please create an issue in this repository.
- Download the Task09_Spleen zip file
- Decompressed the file
- Write the full path in the input flag in the train file
This is an extension for train.py that redefines basic default arguments to run 3D training.
# Run to know all possible options
python ./train.py -h
# Train a DeepEdit model
python ./train.py
--input deepedit/Task09_Spleen \
--output deepedit_model/ \
--epochs 100
# Train a DeepEdit model using multi gpu
# We recommend using the new PyTorch API
torchrun --standalone \
--nnodes=1 \
--nproc_per_node=`nvidia-smi -L | wc -l` train.py --input /PATH_TO_DATASET/ --output deepedit_model/ --epochs 100 --multi_gpu True
# Using to-be deprecated PyTorch API
python -m torch.distributed.launch \
--nproc_per_node=`nvidia-smi -L | wc -l` \
--nnodes=1 \
--node_rank=0 \
--master_addr="localhost" \
--master_port=1234 train.py --input /PATH_TO_DATASET/ --output deepedit_model/ --epochs 100 --multi_gpu True
# After training to export/save as torch script model
python ./train.py
--input deepedit_model/model.pt \
--output deepedit_model/model.ts \
--export trueThis notebook helps to run any pre-transforms before running inference over a DeepEdit single label model. It also helps to run post-transforms to get the final label mask.
By-default Tensorboard handlers are added as part of training/validation steps.