An Interpretable Deep Learning Workflow for Discovering Sub-Visual Abnormalities in CT Scans of COVID-19 Inpatients and Survivors
This repository provides the method described in the paper:
Longxi Zhou, et al. "An Interpretable Deep Learning Workflow for Discovering Sub-Visual Abnormalities in CT Scans of COVID-19 Inpatients and Survivors", Nature Machine Intelligence 2022.
- Colab Link: https://github.com/LongxiZhou/DLPE-method/blob/master/colab.ipynb (Note DLPE method requires GPU ram >= 4 GB and CPU ram >= 24 GB, so a free colab account can not hold this.)
- Open Accessed Article: https://www.nature.com/articles/s42256-022-00483-7
- All Tricks Explained (36 pages for detailed manual for the replication of our study): https://static-content.springer.com/esm/art%3A10.1038%2Fs42256-022-00483-7/MediaObjects/42256_2022_483_MOESM1_ESM.pdf
- Fast and state-of-the-art segmentation models for COVID-19 lesions, lungs, airways and blood vessels.
- The first segmentation model for COVID-19 sub-visual lesions.
- A method to remove scan-level bias of chest CT data.
- A method to calculate the optimal window for observing pulmonary parechyma.
- A method to quantify good radiomics.
Remove Scan-level Bias: Same Model, Same Training Protocol, Same Ground Truth, DIFFERENT Performance!
Deep-LungParenchyma-Enhancing (DLPE) is a computer-aided detection (CADe) method for detecting and quantifying pulmonary parenchyma lesions on chest computerized tomography (CT). Using deep-learning, DLPE removes irrelevant tissues other than pulmonary parenchyma, and calculates the scan-level optimal window which enhances parenchyma lesions for dozens of times compared to the lung window. Aided by DLPE, radiologists discovered novel and interpretable lesions from COVID-19 inpatients and survivors, which are previously invisible under the original lung window and have strong predictive power for key COVID-19 clinical metrics and sequelae.
- Step 1): Download the file: "trained_models/" and "example_data/" from Google Drive.
- Step 2): Dowload the source codes from github (note in github, "trained_models/" and "example_data/" are empty files).
- Step 3): Replace the "trained_models/" and "example_data/" with Google Drive downloaded.
- Step 4): Establish the python environment by './environment.yml'.
- Step 5): Open 'dcm_to_enhanced.py', follow the instructions to change global parameters "trained_model_top_dict", "dcm_directory" and "enhance_array_output_directory".
- Step 6): Run './dcm_to_enhanced.py'.
- DLPE method requires GPU ram >= 4 GB and CPU ram >= 24 GB.
- Enhancing one chest CT scan needs about two minutes on one V100 GPU.
- Step 1): Open "Reproduce Follow-up Dataset/rescaled_ct_compressed/" from Google Drive. All normalized follow-up CT scans are stored here. Each normalized scan is a 'float32' numpy array with shape (512, 512, 512).
- Step 2): Load one of these normalized scan.
- Step 3): Open './dcm_to_enhanced.py', see line 24. Change the "rescaled_ct_array" with what you loaded in Step 2).
- Step 4): Run the following part of './dcm_to_enhanced.py'. You will get the enhanced arrays used in our study. You may also check the segmentations for lungs, airways and blood vessels, e.g. visualize them in 3D stl. Many hospitals already used our models to segment airways and blood vessels for operations.
- All expected outputs including lung masks, airway masks, blood vessel masks and the enhanced arrays for follow-up CT are stored in "Reproduce Follow-up Dataset/expected_outputs/" from Google Drive. "Reproduce Follow-up Dataset/visualize_chest_tisse_seg/" stores the 2D semantic map for lungs (green), airways (blue) and blood vessels (red).
If you request our training code for DLPE method, please contact Prof. Xin Gao at [email protected].
