Skip to content

[EARTH@MICCAI'2024] EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting

Notifications You must be signed in to change notification settings

HKU-MedAI/EndoGS

Repository files navigation

EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting

This is the official code for https://arxiv.org/abs/2401.11535.

Overview

Installation

Clone this repository and install packages:

git clone https://github.com/HKU-MedAI/EndoGS.git
conda env create --file environment.yml
conda activate gs
pip install git+https://github.com/ingra14m/depth-diff-gaussian-rasterization.git@depth
pip install git+https://github.com/facebookresearch/pytorch3d.git

Note: for the submodule diff-gaussian-rasterization of the 3D-GS, we use the depth branch of https://github.com/ingra14m/depth-diff-gaussian-rasterization.

Dataset

We use the dataset in EndoNeRF. Download the data from their website.

Use COLMAP to estimate the initial point clouds. Store the files (cameras.bin, images.bin, points3D.bin) in the data path (e.g., ./data/cutting_tissues_twice/sparse/).

Training

python train.py {data path} --workspace {workspace}
## e.g.,
python train.py data/cutting_tissues_twice/ --workspace output/cutting/

Inference

python inference.py {data path} --model_path {model path}
## e.g.,
python inference.py data/cutting_tissues_twice/ --model_path output/cutting/point_cloud/iteration_60000

Evaluation

python eval_rgb.py --gt_dir {gt_dir path} --mask_dir {mask_dir path} --img_dir {rendered image path}
## e.g.,
python eval_rgb.py --gt_dir data/cutting_tissues_twice/images --mask_dir data/cutting_tissues_twice/gt_masks --img_dir output/cutting/point_cloud/iteration_60000/render

Note: we should use the same masks in training and evaluation. If the name 'gt_masks' exist, we use 'gt_masks'; if not, use 'masks'. And we exclude the unseen pixels in gt and rendered images for PSNR.

Citation

If you find our work useful, please kindly cite as (v1 version of arxiv bib to avoid tracking missing):

@article{zhu2024deformable,
  title={Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting},
  author={Zhu, Lingting and Wang, Zhao and Jin, Zhenchao and Lin, Guying and Yu, Lequan},
  journal={arXiv preprint arXiv:2401.11535},
  year={2024}
}

Acknowledgement

  • The codebase is developed based on 3D-GS (Kerbl et al.), 4D-GS (Wu et al.), SuGaR (Guédon et al.), and EndoNeRF (Wang et al.).

About

[EARTH@MICCAI'2024] EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages