Peiqi Yang1, Zhangkai Ni1, Hanli Wang1, Wenhan Yang2, Shiqi Wang3, Sam Kwong4
1Tongji University, 2Peng Cheng Laboratory, 3City University of Hong Kong, 4Lingnan Univerity
This repository provides the official implementation for the paper "SVRF: Shell-guided Compression of Voxel Radiance Fields," IEEE Transactions on Image Processing, vol. 34, pp. 1179-1191, 2025. Paper
SVRF (Shell-guided compression of Voxel Radiance Fields) was designed to address the challenge of significant memory consumption and redundant components in large-scale voxel-based model, which are commonly encountered in real-world 3D reconstruction scenarios. In this work, we aimed at optimizing voxel-based model into a shell-like structure to reduce storage costs while maintaining rendering accuracy. Specifically, we first introduce a Shell-like Constraint, operating in two main aspects: 1) enhancing the influence of voxels neighboring the surface in determining the rendering outcomes, and 2) expediting the elimination of redundant voxels both inside and outside the surface. Additionally, we introduce an Adaptive Thresholds to ensure appropriate pruning criteria for different scenes. To prevent the erroneous removal of essential object parts, we further employ a Dynamic Pruning Strategy to conduct smooth and precise model pruning during training. The compression method we propose does not necessitate the use of additional labels. It merely requires the guidance of self-supervised learning based on predicted depth. Furthermore, it can be seamlessly integrated into any voxel-grid-based method. Extensive experimental results demonstrate that our method achieves comparable rendering quality while compressing the original number of voxel grids by more than 70%.
TL;DR: We introduce a novel method aimed at optimizing voxel-based model into a shell-like structure to reduce storage costs while maintaining rendering accuracy.
To start, we prefer creating the environment using conda:
conda create -n svrf
conda activate svrf
pip install -r requirements.txt
cd lib/cuda
python setup.py installPytorch and torch_scatter installation is machine dependent, please install the correct version for your machine.
Dependencies (click to expand)
PyTorch,numpy,torch_scatter: main computation.scipy,lpips: SSIM and LPIPS evaluation.tqdm: progress bar.mmengine: config system.opencv-python: image processing.imageio,imageio-ffmpeg: images and videos I/O.Ninja: to build the newly implemented torch extention just-in-time.einops: torch tensor shaping with pretty api.torch_efficient_distloss: O(N) realization for the distortion loss.
The primary datasets evaluated in our paper are listed below. Please download the rectified images and pose data as follows:
- NeRF
- NSVF
- BlendedMVS
- T&T (masked). Remember to fixing TanksandTemples dataset following this issue
- DeepVoxels
(click to expand;)
data
├── nerf_synthetic # Link: https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1
│ └── [chair|drums|ficus|hotdog|lego|materials|mic|ship]
│ ├── [train|val|test]
│ │ └── r_*.png
│ └── transforms_[train|val|test].json
│
├── Synthetic_NSVF # Link: https://dl.fbaipublicfiles.com/nsvf/dataset/Synthetic_NSVF.zip
│ └── [Bike|Lifestyle|Palace|Robot|Spaceship|Steamtrain|Toad|Wineholder]
│ ├── intrinsics.txt
│ ├── rgb
│ │ └── [0_train|1_val|2_test]_*.png
│ └── pose
│ └── [0_train|1_val|2_test]_*.txt
│
├── BlendedMVS # Link: https://dl.fbaipublicfiles.com/nsvf/dataset/BlendedMVS.zip
│ └── [Character|Fountain|Jade|Statues]
│ ├── intrinsics.txt
│ ├── rgb
│ │ └── [0|1|2]_*.png
│ └── pose
│ └── [0|1|2]_*.txt
│
├── TanksAndTemple # Link: https://dl.fbaipublicfiles.com/nsvf/dataset/TanksAndTemple.zip
│ └── [Barn|Caterpillar|Family|Ignatius|Truck]
│ ├── intrinsics.txt
│ ├── rgb
│ │ └── [0|1|2]_*.png
│ └── pose
│ └── [0|1|2]_*.txt
│
├── DeepVoxels # Link: https://drive.google.com/drive/folders/1ScsRlnzy9Bd_n-xw83SP-0t548v63mPH
└── [train|validation|test]
└── [armchair|cube|greek|vase]
├── intrinsics.txt
├── rgb/*.png
└── pose/*.txt
There are two training options available: one is training for a single scene, and the other is batch training for the entire dataset. The batch training code is inspired by VQRF.
$ python run.py --config configs/nerf/lego.py --render_test --render_fineUse --i_print and --i_weights to change the log interval.
The remaining scenes can be referenced using the commands in single_scene.sh.
# for nerf_synthetic datasets
python autotask_final.py -g "0 1" --configname syn
Set -g option according to the availible gpu on your machine. The remaining scenes can be referenced using the commands in batch_scenes.sh
To only evaluate the testset PSNR, SSIM, and LPIPS of the trained lego without re-training, run:
$ python run.py --config configs/nerf/lego.py --render_only --render_test \
--render_fine --eval_ssim --eval_lpips_vggUse --eval_lpips_alex to evaluate LPIPS with pre-trained Alex net instead of VGG net.
# for nerf_synthetic datasets
python autotask_eval_only.py -g "0 1 2 3 4 5 6 7" --configname syn
Or you can directly add --eval when training the model in batches:
python autotask_final.py -g "0 1 2 3 4 5 6 7" --configname syn --eval
$ python run.py --config configs/nerf/lego.py --render_only --render_videoUse --render_video_factor 4 for a fast preview.
If you find our work useful, please cite it as
@article{yang2025shell,
title={Shell-guided Compression of Voxel Radiance Fields},
author={Yang, Peiqi, and Ni, Zhangkai, and Wang, Hanli, and Yang, Wenhan and Wang, Shiqi and Kwong, Sam},
journal={IEEE Transactions on Image Processing},
volume={34},
pages={1179-1191},
year={2025},
publisher={IEEE}
}
This code is inspired by DVGO. We thank the authors for the nicely organized code!
Thanks for your attention! If you have any suggestion or question, feel free to leave a message here or contact Dr. Zhangkai Ni ([email protected]).