Hartmut Surmann · Marc Thurow · Dominik Slomma · Niklas Digakis
The successor can be found at: stella_vslam_dense ⬅️
Additional material for the paper: "PatchMatch-Stereo-Panorama, a fast dense reconstruction from 360° video images"
| Abstract — This work proposes a new method for real-time dense 3d reconstruction for common 360° action cams, which can be mounted on small scouting UAVs during USAR missions. The proposed method extends a feature based Visual monocular SLAM (OpenVSLAM, based on the popular ORB-SLAM) for robust long-term localization on equirectangular video input by adding an additional densification thread that computes dense correspondences for any given keyframe with respect to a local keyframe-neighboorhood using a PatchMatch-Stereo-approach. While PatchMatch-Stereo-types of algorithms are considered state of the art for large scale Mutli-View-Stereo they had not been adapted so far for real-time dense 3d reconstruction tasks. This work describes a new massivelly parallel variant of the PatchMatch-Stereo-algorithm that differs from current approaches in two ways:
Keywords: PatchMatch-Stereo, 360°-Panorama, visual monocular SLAM, UAV, Rescue Robotics
git clone https://github.com/RoblabWh/PatchMatch
cd ~/PathMatch
docker build -t pmdvslam --build-arg NUM_THREADS=$(nproc) .
- Start Container
xhost +local:
nvidia-docker run -it --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix:ro -v /HOST/DATA/PATH:/data pmdvslam
- Run VSLAM (press "Terminate" to export and exit)
./run_video_slam -v /PATH/TO/ORB_VOCAB.dbow2 -c /PATH/TO/CONFIG.yaml -m /PATH/TO/VIDEO [--mask /PATH/TO/MASK] [-p /PATH/TO/DATABASE.msg]
- Export .msg to .ply (already included in normal .msg export)
python3 ./export_dense_msg_to_ply.py -i /PATH/TO/FILE.msg_dense -o /PATH/TO/OUTPUT.ply
- Start Container and mount this repository as docker volume to /data
cd ~/PatchMatch
xhost +local:
nvidia-docker run -it --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix:ro -v ${PWD}:/data pmdvslam
- Run Demo
./run_video_slam -v /data/orb_vocab/orb_vocab.dbow2 -c /data/example/patchmatch/config.yaml -m /data/example/patchmatch/video.mp4 --mask /data/example/patchmatch/mask.png -p /data/example/patchmatch/output.msg --frame-skip 2 --no-sleep
@INPROCEEDINGS{10018698,
author={Surmann, Hartmut and Thurow, Marc and Slomma, Dominik},
booktitle={2022 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR)},
title={PatchMatch-Stereo-Panorama, a fast dense reconstruction from 360° video images},
year={2022},
volume={},
number={},
pages={366-372},
doi={10.1109/SSRR56537.2022.10018698}}