This repository contains the official implementation of the proposed framework in manuscript.
DQS3D: Densely-matched Quantization-aware Semi-supervised 3D Detection
Authors: Huan-ang Gao, Beiwen Tian, Pengfei Li, Hao Zhao, Guyue Zhou
Institute for AI Industry Research (AIR), Tsinghua University
In this paper, we study the problem of semi-supervised 3D object detection, which is of great importance considering the high annotation cost for cluttered 3D indoor scenes. We resort to the robust and principled framework of selfteaching, which has triggered notable progress for semisupervised learning recently. While this paradigm is natural for image-level or pixel-level prediction, adapting it to the detection problem is challenged by the issue of proposal matching. Prior methods are based upon two-stage pipelines, matching heuristically selected proposals generated in the first stage and resulting in spatially sparse training signals. In contrast, we propose the first semisupervised 3D detection algorithm that works in the singlestage manner and allows spatially dense training signals. A fundamental issue of this new design is the quantization error caused by point-to-voxel discretization, which inevitably leads to misalignment between two transformed views in the voxel domain. To this end, we derive and implement closed-form rules that compensate this misalignment onthe-fly. Our results are significant, e.g., promoting ScanNet [email protected] from 35.2% to 48.5% using 20% annotation. Codes and data will be publicly available.
For installing the environment, we follow FCAF3D, except that we use pytorch==1.10.2+cu113.
We recommend that you install the environment manually by reading the instructions provided in the Dockerfile. You can refer to installation guide of FCAF3D for more details.
Most of the DQS3D-related code locates in the following files:
And DQS3D-related running configs in configs/fcaf3d/*.py.
We follow the mmdetection3d data preparation protocol.
You should create folders with prefix data/scannet and data/sunrgbd for ScanNet and SUN RGB-D datasets, respectively.
Please follow the instructions here for ScanNet dataset and here for SUN RGB-D dataset for downloading and organizing the data.
Training
To start training, run dist_train with DQS3D configs:
bash tools/dist_train.sh <Config Path> <GPU Count>Testing
Test pre-trained model using dist_test with DQS3D configs:
bash tools/dist_test.sh <Config Path> <Checkpoint Path> <GPU Count> --eval mAPWe provide you with the bold-styled models in the following table:
ScanNet
| [email protected] (5%) | [email protected] (5%) | [email protected] (10%) | [email protected] (10%) | [email protected] (20%) | [email protected] (20%) | [email protected] (100%) | [email protected] (100%) | |
|---|---|---|---|---|---|---|---|---|
| VoteNet | 27.9 | 10.8 | 36.9 | 18.2 | 46.9 | 27.5 | 57.8 | 36.0 |
| FCAF3D | 43.8 | 29.3 | 51.1 | 35.7 | 58.2 | 42.1 | 69.5 | 55.1 |
| SESS | 32.0 | 14.4 | 39.5 | 19.8 | 49.6 | 29.0 | 61.3 | 39.0 |
| 3DIoUMatch | 40.0 | 22.5 | 47.2 | 28.3 | 52.8 | 35.2 | 62.9 | 42.1 |
| DQS3D (Ours) | 49.2 | 35.0 | 57.1 | 41.8 | 64.3 | 48.5 | 71.9 | 56.3 |
| Improv. | +9.2 | +12.5 | +9.9 | +13.5 | +11.5 | +13.3 | +2.4 | +1.2 |
SUN RGB-D
| [email protected] (5%) | [email protected] (5%) | [email protected] (10%) | [email protected] (10%) | [email protected] (20%) | [email protected] (20%) | [email protected] (100%) | [email protected] (100%) | |
|---|---|---|---|---|---|---|---|---|
| VoteNet | 29.9 | 10.5 | 38.9 | 17.2 | 45.7 | 22.5 | 58.0 | 33.4 |
| FCAF3D | 49.5 | 31.7 | 50.7 | 33.4 | 54.3 | 36.5 | 63.6 | 47.5 |
| SESS | 34.2 | 13.1 | 42.1 | 20.9 | 47.1 | 24.5 | 60.5 | 38.1 |
| 3DIoUMatch | 39.0 | 21.1 | 45.5 | 28.8 | 49.7 | 30.9 | 61.5 | 41.3 |
| DQS3D (Ours) | 53.2 | 35.6 | 55.7 | 38.2 | 58.0 | 42.3 | 64.1 | 48.2 |
| Improv. | +14.2 | +14.5 | +10.2 | +9.4 | +8.3 | +11.4 | +0.5 | +0.7 |
You can download these models at [ Google Drive | Tsinghua Cloud Storage ].
If you find this work useful for your research, please cite our paper:
@article{gao2023dqs3d,
title={DQS3D: Densely-matched Quantization-aware Semi-supervised 3D Detection},
author={Gao, Huan-ang and Tian, Beiwen and Li, Pengfei and Zhao, Hao and Zhou, Guyue},
journal={arXiv preprint arXiv:2304.13031},
year={2023}
}We build our codebase on mmdetection3d and FCAF3D. We also give credits to Mean Teacher, SESS and 3DIoUMatch for providing comparison methods.