Skip to content

Latest commit

 

History

History
135 lines (99 loc) · 6.59 KB

README.md

File metadata and controls

135 lines (99 loc) · 6.59 KB
Raw

MVPose (Single frame)

Introduction

We provide the config files for MVPose (Single frame): Fast and robust multi-person 3d pose estimation from multiple views.

Official Implementation

@inproceedings{dong2019fast,
  title={Fast and robust multi-person 3d pose estimation from multiple views},
  author={Dong, Junting and Jiang, Wen and Huang, Qixing and Bao, Hujun and Zhou, Xiaowei},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={7792--7801},
  year={2019}
}

Prepare models and datasets

  • Prepare models:
sh scripts/download_weight.sh

You could find perception models in weight file.

  • Download body model

Please refer to Body Model Preparation.

  • Prepare the datasets:

You could download Shelf, Campus or CMU-Panoptic datasets, and convert original dataset to our unified meta-data. Considering that it takes long to run a converter, we have done it for you. Please download compressed zip file for converted meta-data from here, and place meta-data under ROOT/xrmocap_data/DATASET.

The final file structure would be like:

xrmocap
├── xrmocap
├── docs
├── tools
├── configs
├── weight
|   ├── mvpose
|   |   └── resnet50_reid_camstyle-98d61e41_20220921.pth
|   ├── ...
|   └── tracktor_reid_r50_iter25245-a452f51f.pth
└── xrmocap_data
    ├── body_models
    |   ├── gmm_08.pkl
    |   ├── smpl_mean_params.npz
    |   └── smpl
    |       ├── SMPL_FEMALE.pkl
    |       ├── SMPL_MALE.pkl
    |       └── SMPL_NEUTRAL.pkl
    |
    ├── CampusSeq1
    ├── Shelf
    |   ├── Camera0
    |   ├── ...
    |   ├── Camera4
    |   ├── xrmocap_meta_testset_fasterrcnn
    |   └── xrmocap_meta_testset
    └── Panoptic
        ├── xrmocap_meta_ian5
        |   ├── hd_00_03
        |   ├── ...
        |   ├── hd_00_23
        |   ├── camera_parameters
        |   ├── keypoints3d_GT.npz
        |   └── perception_2d.npz
        ├── xrmocap_meta_pizza1
        ├── xrmocap_meta_band4
        └── xrmocap_meta_haggling1

You can download just one dataset of Shelf, Campus and CMU-Panoptic.

Results

We evaluate MVPose (Single frame) on 3 popular benchmarks, report the Percentage of Correct Parts (PCP) on Shelf/Campus/CMU-Panoptic datasets.

You can find the recommended configs in configs/mvpose/*/eval_keypoints3d.py, where __bbox_thr__ is the threshold of bbox2d, you can set a high threshold to ignore incorrect 2D perception data, and we recommen setting it to 0.8~0.9. n_cam_min is the amount of views required for triangulation, which defaults to 2.

Campus

The 2D perception data we use is generated by fasterrcnn, and you can download it from here. What's more, we set __bbox_thr__=0.9 and n_cam_min=2.

Config Actor 0 Actor 1 Actor 2 Average Download
eval_keypoints3d.py 94.17 88.19 98.38 93.58 log

Shelf

The 2D perception data we use is generated by fasterrcnn, and you can download it from here. What's more, we set __bbox_thr__=0.9 and n_cam_min=3.

Config Actor 0 Actor 1 Actor 2 Average Download
eval_keypoints3d.py 97.53 94.05 97.89 96.49 log

CMU Panoptic

The 2D perception data we use is generated by mmpose, and you can download it from here. The selection principle of the camera is to cover as much information as possible about the human body, so we selected cameras 3, 6, 12, 13 and 23.

The CMU Panoptic dataset contains four sequences that share the same config file. You can find more details in config files. For different sequences, you need to change the __meta_path__. In addition, we set __bbox_thr__=0.85 and n_cam_min=2.

  • 160906_band4
Config Actor 0 Actor 1 Actor 2 Average Download
eval_keypoints3d.py 98.07 93.36 90.86 94.10 log
  • 160906_ian5
Config Actor 0 Actor 1 Average Download
eval_keypoints3d.py 96.30 71.08 83.69 log
  • 160906_pizza1
Config Actor 0 Actor 1 Actor 2 Actor 3 Actor 4 Actor 5 Actor 6 Average Download
eval_keypoints3d.py 99.09 93.28 89.80 90.90 91.41 79.75 93.12 91.05 log
  • 160422_haggling1
Config Actor 0 Actor 1 Actor 2 Actor 3 Actor 4 Actor 5 Actor 6 Actor 7 Actor 8 Actor 9 Actor 10 Actor 11 Actor 12 Actor 13 Actor 14 Actor 15 Actor 16 Actor 17 Average Download
eval_keypoints3d.py 98.66 97.38 96.48 98.64 95.86 95.59 89.82 81.03 97.13 91.16 98.77 97.65 89.18 93.45 97.23 93.13 98.15 89.82 94.40 log