Video to Pose3D

Predict 3d human pose from video

All video is Here : https://youtu.be/7jsDbM4apKk

Prerequisite

1. Environment
   • Linux system
   • Python > 3.6 distribution
2. Dependencies
   • Packages
     • Pytorch > 1.0.0
     • torchsample
     • ffmpeg
     • tqdm
     • pillow
     • scipy
     • pandas
     • h5py
     • visdom
     • nibabel
     • opencv-python (install with pip)
     • matplotlib
   • 2D Joint detectors
     • Alphapose (Recommended)
       • Download duc_se.pth from (Google Drive | Baidu pan), place to ./joints_detectors/Alphapose/models/sppe
       • Download yolov3-spp.weights from (Google Drive | Baidu pan), place to ./joints_detectors/Alphapose/models/yolo
     • HR-Net (Bad 3d joints performance in my testing environment)
       • Download pose_hrnet* from Google Drive | Baidu pan), place to ./joints_detectors/hrnet/models/pytorch/pose_coco/
       • Download yolov3.weights from here, place to ./joints_detectors/hrnet/lib/detector/yolo
     • OpenPose (Not tested, PR to README.md is highly appreciated )
   • 3D Joint detectors
     • Download pretrained_h36m_detectron_coco.bin from here, place it into ./checkpoint folder
   • 2D Pose trackers (Optional)
     • PoseFlow (Recommended) No extra dependences
     • LightTrack (Bad 2d tracking performance in my testing environment)
       • See original README, and perform same get started step on ./pose_trackers/lighttrack

Usage

0. place your video into ./outputs folder. (I've prepared a test video).

Single person video

1. change the video_path in the ./videopose.py
2. Run it! You will find the rendered output video in the ./outputs folder.

Multiple person video (Not implemented yet)

1. For developing, check ./videopose_multi_person

   video = 'kobe.mp4'
   
   handle_video(f'outputs/{video}') 
   # Run AlphaPose, save the result into ./outputs/alpha_pose_kobe
   
   track(video)					 
   # Taking the result from above as the input of PoseTrack, output poseflow-results.json # into the same directory of above. 
   # The visualization result is save in ./outputs/alpha_pose_kobe/poseflow-vis
   
   # TODO: Need more action:
   #  1. "Improve the accuracy of tracking algorithm" or "Doing specific post processing 
   #     after getting the track result".
   #  2. Choosing person(remove the other 2d points for each frame)

Google Colab Usage

0. Open the .ipynb file in the repo with colab.

1. place your video into ./outputs folder.

Single person video

2. change the video_path in the ./videopose.py
3. Run it! You will find the rendered output video in the ./outputs folder.

Multiple person video

1. For developing, check ./videopose_multi_person

   video = 'kobe.mp4'
   
   handle_video(f'outputs/{video}') 
   # Run AlphaPose, save the result into ./outputs/alpha_pose_kobe
   
   track(video)					 
   # Taking the result from above as the input of PoseTrack, output poseflow-results.json # into the same directory of above. 
   # The visualization result is save in ./outputs/alpha_pose_kobe/poseflow-vis
   
   # TODO: Need more action:
   #  1. "Improve the accuracy of tracking algorithm" or "Doing specific post processing 
   #     after getting the track result".
   #  2. Choosing person(remove the other 2d points for each frame)
   

Acknowledgement

The 2D pose to 3D pose and visualization part is from VideoPose3D.

Some of the "In the wild" script is adapted from the other fork.

The project structure and ./videopose.py running script is adapted from this repo.

I got from its great integration and expression this repo.