Skip to content

Project for the course "Object recognition and computer vision"

Notifications You must be signed in to change notification settings

ManifoldFR/recvis-project

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

66 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Project: Humanoid Robot Imitation of Human Motion from Instructional Videos

Project for the MVA "Object Recognition and Computer Vision" class (https://www.di.ens.fr/willow/teaching/recvis19/).

Clone the repository

git clone https://github.com/ManifoldFR/recvis-project

Dependencies

The initial HMR repository uses Python 2.7 and TensorFlow 1.3. We patched HMR to be able to use more recent versions of Python and TensorFlow 1.x (tested on TensorFlow 1.15 and Python 3.7).

We reverse-engineered motion-reconstruction to use with AlphaPose (included in the repository) and Python 3 instead of Openpose.

The patched versions of the repos are included in this repository.

DeepMimic requires a specific format for motion capture files. They can be obtained from BVH (BioVision Hierarchy Animation) files. Converting from BVH to DeepMimic requires the BvhToDeepMimic package

pip install bvhtodeepmimic

Use

Getting MoCap data from a video

Put the videos in a directory (by default data/vid) and call

python -m run_alphapose

Then run

python -m refine_hmr

The output will be in the refined directory.

Conversion of motion to DeepMimic JSON format

Use https://github.com/BartMoyaers/BvhToDeepMimic to convert the BVH files to DeepMimic-formatted files.

We repurposed MoCap conversion files from the PyBullet reimplementation of DeepMimic (credit to Erwin Coumans and Yihang Yin), inverse_kinematics.py and transformation.py. We wrote a wrapper for this code that you can modify and call as

python ik_hmr_deepmimic.py

For information, the HMR output joints are as follow (from a comment inside of the HMR source):

  1. Right ankle
  2. Right knee
  3. Right hip
  4. Left hip
  5. Left knee
  6. Left ankle
  7. Right wrist
  8. Right elbow
  9. Right shoulder
  10. Left shoulder
  11. Left elbow
  12. Left wrist
  13. Neck
  14. Head top
  15. nose
  16. left_eye
  17. right_eye
  18. left_ear
  19. right_ear

Bibliography

About

Project for the course "Object recognition and computer vision"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published