Skip to content

Latest commit

 

History

History
84 lines (60 loc) · 2.47 KB

readme.md

File metadata and controls

84 lines (60 loc) · 2.47 KB
Raw

pym3t

Python binding for M3T: A Multi-body Multi-modality Multi-camera 3D Tracker.

Modification

1. Limitation

The original M3T framework considers multi-body multi-modality multi-camera tracking. Currently, pym3t only focuses on multi-rigid-object multi-modality tracking with one camera.

2. Removed Modules

  • Special Camera The original modules for Azure Kinect camera and RealSense camera are removed. You will need to provide RGB image (and depth image) manually in OpenCV::Mat format.
  • Publisher and Subscriber The two modules are not important to the core tracking pipeline.
  • Detector The original StaticDetector and ManualDetector are removed, as there are ways to do it better. The Refiner is not used as well (reserved).
  • Image Viewer The original ColorImageViewer and DepthImageViewer are removed. As the input images are provided manually, the visualization of them can be easily implemented through opencv-python.

3. Detached Normal Viewer

The normal viewers are detached from the original tracker. You can manually create one if needed.

Installation

Make sure that you have installed the following packages:

  • GLEW
  • glfw3
  • Eigen3
  • OpenCV 4 with Contrib Modules
  • OpenMP (may have already installed on your system)

Your compilation environment should support C++ 17.

Dependency Instruction

  • [method 1] Through system path

    # build tools
$ sudo apt install build-essential cmake

# libraries
$ sudo apt install libglew-dev libglfw3-dev libeigen3-dev

# BUILD opencv with extra contrib modules from source code:
  - https://github.com/opencv/opencv
  - https://github.com/opencv/opencv_contrib

# python modules for example/demo.py
$ pip install numpy==1.26.0
$ pip install opencv-python

  • [method 2] Under conda env

    $ conda create -n ${your_env_name} python=3.9
$ conda activate ${your_env_name}
$ conda install cmake eigen glfw glew libopencv

Package Installation

cd ${repo_root}
pip install .

Demo

The demo data is from SMu1 sequence of HO3D_v3

cd ${repo_root}/example
python demo.py

Interface

Follow source/pym3t/pym3t.cpp.

Note

This algorithm is for object tracking only, without global pose estimation. In consequence, an initial pose (4 by 4 matrix under opencv coordinate) must be provided before you start tracking.