HaGRID Dynamic Gestures

HaGRID main repo

Overview

This repository contains using HaGRID dataset for dynamic gesture recognition. The dataset is available here.

Project

├── ocsort/ # source code for Observation-Centric Sorting
│   ├── kalmanfilter.py # Kalman filter
│   ├── kalmanboxtracker.py # Kalman box tracker
│   ├── association.py # Association of boxes with trackers
├── utils/ # useful utils
│   ├── action_controller.py # Action controller for dynamic gestures
│   ├── box_utils_numpy.py # Box utils for numpy
│   ├── enums.py # Enums for dynamic gestures and actions
│   ├── hand.py # Hand class for dynamic gestures recognition
│   ├── drawer.py # Debug drawer
├── onnx_models.py # ONNX models for gesture recognition
├── main_controller.py # Main controller for dynamic gestures recognition, uses ONNX models, ocsort and utils
├── run_demo.py # Demo script for dynamic gestures recognition

Installation

Clone and install required python packages:

git clone https://github.com/nagadit/hagrid-dynamic.git
# or mirror link:
cd hagrid
# Create virtual env by conda or venv
conda create -n dynamic_gestures python=3.9 -y
conda activate dynamic_gestures
# Install requirements
pip install -r requirements.txt

Demo

To run demo, you just need to run run_demo.py script.

python run_demo.py --detector <path_to_detector> --classifier <path_to_classifier> --debug

--detector (optional) Path to the hand detector model.
Default: models/hand_detector.onnx

--classifier (optional) Path to the crops classifier model.
Default: models/crops_classifier.onnx

--debug (optional) Enables debug mode to see bounding boxes and class labels.

Dynamic gestures

Next, we will show dynamic gestures in user mode and debug mode. In user mode, we show only the final result of dynamic gesture recognition. In debug mode, we show the result of each step of dynamic gesture recognition:

1. hand detection
2. hand tracking
3. gesture recognition
4. action recognition

At the moment the code supports 6 groups of dynamic gestures:

ZOOM DRAG AND DROP

FAST SWIPE UP / DOWN CLICK SWIPES LEFT / RIGHT

SWIPES UP / DOWN

License

This work is licensed under a variant of Creative Commons Attribution-ShareAlike 4.0 International License.

Please see the specific license.

Citation

You can cite the paper using the following BibTeX entry:

@misc{nuzhdin2024hagridv21mimagesstatic,
      title={HaGRIDv2: 1M Images for Static and Dynamic Hand Gesture Recognition}, 
      author={Anton Nuzhdin and Alexander Nagaev and Alexander Sautin and Alexander Kapitanov and Karina Kvanchiani},
      year={2024},
      eprint={2412.01508},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2412.01508}, 
}