Name		Name	Last commit message	Last commit date
parent directory ..
exe_v2h		exe_v2h
images		images
src		src
README.md		README.md

README.md

Yoga Pose Estimation

Application: Overview

The Yoga pose estimation application automatically detects and analyze human body poses while doing yoga. It can be used for a wide variety of applications such as:

Virtual yoga sessions: Real-time feedback on poses and alignment during virtual sessions.
Fitness app: Tracking yoga practice, correcting postures, and guiding users through routines.
Rehabilitation: Monitoring exercises in physical therapy to ensure correct form.

Key Features

Here are some of the key features of the Fish Detection Application:

Full body joint detection: The application utilizes HR-Net model to analyze an image or video feed to locate key body parts and estimate their positions. It identifies the key joints and their connections to form a "skeleton" of the body.
Pose classification: The detected pose is compared against a set of known yoga poses. The classification model can then classify which yoga pose is being performed.

It has following modes of running.

Mode	RZ/V2H
MIPI Camera	`-`
USB Camera	`Supported`
Image	`Supported`
Video	`Supported`

Supported Product

RZ/V2H Evaluation Board Kit (RZ/V2H EVK)

Demo

Following is the demo for RZ/V2H EVK.

Application: Requirements

Hardware Requirements

For	Equipment	Details
RZ/V2H	RZ/V2H EVK	Evaluation Board Kit for RZ/V2H.
	AC Adapter	USB Power Delivery adapter for the board power supply. 100W is required.
	HDMI Cable	Used to connect the HDMI Monitor and the board. RZ/V2H EVK has HDMI port.
	USB Camera	Used as a camera input source.
Common	USB Cable Type-C	Connect AC adapter and the board.
	HDMI Monitor	Used to display the graphics of the board.
	microSD card	Used as the filesystem. Must have over 4GB capacity of blank space. Operating Environment: Transcend UHS-I microSD 300S 16GB
	Linux PC	Used to build application and setup microSD card. Operating Environment: Ubuntu 20.04
	SD card reader	Used for setting up microSD card.
	USB Hub	Used to connect USB Keyboard and USB Mouse to the board.
	USB Keyboard	Used to type strings on the terminal of board.
	USB Mouse	Used to operate the mouse on the screen of board.

Note: All external devices will be attached to the board and does not require any driver installation (Plug n Play Type)

Connect the hardware as shown below.

RZ/V2H EVK

Note 1: When using the keyboard connected to RZ/V Evaluation Board, the keyboard layout and language are fixed to English.
Note 2: For RZ/V2H EVK, there are USB 2.0 and USB 3.0 ports.
USB camera needs to be connected to appropriate port based on its requirement.

Application: Build Stage

Note: User can skip to the next stage (deploy) if they do not want to build the application.
All pre-built binaries are provided.

Prerequisites

This section expects the user to have completed Step 5 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

AI SDK setup is done.
Following docker container is running on the host machine.

Board Docker container

RZ/V2H EVK rzv2h_ai_sdk_container

Note: Docker environment is required for building the sample application.

Board	Docker container
RZ/V2H EVK	`rzv2h_ai_sdk_container`

Application File Generation

On your host machine, copy the repository from the GitHub to the desired location.
1. It is recommended to copy/clone the repository on the data folder, which is mounted on the Docker container.
```
cd <path_to_data_folder_on_host>/data
git clone https://github.com/renesas-rz/rzv_ai_sdk.git
```
Note: This command will download the whole repository, which include all other applications.
If you have already downloaded the repository of the same version, you may not need to run this command.
Run (or start) the docker container and open the bash terminal on the container.
E.g.,use the rzv2h_ai_sdk_container as the name of container created from rzv2h_ai_sdk_image docker image.

Note that all the build steps/commands listed below are executed on the docker container bash terminal.
Set your clone directory to the environment variable.
```
export PROJECT_PATH=/drp-ai_tvm/data/rzv_ai_sdk
```

Go to the application source code directory.

cd ${PROJECT_PATH}/Q12_yoga_pose_estimation/src

Create and move to the build directory.
```
mkdir -p build && cd build
```

Build the application by following the commands below.

cmake -DCMAKE_TOOLCHAIN_FILE=./toolchain/runtime.cmake ..
make -j$(nproc)

The following application file would be generated in the ${PROJECT_PATH}/Q12_yoga_pose_estimation/src/build directory
- pose_estimator

Application: Deploy Stage

Prerequisites

This section expects the user to have completed Step 7-1 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

microSD card setup is done.

File Configuration

For the ease of deployment all the deployable files and folders are provided in following folders.

Board	`EXE_DIR`
RZ/V2H EVK	exe_v2h

Each folder contains following items.

File	Details
keypoint_detection_model	keypoint Model object files for deployment.
pose_classifier_model	classifier Model object files for deployment.
labels.txt	Label list for Pose classification.
pose_estimator	application file.
sample_images	Sample image files.
yoga_sample.mp4	Sample video file.

Instruction

Copy the files to the /home/root/tvm directory of the rootfs (SD Card) for the board.

File Details

All files in EXE_DIR directory Including deploy.so file.

pose_estimator application file Generated the file according to Application File Generation
Check if libtvm_runtime.so exists under /usr/lib64 directory of the rootfs (SD card) on the board.

File	Details
All files in `EXE_DIR` directory	Including `deploy.so` file.
`pose_estimator` application file	Generated the file according to Application File Generation

Folder structure in the rootfs (SD Card) would look like:

|-- usr
|   `-- lib64
|       `-- libtvm_runtime.so
`-- home
    `-- root
        `-- tvm
            |-- keypoint_detection_model
            |   |-- deploy.json
            |   |-- deploy.params
            |   |-- deploy.so
            |
            |-- pose_classifier_model
            |   |-- deploy.json
            |   |-- deploy.params
            |   |-- deploy.so
            |
            |-- sample_images
            |-- yoga_sample.mp4
            |-- labels.txt
            `-- pose_estimator

Note: The directory name could be anything instead of tvm. If you copy the whole EXE_DIR folder on the board, you are not required to rename it tvm.

Application: Run Stage

Prerequisites

This section expects the user to have completed Step 7-3 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

The board setup is done.
The board is booted with microSD card, which contains the application file.

Instruction

On Board terminal, go to the tvm directory of the rootfs.
```
cd /home/root/tvm
```

Run the application.

For USB Camera Mode

./pose_estimator USB

For IMAGE Mode

./pose_estimator IMAGE "path to the input image"

For VIDEO Mode

./pose_estimator VIDEO "path to the input video"

Following window shows up on HDMI screen.

RZ/V2H EVK

On application window, following information is displayed.
- Camera capture
- Pose estimation result (Bounding boxes, Pose name and score.)
- Processing time
  - Total AI time: Total time taken for the end to end inference
  - Preprocess: Total time taken for pre-processing the input for both key-point detection and pose classifier models.
  - Inference: Processing time taken for AI inference for both key-point detection and pose classifier models.
  - Postprocess: Total time taken for post-processing the model outputs for both key-point detection and pose classifier models.
To terminate the application, switch the application window to the terminal by using Super(windows key)+Tab and press ENTER key on the terminal of the board.

Application: Configuration

AI Model

RZ/V2H
- HR-Net: Github for keypoint detection
  Dataset: COCO dataset
  Input size: 1x3x256x192
  Output size: 1x17x64x48
- NeuralNet(custom): for pose classification
  Dataset: Kaggle dataset
  Input size: 1x1x1x24
  Output size: 1x1x1x5
Target classes:
- Downdog Pose
- Goddess Pose
- Plank Pose
- Tree Pose
- Warrior II Pose

AI inference time

Board	AI inference time for keypoint detection	AI inference time for pose classification
RZ/V2H EVK	Approximately 18 ms	Approximately 1 ms

Processing

Processing	Details
Pre-processing	Processed by CPU.
Inference	Processed by DRP-AI and CPU.
Post-processing	Processed by CPU.

Image buffer size

Board	Camera capture buffer size	HDMI output buffer size
RZ/V2H EVK	VGA (640x480) in YUYV format	FHD (1920x1080) in BGRA format

Reference

For RZ/V2H EVK, this application supports USB camera only with 640x480 resolution.
To use FHD, please use MIPI camera.
Please refer to following URL for how to change camera input to MIPI camera.
https://renesas-rz.github.io/rzv_ai_sdk/latest/about-applications.

License

Apache License 2.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Q12_yoga_pose_estimation

Q12_yoga_pose_estimation

README.md

Yoga Pose Estimation

Application: Overview

Key Features

Supported Product

Demo

Application: Requirements

Hardware Requirements

Application: Build Stage

Prerequisites

Application File Generation

Application: Deploy Stage

Prerequisites

File Configuration

Instruction

Application: Run Stage

Prerequisites

Instruction

Application: Configuration

AI Model

Target classes:

AI inference time

Processing

Image buffer size

Reference

License

Files

Q12_yoga_pose_estimation

Directory actions

More options

Directory actions

More options

Latest commit

History

Q12_yoga_pose_estimation

Folders and files

parent directory

README.md

Yoga Pose Estimation

Application: Overview

Key Features

Supported Product

Demo

Application: Requirements

Hardware Requirements

Application: Build Stage

Prerequisites

Application File Generation

Application: Deploy Stage

Prerequisites

File Configuration

Instruction

Application: Run Stage

Prerequisites

Instruction

Application: Configuration

AI Model

Target classes:

AI inference time

Processing

Image buffer size

Reference

License