Jetson Fit

In order to keep my kids doing exercise during the shelter-in-place order, we started playing Nintendo Ring Fit Adventure. Thanks a lot Nintendo 👍 !

It is an awesome exercise based game. However, after three months and leveling up over 200, the kids started losing interest.

We moved on to play Mario & Sonic at the Olympic Games Tokyo 2020 and Yoshi's Crafted World. Unfortunately, these games are not exercise based and only some sport events support motion control. In particular, my 3 yr old kid's finger is not developed enough to play with the key pad.

So this hack is created, for converting games into motion controllable one for small kids.

Demo

• Yoshi's Crafted World

• Mario & Sonic at the Olympic Games Tokyo 2020 - Karate

Hardware

• Nvidia Jetson NX
• Logitech C920 at 30fps
• Nintendo Switch

Design

Prerequisites

1. TensorRT Pose Estimation https://github.com/NVIDIA-AI-IOT/trt_pose

2. Joycontrol https://github.com/mart1nro/joycontrol

3. Jupyter Notebook

Setup

1. Install trt_pose from https://github.com/NVIDIA-AI-IOT/trt_pose
2. Install joycontrol from https://github.com/mart1nro/joycontrol
3. Clone this repo
4. Download this weights resnet18_baseline_att_224x224_A (81MB) and save it in the trt_pose folder
5. Install jupyter
6. Open trt_pose/live_demo_karate.ipynb with jupyter notebook and follow the notes
7. Start joycontrol tcp server with

sudo python3 joycontrol/run_controller_server.py
OR
sudo python3 joycontrol/run_controller_server.py PRO_CONTROLLER -r <your Switch's BLE MAC address>