arm-teleop

---

Bi-manual robotic teleoperation system using VR hand tracking and camera streaming. Saves data to .h5 files for training. Visualizes the robot using rerun.

Setup

git clone https://github.com/kscalelabs/arm-teleop.git && cd arm-teleop
conda create -y -n arm-teleop python=3.8 && conda activate arm-teleop
pip install -r requirements.txt

Usage - Demo Mode

Start the server on the robot computer.

python play.py --robot="5dof"

Start ngrok on the robot computer.

ngrok http 8012

Open the browser app on the HMD and go to the ngrok URL.

Usage - Recording Data

Start the server on the robot computer.

🤗 K-Scale HuggingFace Datasets

Dependencies

• Vuer is used for visualization
• PyBullet is used for IK
• ngrok is used for networking
• Rerun is used for visualization
• H5Py is used for logging datasets
• HuggingFace is used for dataset & model storage

Citation

@misc{teleop-2024,
  title={Bi-Manual Remote Robotic Teleoperation},
  author={Hugo Ponte},
  year={2024},
  url={https://github.com/kscalelabs/teleop}
}