RSL-RL

A fast and simple implementation of learning algorithms for robotics. For an overview of the library please have a look at https://arxiv.org/pdf/2509.10771.

Environment repositories using the framework:

• Isaac Lab (built on top of NVIDIA Isaac Sim): https://github.com/isaac-sim/IsaacLab
• Legged Gym (built on top of NVIDIA Isaac Gym): https://leggedrobotics.github.io/legged_gym/
• MuJoCo Playground (built on top of MuJoCo MJX and Warp): https://github.com/google-deepmind/mujoco_playground/

The library currently supports PPO and Student-Teacher Distillation with additional features from our research. These include:

• Random Network Distillation (RND) - Encourages exploration by adding a curiosity driven intrinsic reward.
• Symmetry-based Augmentation - Makes the learned behaviors more symmetrical.

We welcome contributions from the community. Please check our contribution guidelines for more information.

Maintainer: Mayank Mittal and Clemens Schwarke
Affiliation: Robotic Systems Lab, ETH Zurich & NVIDIA
Contact: [email protected]

Setup

The package can be installed via PyPI with:

pip install rsl-rl-lib

or by cloning this repository and installing it with:

git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .

The package supports the following logging frameworks which can be configured through logger:

• Tensorboard: https://www.tensorflow.org/tensorboard/
• Weights & Biases: https://wandb.ai/site
• Neptune: https://docs.neptune.ai/

For a demo configuration of PPO, please check the example_config.yaml file.

Contribution Guidelines

For documentation, we adopt the Google Style Guide for docstrings. Please make sure that your code is well-documented and follows the guidelines.

We use the following tools for maintaining code quality:

• pre-commit: Runs a list of formatters and linters over the codebase.
• ruff: An extremely fast Python linter and code formatter, written in Rust.

Please check here for instructions to set these up. To run over the entire repository, please execute the following command in the terminal:

# for installation (only once)
pre-commit install
# for running
pre-commit run --all-files

Citing

If you use this library for your research, please cite the following work:

@article{schwarke2025rslrl,
  title={RSL-RL: A Learning Library for Robotics Research},
  author={Schwarke, Clemens and Mittal, Mayank and Rudin, Nikita and Hoeller, David and Hutter, Marco},
  journal={arXiv preprint arXiv:2509.10771},
  year={2025}
}

If you use the library with curiosity-driven exploration (random network distillation), please cite:

@InProceedings{schwarke2023curiosity,
  title = 	 {Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks},
  author =       {Schwarke, Clemens and Klemm, Victor and Boon, Matthijs van der and Bjelonic, Marko and Hutter, Marco},
  booktitle = 	 {Proceedings of The 7th Conference on Robot Learning},
  pages = 	 {2594--2610},
  year = 	 {2023},
  volume = 	 {229},
  series = 	 {Proceedings of Machine Learning Research},
  publisher =    {PMLR},
  url = 	 {https://proceedings.mlr.press/v229/schwarke23a.html},
}

If you use the library with symmetry augmentation, please cite:

@InProceedings{mittal2024symmetry,
  author={Mittal, Mayank and Rudin, Nikita and Klemm, Victor and Allshire, Arthur and Hutter, Marco},
  booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
  title={Symmetry Considerations for Learning Task Symmetric Robot Policies},
  year={2024},
  pages={7433-7439},
  doi={10.1109/ICRA57147.2024.10611493}
}