![oprl_logo](https://private-user-images.githubusercontent.com/23639048/307557134-c7ea0fee-3472-4d9c-86f3-9ab01f02222d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwNTYyNzQsIm5iZiI6MTcyNDA1NTk3NCwicGF0aCI6Ii8yMzYzOTA0OC8zMDc1NTcxMzQtYzdlYTBmZWUtMzQ3Mi00ZDljLTg2ZjMtOWFiMDFmMDIyMjJkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDA4MjYxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTMwODg1NGY4Nzc4MDk2MGE0NmFlZDk4MDY0NWM4MGEwYTljZDExZjU3NTg5OTVjODg4MDdkMTU3Y2QyNTY0MDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ve-e1pPcT_5UBJxZV--SwJZmDN0tiXsGpcRv6ZGEhv0)
A Modular Library for Off-Policy Reinforcement Learning with a focus on SafeRL and distributed computing. Benchmarking resutls are available at associated homepage: Homepage
The project is under an active renovation, for the old code with D4PG algorithm working with multiprocessing queues and mujoco_py
please refer to the branch d4pg_legacy
.
- Switching to
mujoco 3.1.1
- Replacing multiprocessing queues with RabbitMQ for distributed RL
- Baselines with DDPG, TQC for
dm_control
for 1M step - Tests
- Support for SafetyGymnasium
- Style and readability improvements
- Baselines with Distributed algorithms for
dm_control
- D4PG logic on top of TQC
pip install -r requirements.txt
cd src && pip install -e .
For working with SafetyGymnasium install it manually
git clone https://github.com/PKU-Alignment/safety-gymnasium
cd safety-gymnasium && pip install -e .
To run DDPG in a single process
python src/oprl/configs/ddpg.py --env walker-walk
To run distributed DDPG
Run RabbitMQ
docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.12-management
Run training
python src/oprl/configs/d3pg.py --env walker-walk
cd src && pip install -e .
cd .. && pip install -r tests/functional/requirements.txt
python -m pytest tests
Results for single process DDPG and TQC:
- DDPG and TD3 code is based on the official TD3 implementation: sfujim/TD3
- TQC code is based on the official TQC implementation: SamsungLabs/tqc
- SafetyGymnasium: PKU-Alignment/safety-gymnasium