Equivariant Reinforcement Learning under Partial Observability

Contents

Website and Camerea-Ready Paper

Setup

Domains

Train

License, Acknowledgments

---

Setup

1. Install anaconda
2. Create and activate environment

conda create --env equi_pomdps python=3.8.16
conda activate equi_pomdps

3. Clone this repository and install required packages

git clone https://github.com/hai-h-nguyen/equi-rl-for-pomdps.git
pip install -r requirements.txt

4. Install Pytorch (I used 1.12.0 for cuda 10.2 but other versions should work)

conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=10.2 -c pytorch

5. Install submodules

cd escnn
pip install -r requirements.txt
pip install -e .
cd ..
cd pomdp_robot_domains
pip install -r requirements.txt
pip install -e .
cd ..
cd pomdp-domains
pip install -e .
cd ..

---

Train

Domains: Block-Picking, Block-Pulling, Block-Pushing, Drawer-Opening

Before Training

export PYTHONPATH=${PWD}:$PYTHONPATH

Training (RSAC, Equi-RSAC, RAD-Crop-RSAC, DrQ-Shift-RSAC)

python3 policies/main.py --cfg configs/block_pulling/rnn.yml --algo sac --seed 0 --cuda 0 --num_expert_episodes 80
python3 policies/main.py --cfg configs/block_pulling/rnn-equi-all.yml --algo sac --seed 0 --cuda 0 --num_expert_episodes 80
python3 policies/main.py --cfg configs/block_pulling/rnn.yml --algo sac --seed 0 --cuda 0 --env BlockPulling-Symm-RAD-v0 --num_expert_episodes 80 --buffer_type seq_rad_rot
python3 policies/main.py --cfg configs/block_pulling/rnn.yml --algo sac_drq --seed 3 --cuda 0 --num_rotations 4 --num_expert_episodes 80

Simulate a Trained Policy

python3 policies/main.py --cfg configs/block_pulling/rnn-equi-all.yml --algo sac --seed 0 --cuda 0 --time_limit 1000 --num_expert_episodes 80 --replay --policy_dir saved_policies/agent_block_pulling.pt

---

License

This code is released under the MIT License.

---

Acknowledgments

This codebase evolved from the pomdp-baselines.