Mujoco bindings and update to gym v0.26 (#18)

* add utils folder

* mujoco bindings robot env base

* move utils

* fetch mujoco bindings

* register mujoco envs

* mujoco model name properties

* latest changes

* split fetch classes

* Farama Foundation code of conduct

* split mujoco_py and mujoco envs

* fix inheritance

* add pre-commit tests

* remove init

* reset sim fetch env

* new step api

* return info mandatory

* fix pickling

* new step api

* env pytests

* fix renderer

* revert info in reset

* offset site init hand reach

* update to mujoco 2.2.2

* fix pyright

* fix pytest

* update py.Docker

* move pyright test to pre-commit

* update touch sensor env id's

* update readme

* versioneer

* remove mising imports from pyright

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,45 @@
+# Description
+
+Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
+
+Fixes # (issue)
+
+## Type of change
+
+Please delete options that are not relevant.
+
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] This change requires a documentation update
+
+### Screenshots
+Please attach before and after screenshots of the change if applicable.
+
+<!--
+Example:
+
+| Before | After |
+| ------ | ----- |
+| _gif/png before_ | _gif/png after_ |
+
+
+To upload images to a PR -- simply drag and drop an image while in edit mode and it should upload the image directly. You can then paste that source into the above before/after sections.
+-->
+
+# Checklist:
+
+- [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with `pre-commit run --all-files` (see `CONTRIBUTING.md` instructions to set it up)
+- [ ] I have commented my code, particularly in hard-to-understand areas
+- [ ] I have made corresponding changes to the documentation
+- [ ] My changes generate no new warnings
+- [ ] I have added tests that prove my fix is effective or that my feature works
+- [ ] New and existing unit tests pass locally with my changes
+
+<!--
+As you go through the checklist above, you can mark something as done by putting an x character in it
+
+For example,
+- [x] I have done this task
+- [ ] I have not done this task
+-->

.github/workflows/build.yml

@@ -12,6 +12,6 @@ jobs:
       - run: |
            docker build -f py.Dockerfile \
              --build-arg PYTHON_VERSION=${{ matrix.python-version }} \
-             --tag gym-docker .
+             --tag gym-robotics-docker .
       - name: Run tests
-        run: docker run gym-docker pytest --forked --import-mode=append
+        run: docker run gym-robotics-docker pytest

.pre-commit-config.yaml

@@ -1,15 +1,7 @@
+# exclude: '^gym_robotics/'
 repos:
-  - repo: https://github.com/PyCQA/bandit/
-    rev: 1.7.0
-    hooks:
-      - id: bandit
-        args:
-          - --recursive
-          - --skip
-          - B101,B108,B301,B403,B404,B603
-          - .
   - repo: https://github.com/python/black
-    rev: 21.7b0
+    rev: 22.3.0
     hooks:
       - id: black
   - repo: https://github.com/codespell-project/codespell
@@ -19,12 +11,37 @@ repos:
         args:
           - --ignore-words-list=nd,reacher,thist,ths
   - repo: https://gitlab.com/pycqa/flake8
-    rev: 3.9.2
+    rev: 4.0.1
     hooks:
       - id: flake8
         args:
-          - --ignore=E203,E402,E712,E722,E731,E741,F401,F403,F405,F524,F841,W503
+          - '--per-file-ignores=*/__init__.py:F401'
+          - --ignore=E203,W503
           - --max-complexity=30
           - --max-line-length=456
           - --show-source
           - --statistics
+  - repo: https://github.com/PyCQA/isort
+    rev: 5.10.1
+    hooks:
+      - id: isort
+        args: ["--profile", "black"]
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v2.32.0
+    hooks:
+      - id: pyupgrade
+        args: ["--py37-plus"]
+  - repo: local
+    hooks:
+      - id: pyright
+        exclude: ^gym_robotics/_version.py
+        name: pyright
+        entry: pyright
+        language: node
+        pass_filenames: false
+        types: [python]
+        additional_dependencies: ["pyright"]
+        args:
+          - --project=pyproject.toml
+
+

CODE_OF_CONDUCT.rst

@@ -1,12 +1,12 @@
-OpenAI Gym is dedicated to providing a harassment-free experience for
+Farama Foundation is dedicated to providing a harassment-free experience for
 everyone, regardless of gender, gender identity and expression, sexual
 orientation, disability, physical appearance, body size, age, race, or
 religion. We do not tolerate harassment of participants in any form.
 
-This code of conduct applies to all OpenAI Gym spaces (including Gist
+This code of conduct applies to all Farama Foundation spaces (including Gist
 comments) both online and off. Anyone who violates this code of
 conduct may be sanctioned or expelled from these spaces at the
-discretion of the OpenAI team.
+discretion of the Farama Foundation team.
 
 We may add additional rules over time, which will be made clearly
 available to participants. Participants are responsible for knowing

README.md

@@ -1,12 +1,21 @@
+[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/) 
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 <p align="center">
     <img src="readme.png" width="500px"/>
 </p>
 
+# Gym-Robotics
 Details and documentation on these robotics environments are available in OpenAI's [blog post](https://blog.openai.com/ingredients-for-robotics-research/) and the accompanying [technical report](https://arxiv.org/abs/1802.09464).
 
+Requirements:
+- Python 3.7 to 3.10
+- Gym v0.26
+- NumPy 1.18+
+- Mujoco 2.2.2
+
 If you use these environments, please cite the following paper:
 
-```
+```bibtex
 @misc{1802.09464,
   Author = {Matthias Plappert and Marcin Andrychowicz and Alex Ray and Bob McGrew and Bowen Baker and Glenn Powell and Jonas Schneider and Josh Tobin and Maciek Chociej and Peter Welinder and Vikash Kumar and Wojciech Zaremba},
   Title = {Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research},
@@ -15,42 +24,71 @@ If you use these environments, please cite the following paper:
 }
 ```
 
+## New MuJoCo Python Bindings
+
+The latest version and future versions of the MuJoCo environments will no longer depend on `mujoco-py`. Instead the new [mujoco](https://mujoco.readthedocs.io/en/latest/python.html) python bindings will be the required dependency for future gym MuJoCo environment versions. Old gym MuJoCo environment versions that depend on `mujoco-py` will still be kept but unmaintained.
+Dependencies for old MuJoCo environments can still be installed by `pip install gym_robotics[mujoco_py]`.
+
 ## Fetch environments
 <img src="https://openai.com/content/images/2018/02/fetch-reach.png" width="500">
 
-[FetchReach-v0](https://gym.openai.com/envs/FetchReach-v0/): Fetch has to move its end-effector to the desired goal position.
+[FetchReach-v3](https://gym.openai.com/envs/FetchReach-v0/): Fetch has to move its end-effector to the desired goal position.
 
 
 <img src="https://openai.com/content/images/2018/02/fetch-slide.png" width="500">
 
-[FetchSlide-v0](https://gym.openai.com/envs/FetchSlide-v0/): Fetch has to hit a puck across a long table such that it slides and comes to rest on the desired goal.
+[FetchSlide-v2](https://gym.openai.com/envs/FetchSlide-v0/): Fetch has to hit a puck across a long table such that it slides and comes to rest on the desired goal.
 
 
 <img src="https://openai.com/content/images/2018/02/fetch-push.png" width="500">
 
-[FetchPush-v0](https://gym.openai.com/envs/FetchPush-v0/): Fetch has to move a box by pushing it until it reaches a desired goal position.
+[FetchPush-v2](https://gym.openai.com/envs/FetchPush-v0/): Fetch has to move a box by pushing it until it reaches a desired goal position.
 
 
 <img src="https://openai.com/content/images/2018/02/fetch-pickandplace.png" width="500">
 
-[FetchPickAndPlace-v0](https://gym.openai.com/envs/FetchPickAndPlace-v0/): Fetch has to pick up a box from a table using its gripper and move it to a desired goal above the table.
+[FetchPickAndPlace-v2](https://gym.openai.com/envs/FetchPickAndPlace-v0/): Fetch has to pick up a box from a table using its gripper and move it to a desired goal above the table.
 
 ## Shadow Dexterous Hand environments
 <img src="https://openai.com/content/images/2018/02/hand-reach.png" width="500">
 
-[HandReach-v0](https://gym.openai.com/envs/HandReach-v0/): ShadowHand has to reach with its thumb and a selected finger until they meet at a desired goal position above the palm.
+[HandReach-v1](https://gym.openai.com/envs/HandReach-v0/): ShadowHand has to reach with its thumb and a selected finger until they meet at a desired goal position above the palm.
 
 
 <img src="https://openai.com/content/images/2018/02/hand-block.png" width="500">
 
-[HandManipulateBlock-v0](https://gym.openai.com/envs/HandManipulateBlock-v0/): ShadowHand has to manipulate a block until it achieves a desired goal position and rotation.
+[HandManipulateBlock-v1](https://gym.openai.com/envs/HandManipulateBlock-v0/): ShadowHand has to manipulate a block until it achieves a desired goal position and rotation.
 
 
 <img src="https://openai.com/content/images/2018/02/hand-egg.png" width="500">
 
-[HandManipulateEgg-v0](https://gym.openai.com/envs/HandManipulateEgg-v0/): ShadowHand has to manipulate an egg until it achieves a desired goal position and rotation.
+[HandManipulateEgg-v1](https://gym.openai.com/envs/HandManipulateEgg-v0/): ShadowHand has to manipulate an egg until it achieves a desired goal position and rotation.
 
 
 <img src="https://openai.com/content/images/2018/02/hand-pen.png" width="500">
 
-[HandManipulatePen-v0](https://gym.openai.com/envs/HandManipulatePen-v0/): ShadowHand has to manipulate a pen until it achieves a desired goal position and rotation.
+[HandManipulatePen-v1](https://gym.openai.com/envs/HandManipulatePen-v0/): ShadowHand has to manipulate a pen until it achieves a desired goal position and rotation.
+
+# Hand environments with Touch Sensors
+
+Touch sensor observations are also available in all Hand environments, with exception of `HandReach`. These environments add to the palm of the hand and the phalanges of the fingers two types of touch sensors depending on the environment. These touch sensors are:
+- **Boolean Touch Sensor**: the observations of each touch sensor can return a value of `0` if no contact is detected with and object, and `1` otherwise.
+- **Continuous Touch Sensor**: the value returned by each touch sensor is a continuous value that represents the external force made by an object over the sensor.
+
+These environments are instanceated by adding the following strings to the Hand environment id's: `_BooleanTouchSensor` or `_ContinuousTouchSensor`. For example, to add boolean touch sensors to `HandManipulateBlock-v1`, make the environment in the following way:
+```
+env = gym.make('HandManipulateBlock_BooleanTouchSensor-v1')
+```
+
+If using these environments please also cite the following paper:
+
+```bibtex
+@article{melnik2021using,
+  title={Using tactile sensing to improve the sample efficiency and performance of deep deterministic policy gradients for simulated in-hand manipulation tasks},
+  author={Melnik, Andrew and Lach, Luca and Plappert, Matthias and Korthals, Timo and Haschke, Robert and Ritter, Helge},
+  journal={Frontiers in Robotics and AI},
+  pages={57},
+  year={2021},
+  publisher={Frontiers}
+}
+```