125/96 Episodes with malfunction for benchmarking and regression tests. 8 Policy abstraction. #131
Conversation
…bservations and infos from reset() to env_generator interface.
chenkins
changed the title
chenkins
changed the title
…ut loading the reset env directly..
chenkins
changed the title
|
|
||
|
|
||
| class Policy: | ||
| def act(self, handle: int, observation: Any, **kwargs) -> RailEnvActions: |
There was a problem hiding this comment.
@manuschn Not sure we want/need the handle as well? The policy's acting should be based on the observation only and not depend on the handle?
There was a problem hiding this comment.
Do we break all python version below 3.9 !!!!
TypeError: Type subscription requires python >= 3.9
(flatland-rl-test) u216993@K57156:~/flatland/test/flatland_solver_policy$ python example/flatland_dynamics/example_flatland_dynamics.py
Traceback (most recent call last):
File "example/flatland_dynamics/example_flatland_dynamics.py", line 4, in
from environment.flatland_railway_extension.flatland_dynamics import FlatlandDynamicsEnvironment
File "/home/u216993/flatland/test/flatland_solver_policy/environment/flatland_railway_extension/flatland_dynamics.py", line 3, in
from flatland_railway_extension.FlatlandEnvironmentHelper import FlatlandEnvironmentHelper
File "/home/u216993/flatland/test/flatland_railway_extension/flatland_railway_extension/FlatlandEnvironmentHelper.py", line 6, in
from flatland.envs.malfunction_generators import MalfunctionParameters, ParamMalfunctionGen
File "/home/u216993/flatland/test/flatland-rl/flatland/envs/malfunction_generators.py", line 8, in
from flatland.envs import persistence
File "/home/u216993/flatland/test/flatland-rl/flatland/envs/persistence.py", line 10, in
from flatland.envs import rail_env
File "/home/u216993/flatland/test/flatland-rl/flatland/envs/rail_env.py", line 28, in
from flatland.utils import seeding
File "/home/u216993/flatland/test/flatland-rl/flatland/utils/seeding.py", line 97, in
HashableRandomState = Tuple[str, np.ndarray[np.uint], int, int, float]
TypeError: Type subscription requires python >= 3.9
…to 125-episodes-refactoring
…to 125-episodes-refactoring
| # it's not sufficient to store random_seed, as seeding from random_seed is done | ||
| # at start of reset (before rail/line/timetable (re-)generation, | ||
| # hence np_random depends on rail/line/timetable generation | ||
| # TODO would it be better to have env_generation without reset? Conceptually, the env should be initialised with a state incl. the seed |
There was a problem hiding this comment.
Discuss/resolve TODO.
Current approach: reset calls rail/line/timetable generators -> random state at start depends on these generators.
Desireable approach (anticipated by env_generator): the env starts is already reset in initial state, therefore need to persist np_random after reset is called.
Is the current implementation self-consistent? Add test
- save env without reset
- save env after reset
load both and reset w/ and w/o reset -> the result should be the same in all 4 cases. Is this enough?
| ============ | ||
|
|
||
|
|
||
| TODO move to `flatland-book`? |
There was a problem hiding this comment.
@manuschn are these diagrams helpful ?
| ``` | ||
|
|
||
| Flow Env Step |
…to 125-episodes-refactoring
Changes
Related issues
Closes #125
Closes to #8
Closes #96
Checklist
[Unreleased]tag inCHANGELOG.md.pyproject.tomlfile.Requirement files have been updated by running
tox -e requirements.required to run successfully.
CONTRIBUTING.mdare followed.