74 Policy evaluation and training cli (rllib) #85

chenkins · 2024-11-01T08:36:25Z

Changes

extract ml dependencies (torch, gym, ray, etc.) to optional dependencies in pyproject.toml, new requirements-ml.txt and slimmer requirements[-dev].txt, keep core gym free.
accordingly, move corresponding code to new flatland.ml module
accordingly, move corresponding tests to tests.ml (tests becomes a Python module)
preparations for rllib (ray.io), gym(nasium), Pettingzoo compatibility #73 / 73 Get Pettingzoo example to work again. #102 (cleanup of flatland.contrib s)

Related issues

Closes #74.
Fixes #23.
Closes #75

Checklist

Tests are included for relevant behavior changes.
Documentation is added in the docs folder for relevant behavior changes. If you made important user-facing
changes, describe them under the [Unreleased] tag in CHANGELOG.md.
New package dependencies are declared in the pyproject.toml file.
Requirement files have been updated by running tox -e requirements.
Code works with all supported Python versions (3.8, 3.9 and 3.10). Checks run with all three version and are
required to run successfully.
Code is formatted according to PEP 8 (an IDE like PyCharm can do this for you).
Technical guidelines listed in CONTRIBUTING.md are followed.

flatland/envs/rail_env.py

flatland/ml/observations/flatten_tree_observation_for_rail_env.py

tests/test_flatland_malfunction.py

flatland/ml/observations/flatten_tree_observation_for_rail_env.py

flatland/ml/ray/ray_multi_agent_rail_env.py

tests/ml/ray/examples/test_flatland_training_with_parameter_sharing.py

flatland/env_generation/env_creator.py

flatland/ml/ray/wrappers.py

flatland/envs/rail_env.py

flatland/ml/ray/ray_multi_agent_rail_env.py

Co-authored-by: Serge Croisé <[email protected]>

flatland/ml/ray/ray_multi_agent_rail_env.py

tests/test_env_creator.py

flatland/ml/observations/flatten_tree_observation_for_rail_env.py

tests/ml/observations/test_flatten_tree_observation_for_rail_env.py

flatland/ml/ray/ray_multi_agent_rail_env.py

* Prepare changelog for 4.0.4. * Prepare changelog for 4.0.4. * Prepare changelog for 4.0.4. * Prepare changelog for 4.0.4. * Add pr #115. * Add pr #115. * Add pr #85.

chenkins changed the base branch from main to python-base-version-310 November 1, 2024 08:37

chenkins commented Nov 1, 2024

View reviewed changes

flatland/envs/rail_env.py Outdated Show resolved Hide resolved

chenkins commented Nov 1, 2024

View reviewed changes

flatland/ml/observations/flatten_tree_observation_for_rail_env.py Outdated Show resolved Hide resolved

Base automatically changed from python-base-version-310 to main November 18, 2024 14:36

chenkins changed the title ~~74 policy evaluation and training cli~~ 74 Policy evaluation and training cli (rllib) Nov 20, 2024

chenkins force-pushed the 74-policy-evaluation-and-training-cli branch from daec963 to 3f903ca Compare December 4, 2024 13:13

SergeCroise reviewed Dec 4, 2024

View reviewed changes

tests/test_flatland_malfunction.py Outdated Show resolved Hide resolved

SergeCroise reviewed Dec 4, 2024

View reviewed changes

tests/test_flatland_malfunction.py Outdated Show resolved Hide resolved

SergeCroise reviewed Dec 4, 2024

View reviewed changes

tests/test_flatland_malfunction.py Outdated Show resolved Hide resolved

SergeCroise reviewed Dec 22, 2024

View reviewed changes

flatland/ml/observations/flatten_tree_observation_for_rail_env.py Outdated Show resolved Hide resolved

chenkins mentioned this pull request Jan 10, 2025

Bump ray from 1.5.2 to 2.8.1 in /flatland/contrib #108

Closed

chenkins commented Jan 10, 2025

View reviewed changes

flatland/ml/ray/ray_multi_agent_rail_env.py Outdated Show resolved Hide resolved

tests/ml/ray/examples/test_flatland_training_with_parameter_sharing.py Outdated Show resolved Hide resolved

chenkins requested a review from manuschn January 10, 2025 14:01

chenkins marked this pull request as ready for review January 10, 2025 14:02

chenkins requested a review from a team as a code owner January 10, 2025 14:02

chenkins added this to the 4.0.4 milestone Jan 17, 2025

manuschn reviewed Jan 24, 2025

View reviewed changes

flatland/env_generation/env_creator.py Outdated Show resolved Hide resolved

flatland/ml/ray/wrappers.py Outdated Show resolved Hide resolved

aiAdrian previously approved these changes Feb 3, 2025

View reviewed changes

flatland/envs/rail_env.py Outdated Show resolved Hide resolved

flatland/ml/ray/ray_multi_agent_rail_env.py Outdated Show resolved Hide resolved

chenkins dismissed aiAdrian’s stale review via 5a295a2 February 13, 2025 15:28

chenkins force-pushed the 74-policy-evaluation-and-training-cli branch from 5a295a2 to 3b5280b Compare February 13, 2025 15:39

chenkins added 10 commits February 13, 2025 16:55

Add flatland training cli with ray.

8827ea9

Fix error from ray passive_env_checker.

ad9119d

Fix flattened gym observation builders.

4fe5848

Unit test for (gym) observation builder returned (d)types sizes/shapes.

b09505a

Extract ray examples.

a8f47d8

Update TODO.

18f9bf0

Add ray address cli param.

b9d613f

Cleanup training cli example.

b247a95

Create test stub for env_creator.

53585f7

Mark training test as slow.

6a6b807

chenkins and others added 7 commits February 13, 2025 16:55

Split flattening and normalization in FlattenTreeObsForRailEnv.

8fc7123

Update TODOs.

01f7375

Add documentation on feature groups in tree obs flattening.

d008640

Refactor tree obs normalization.

25687f4

Refactor tree obs normalization.

97af6b1

Cleanup.

ce547d2

Apply suggestions from code review

d7b9a6d

Co-authored-by: Serge Croisé <[email protected]>

chenkins force-pushed the 74-policy-evaluation-and-training-cli branch from 3b5280b to 1929d82 Compare February 13, 2025 16:04

chenkins added 2 commits February 14, 2025 12:46

Post-rebase fix imports.

e96c319

Post-rebase fix BENCHMARK_EPISODES_FOLDER in tox *-ml envs.

29fa6a6

chenkins force-pushed the 74-policy-evaluation-and-training-cli branch from 48347b6 to 29fa6a6 Compare February 14, 2025 11:47

chenkins added 5 commits February 14, 2025 13:56

Rename env_creator to env_generator.

ec0a8b9

Cleanup.

cecdcf6

Add docstring regarding agents are identified by string AgentIDs.

fa3bc45

Update requirements.

10c469f

Fix dqn training and inference.

6b70750

chenkins added a commit that referenced this pull request Feb 15, 2025

Add pr #85.

9be3084

chenkins commented Feb 17, 2025

View reviewed changes

flatland/ml/ray/ray_multi_agent_rail_env.py Outdated Show resolved Hide resolved

chenkins added 2 commits February 17, 2025 17:15

remove spurious TODO

250293b

Revert discarding self.action_space = [5]

6fe6702

chenkins requested a review from manuschn February 17, 2025 20:50

manuschn previously approved these changes Feb 17, 2025

View reviewed changes

Use wrap.action_space[0] instead of hard-coded 5.

8308d66

chenkins dismissed manuschn’s stale review via 8308d66 February 18, 2025 09:41

chenkins added 3 commits February 18, 2025 10:42

Rename FlattenTreeObsForRailEnv to FlattenedTreeObsForRailEnv.

e11a83b

Rename test_env_generator.py.

eee9bf4

Merge branch 'main' into 74-policy-evaluation-and-training-cli

22dab2d

chenkins merged commit e6871dd into main Feb 18, 2025
6 checks passed

chenkins deleted the 74-policy-evaluation-and-training-cli branch February 18, 2025 13:36

chenkins added a commit that referenced this pull request Feb 18, 2025

Release/4.0.4 (#114)

a4ca8d1

* Prepare changelog for 4.0.4. * Prepare changelog for 4.0.4. * Prepare changelog for 4.0.4. * Prepare changelog for 4.0.4. * Add pr #115. * Add pr #115. * Add pr #85.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

74 Policy evaluation and training cli (rllib) #85

74 Policy evaluation and training cli (rllib) #85

chenkins commented Nov 1, 2024 •

edited

Loading

74 Policy evaluation and training cli (rllib) #85

74 Policy evaluation and training cli (rllib) #85

Conversation

chenkins commented Nov 1, 2024 • edited Loading

Changes

Related issues

Checklist

chenkins commented Nov 1, 2024 •

edited

Loading