Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add script for generating MuJoCo/Ant-v5 #5

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Add script for generating MuJoCo/Ant-v5 #5

wants to merge 11 commits into from

Conversation

Kallinteris-Andreas
Copy link

I have an expert policy (6.7k return) for the MuJoco/Ant-v5 environment (previous best was 5.8k from what I can tell)

I would like a review of the create_dataset.py before I run it, any other comments are welcome

younik
younik reviewed Oct 12, 2023
View reviewed changes
scripts/ant/create_dataset.py Outdated
from minari import DataCollectorV0, StepDataCallback
from stable_baselines3 import A2C, PPO, SAC, TD3
import stable_baselines3
assert stable-baselines3.__version__ == "2.0.0a5"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this line for?

younik
younik reviewed Oct 12, 2023
View reviewed changes
Copy link
Member

@younik younik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds good to me; we may want to use a different dataset name maybe

@Kallinteris-Andreas
Copy link
Author

What dataset name should i use

@younik
Copy link
Member

younik commented Oct 16, 2023

What dataset name should i use

uhm, maybe ant-v5-expert-v0?

@Kallinteris-Andreas
Copy link
Author

Kallinteris-Andreas commented Oct 17, 2023 •
edited
Loading

I have run the script, and it has generated 49 episodes with 200k steps plus 4 episodes with 10k+ steps each (totaling 200k steps)

Is that the desired behavior, or should I set the max_episode_length to 1000?

I am asking since minari has a dataset.sample_episodes() and not a dataset.sample_steps()

younik
younik requested changes Oct 17, 2023
View reviewed changes
Copy link
Member

@younik younik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, I missed the last part of the workflow, that may cause your problem. See the review comment

scripts/ant/create_dataset.py Outdated Show resolved Hide resolved
@Kallinteris-Andreas
Copy link
Author

Kallinteris-Andreas commented Oct 18, 2023 •
edited
Loading

Ok, I have updated it, recheck the script (Note: I will clean it up after you have confirmed the functionality is ok)

younik
younik reviewed Oct 18, 2023
View reviewed changes
scripts/ant/create_dataset.py Outdated
author="Kallinteris Andreas",
author_email="[email protected]",
)
collector_env.save_to_disk("test.hdf5")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this line, then LGTM

scripts/ant/create_dataset.py Outdated Show resolved Hide resolved
@Kallinteris-Andreas
Copy link
Author

Kallinteris-Andreas commented Mar 13, 2024 •
edited
Loading

Update:
I have

Note: I get these warnings, are any of them a problem?

/home/master-andreas/gym/rl/project/temp_env/lib/python3.11/site-packages/minari/utils.py:394: UserWarning: `minari_version` is set to None. The compatible dataset version specifier for Minari will be set to ~=0.4.
  warnings.warn(
/home/master-andreas/gym/rl/project/temp_env/lib/python3.11/site-packages/minari/utils.py:414: UserWarning: `eval_env` is set to None. If another environment is intended to be used for evaluation please specify corresponding Gymnasium environment (gym.Env | gym.envs.registration.EnvSpec).              If None the environment used to collect the data (`env=None`) will be used for this purpose.
  warnings.warn(
/home/master-andreas/gym/rl/project/temp_env/lib/python3.11/site-packages/minari/utils.py:435: UserWarning: env_spec is None, no environment spec is provided during collection for this dataset
  warnings.warn(

@younik

younik
younik reviewed Mar 21, 2024
View reviewed changes
Copy link
Member

@younik younik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good, thanks!

So you are proposing to upload this dataset to Minari remote, right?
@rodrigodelazcano what are your thoughts on that?

scripts/ant/create_dataset.py
from minari import DataCollector, StepDataCallback
from stable_baselines3 import A2C, PPO, SAC, TD3

from envs import ant_v5_release
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this env?

scripts/ant/create_dataset.py Outdated Show resolved Hide resolved
@Kallinteris-Andreas
Copy link
Author

Yes, it is my intention for it to be uploaded to the Minari dataset remote As the policy used is by far the best, It is able to achieve a return of 6.7k While the previous best is below 6k

@Kallinteris-Andreas
Copy link
Author

@younik is there anything blocking it merge?

@younik
Copy link
Member

younik commented Jun 12, 2024

@younik is there anything blocking it merge?

I belline the -v5 environments are available in gymnasium 1.0, isn't it? We should wait for that release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@younik younik younik requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Kallinteris-Andreas @younik