-
|
The GPU usage is low during training, and when I reduce the sim_rate to 0.5, nan exists in actions and states. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Yes, this environment is CPU heavy and spends a decent amount of time on environment propagation as opposed to learning. We recommend using an RL library like RLlib for training, as it properly parallelized environment sampling (as opposed to Stable Baselines, which requires every rollout to complete the step before taking the next step in parallel). Please post a minimal example that produces nans, as I have not encountered that issue using a sim_rate of 0.5. Also, not that sim rate is not properly a rate and actually has units of seconds, so if you would like a lower time-fidelity simulation to increase training speed, you will want to use a higher value for sim_rate. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you. I apologize for typing the wrong sim_rate. When I set the sim_rate to 0.5, the training speed is slow and there is no nan exists during training process. When I use a value such as 2 or higher, as you said, the training speed is fast, but the nans exist. |
Beta Was this translation helpful? Give feedback.
Yes, this environment is CPU heavy and spends a decent amount of time on environment propagation as opposed to learning. We recommend using an RL library like RLlib for training, as it properly parallelized environment sampling (as opposed to Stable Baselines, which requires every rollout to complete the step before taking the next step in parallel).
Please post a minimal example that produces nans, as I have not encountered that issue using a sim_rate of 0.5. Also, not that sim rate is not properly a rate and actually has units of seconds, so if you would like a lower time-fidelity simulation to increase training speed, you will want to use a higher value for sim_rate.