Memory usage constantly growing while training

[hithxh]

env: general_satellite_tasking
alg: rllib.appo

Have you ever encountered this problem?

[Mark2000]

Yes, there are some issues (#31 ) that reference this. We’ve made some improvements but still see about 3-4 Gb/worker/day of memory growth. Currently running some experiments to investigate further.

[Mark2000]

See also some conversations about memory leaks in the Basilisk repository for more details.

[hithxh]

Maybe this question has something to do with the way rllib is implemented?
see ray-project/ray#3884
Since appo mimics IMPALA, and IMPALA have the same problem.

[Mark2000]

I’ve been trying some runs with regular PPO and it seems potentially better from a memory standpoint, but I believe that just looping the environment without training shows some memory growth as well.