System memory keeps on increasing

[kiransarv]

Hi All,

I am Kiran Sarvabhotla and I am using DJL for many linear algebra operations in image processing task. I am using NDList and NDArray with PyTorch Engine.

However, running continuously in a server, i could see System memory increasing slowly and eventually server crash. It starts from 6GB and goes on till 128GB.

What could be possible reason. I am creating and closing the NDManager, attaching all arrays to the NDManager, closing the NDArrays while working inside a loop.

Are there any documented best practices while using NDArray Operations like transpose, slice, div, mul etc.. etc...

[frankfliu]

@kiransarv

I normal case, you should not need to manually attaching NDArray to NDManager, closing NDManager should close all NDArrays it created.

Here is the document about using NDManager: https://docs.djl.ai/master/docs/development/memory_management.html

Do you have your that can reproduce your case?