Exception: "an exception occured at line 641"

[neon5d]

Hello,

I observed the exception "an exception occured at line 641" while running the program.
Initially, there are no exceptions but it happens more frequently while running the program although the program can still run.
What are the root causes and how can I avoid this exception?
I tried to adjust the parameters(num_bigchunkloaders, interval_resched, and maxlength_queue_smallchunk), but it didn't work.

[amirakbarnejad]

Hi neon5,
The exception occurs when a bigchunk is removed from memory and the the bigchunk's queue of smallchunks is being collected before removing the bigchunk.
A quick fix is to set the dataloader argument flag_grabqueue_onunsched to False. Note that this is not a good practice for SlidingWindowDL, but should be fine when, e.g., training a machine learning model.
After printing "an exception occured at line 641", pydmed prints more information about the exception. Could you please provide here the additional information that pydmed prints?

[neon5d]

Thank you for your quick reply.
In my case,
print(str(e)) does not show any messages (It seems only has the next line)
If you give any updates, I can try to catch the message.

After setting flag_grabqueue_onunsched to be False, no more the exception.
Thank you for your help.

[amirakbarnejad]

It turned out that even when that line fails, all smallchunks are correctly collected.
So I changed the error to a warning:

PyDmed/pydmed/lightdl.py

Line 753 in 8ef0f6b

print("Warning: Some smallchunks may have lost. If not, you can safely ignore this warning.")