MultiThreadedAugmenter.abort_event was set

[SnaKey0u0]

Dear author,

I am trying to implement it for my own 3D medical image datasets, but have run into an error when switching between tasks in the ewc training.

I am using this command:

nnUNet_train_ewc 3d_fullres -t 998 999 -f 0 -ewc_lambda 2.3 -num_epoch 250 -save_interval 25 -s seg_outputs --use_compressed --reduce_threads

On task switch, I get the following error:

MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of your workers crashed. This is not the actual error message! Look further up your stdout to see what caused the error. Please also check whether your RAM was full error.

I have checked my CPU and memory usage are around 10-20% when this happens. I have tried adjusting parameters like pin_memory and num_processes but the issue persists (even the epoch is 1).

Could you please advise if I'm missing anything in my configuration or usage? My system specs are:
CPU: i7-7700
GPU: RTX 2070
RAM: 64GB

I would greatly appreciate any pointers you can provide to resolve this.
Thank you!