torch.OutOfMemoryError: CUDA out of memory with Runpod A100

[nanaj96]

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 42.00 MiB. GPU 0 has a total capacity of 44.52 GiB of which 2.00 MiB is free. Process 280468 has 5.57 GiB memory in use. Process 280474 has 5.35 GiB memory in use. Process 280470 has 5.53 GiB memory in use. Process 280467 has 5.57 GiB memory in use. Process 280473 has 5.75 GiB memory in use. Process 280472 has 5.49 GiB memory in use. Process 280469 has 5.65 GiB memory in use. Process 280471 has 5.57 GiB memory in use. Of the allocated memory 4.92 GiB is allocated by PyTorch, and 134.41 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[FurkanGozukara]

Diffusers with optimization works

I have tutorial : https://youtu.be/GjENQfHF4W8

[nanaj96]

Diffusers with optimization works

I have tutorial : https://youtu.be/GjENQfHF4W8

is this multiple GPU training config?
Sana_1600M_img1024.yaml

[FurkanGozukara]

@nanaj96 if you are trying training that is different i replied for inference

[nanaj96]

@nanaj96 if you are trying training that is different i replied for inference
yeah I'm trying training

[nanaj96]

@FurkanGozukara do you know about this?

[FurkanGozukara]

@FurkanGozukara do you know about this?

i didnt train yet so dont know sorry