fine-tuning with multiple GPUs

[Hongyuan-Liu]

I have 8 NVIDIA GeForce RTX 4090 GPUs, and I want to use them for fine-tuning with Unisloth. However, I found that I can only use one GPU at a time. How can I set up my environment to perform fine-tuning with multiple GPUs?

[2662007798]

目前好像不支持 只支持单卡

[shimmyshimmer]

We currently only support single GPU. Hopefully will be supported soon enough

[wuzepei]

So when will we get multi-GPU support and hope to get a reply as soon as possible