How to specify FastLanguageModel into specific gpu

[Hyfred]

In my code, I need to load two models, one specify "cuda:0" one for "cuda:1"
But it doesn't work, my code snippet is shown as below:

`
#. Specify devices for each model
device_model1 = torch.device("cuda:1")
device_model2 = torch.device("cuda:0")
model1, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
max_seq_length = max_seq_length,
dtype = None,
load_in_4bit = True,
)
model1.to(device_model1)

model2, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
max_seq_length = max_seq_length,
dtype = None,
load_in_4bit = True,
)
model2.to(device_model2)
`

Any comment would be helpful, thanks!!

[Hyfred]

now the error is:

Exception has occurred: RuntimeError
CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
File "/home/a1/unsloth_llama_finetune.py", line 40, in
model1.to(device_model1)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

[danielhanchen]

Sorry currently Unsloth does not function on Python runtimes which expose 2 or more GPUs - it's still an active area of developmenet