Need help: Utilize multi-GPUs using DirectML.

[lshqqytiger]

Hello, I am a new contributer working on DirectML tree.
I want to find out whether it is able to move tensors between different/same vendors of GPUs.
I have a AMD card, and also NVIDIA one, but NVIDIA one is too old to run DirectML. (it does not support DirectX12 fully)
If you have any two GPUs (vendor is not important! just ought to support DirectX12. including iGPUs),
please run code below and attach screenshot of the result!

Before run test code, you should install torch-directml.

$ pip install torch-directml

Then, using python interpreter, run the code below.

$ python
>>> import torch, torch_directml
>>> dev0 = torch_directml.device(0)
>>> dev1 = torch_directml.device(1)
>>> ten = torch.randn(1)
>>> ten # for check output
>>> ten = ten.to(dev0)
>>> ten # for check output
>>> ten = ten.to(dev1)
>>> ten # for check output

(Some Intel iGPU cannot print tensors)

Thank you!

[neurogen-dev]

2x 3060Ti

[lshqqytiger]

Thank you for testing! Unfortunately the contents seem to be gone when moved tensors between DirectML devices..
Can you test the code below?

$ python
>>> import torch, torch_directml
>>> dev0 = torch_directml.device(0)
>>> dev1 = torch_directml.device(1)
>>> ten = torch.randn(1)
>>> ten = ten.to(dev0)
>>> ten
>>> ten = ten.cpu()
>>> ten
>>> ten = ten.to(dev1)
>>> ten

Thank you again!

[neurogen-dev]

Can you test the code below?

[lshqqytiger]

Thank you very much!

[neurogen-dev]

Thank you very much!

If you need help with the multi-gpu test, please contact me, I will be happy to help. I don't have an AMD computer at the moment, but I have a computer with a 3060 Ti x3, and I can test on it.

[lshqqytiger]

I found the data parallelism works on torch compiled with CUDA.
So, unfortunately, it is unavailable for DirectML now.
But it is good to know we can move tensors between devices via cpu.