update to rocm 6.3 wheels

[johnnynunez]

• Pytorch supports cuda 12.6 and rocm 6.2.4, 6.3
  Better performance rocm6.3 https://community.amd.com/t5/ai/unlocking-new-horizons-in-ai-and-hpc-with-the-release-of-amd/ba-p/726434

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu126
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.2.4
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.3

[tenpercent]

cc @bottler @danthe3rd

[bottler]

@tenpercent I think you have much more context on a PR like this than me. If you say you want it merged, I can merge it! But I can't really review it.

[tenpercent]

win build is failing and torch version should be same between building and uploading flows

[tenpercent]

looks good, let's wait for CI and merge

[bottler]

@tenpercent @johnnynunez Not sure if you see the errors on the CI? Like this on 6.1 and 6.2

cp: cannot stat 'all-dist/*torch2.5.1+rocm6.2*_facebookresearch/*.whl': No such file or directory

[tenpercent]

@johnnynunez also it looks like rocm6.3 build hasn't actually built the flash attention extension
https://github.com/facebookresearch/xformers/actions/runs/12795689183/job/35728108356?pr=1192
typically the number of sources is around 600

[tenpercent]

@johnnynunez I think the root cause is there is no torch wheel uploaded for rocm6.3 on download.pytorch.org. It'll appear there in some time. I think, for a meantime, we can build at least for rocm6.2

[johnnynunez]

@johnnynunez I think the root cause is there is no torch wheel uploaded for rocm6.3 on download.pytorch.org. It'll appear there in some time. I think, for a meantime, we can build at least for rocm6.2

I ask to pytorch team, and I can confirm that:
https://dev-discuss.pytorch.org/t/pytorch-2-6-release-branch-cut-for-pytorch-core-is-completed/2656/5?u=johnnynunez

rocm6.3 is coming with pytorch 2.7
I think that we can keep it for build but not for upload wheels.
What do you think?

[tenpercent]

@johnnynunez I think the root cause is there is no torch wheel uploaded for rocm6.3 on download.pytorch.org. It'll appear there in some time. I think, for a meantime, we can build at least for rocm6.2

I ask to pytorch team, and I can confirm that: https://dev-discuss.pytorch.org/t/pytorch-2-6-release-branch-cut-for-pytorch-core-is-completed/2656/5?u=johnnynunez

rocm6.3 is coming with pytorch 2.7 I think that we can keep it for build but not for upload wheels. What do you think?

I think right now we need the torch wheel built for rocm6.3 as a dependency to build the xformers wheel for rocm6.3. Even though the cpp extension for ROCm doesn't use any API specific to rocm6.3
The alternative might be to use a torch wheel built for rocm6.2 but then there would be runtime warning about torch version mismatch
upd: in that place

xformers/xformers/_cpp_lib.py

Lines 73 to 75 in 536363e

	msg = f"""xFormers was built for:
	PyTorch {self.build_info.torch_version} with CUDA {self.build_info.cuda_version} (you have {torch.__version__})
	Python {self.build_info.python_version} (you have {platform.python_version()})"""

[johnnynunez]

@johnnynunez I think the root cause is there is no torch wheel uploaded for rocm6.3 on download.pytorch.org. It'll appear there in some time. I think, for a meantime, we can build at least for rocm6.2

I ask to pytorch team, and I can confirm that: https://dev-discuss.pytorch.org/t/pytorch-2-6-release-branch-cut-for-pytorch-core-is-completed/2656/5?u=johnnynunez
rocm6.3 is coming with pytorch 2.7 I think that we can keep it for build but not for upload wheels. What do you think?

I think right now we need the torch wheel built for rocm6.3 as a dependency to build the xformers wheel for rocm6.3. Even though the cpp extension for ROCm doesn't use any API specific to rocm6.3 The alternative might be to use a torch wheel built for rocm6.2 but then there would be a runtime warning about torch version mismatch

Maybe maintain this philosophy. Why did I do this? In my case, rocm is better and better in each version supporting new features and fewer errors builds. For example, I went from 364 compilation errors with rocm6.1 to 20 with rocm6.2 to 7 with rocm6.3 in flash attention for my w7900 dual slot

PY_ROCM = [(PY_VERSIONS[-1], ROCM_VERSIONS[-2])] # Always last version is upcoming version