[Feature] 支持AMD GPU

[yeungtuzi]

检查清单

• 1. 如果您提出的不是新功能而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
• 2. 为方便社区交流，我将使用中文/英文或附上英文/中文翻译（如使用其他语言）。未附带翻译的非英文/中文内容可能会被关闭

需求背景

使用hip来翻译cuda指令到rocm，以支持amd gpu

相关资源

No response

[ubergarm]

@yeungtuzi

使用hip来翻译cuda指令到rocm，以支持amd gpu

我在想是否可以通过Vulkan实现兼容方案，因为它似乎同时支持NVIDIA和AMD显卡。

虽然初衷可能不是专门支持AMD，但考虑到Vulkan在NVIDIA设备上的性能可能超越CUDA，这种方案或许更具普适性。

技术迭代速度实在惊人，二月的进展仿佛跨越全年。衷心感谢开发者们的辛勤付出。若急需解决方案，可根据配置尝试vLLM或llama.cpp。不过就目前而言，在配备充足内存的单CUDA GPU机器上，ktransformers仍是运行R1 671B最快的选择。

祝各位顺利！

---

yeungtuzi

[Feature] Support for AMD GPUs

Use HIP to translate CUDA instructions to ROCm, enabling support for AMD GPUs.

I wonder if there is a path forward with Vulkan given it seems to support both nvidia and amd GPUs.

I imagine this will happen not so much to support AMD specifically as much as vulkan might be faster than CUDA on nvidia.

Anyway, all this stuff is moving fast. February felt like a whole year. I appreciate all the developers hard work. If you need something today, you can also check out vLLM or llama.cpp depending on your exact configuration. However, ktransformers is the fastest available for R1 671B on a machine with enough RAM and a single cuda GPU.

Good luck everybody!