Switch to SPIRV APIs from internal built-in APIs

[jiyang1011]

sycl_compiler_path=/opt/cutlass/compiler/20250304/
gpu_driver_path=/opt/cutlass/gpu_driver/gfx-driver-ci-comp_igc-28698/extract

This PR is a draft. There is something wrong with SHAPE.
The example pvc_gemm performance dropped a lot.
I will check it
@rolandschulz @mehdi-goli @tdeng5 please take a look

[aacostadiaz]

Are these the same functions as the ones defined in https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_2d_block_io.html ?

[jiyang1011]

Are these the same functions as the ones defined in https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_2d_block_io.html ?

Yes

[aacostadiaz]

Are these the same functions as the ones defined in https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_2d_block_io.html ?

Yes

Maybe I'm missing something but the functions in the link use template parameters to define the 2D copy block dimensions, allowing the selection of the internal copy builtin at compile time.

The functions in the PR, however, use runtime variables for this, meaning the selection happens at runtime. This could explain the drop in performance.

[jiyang1011]

Are these the same functions as the ones defined in https://github.khronos.org/SPIRV-Registry/extensions/INTEL/SPV_INTEL_2d_block_io.html ?

Yes

Maybe I'm missing something but the functions in the link use template parameters to define the 2D copy block dimensions, allowing the selection of the internal copy builtin at compile time.

The functions in the PR, however, use runtime variables for this, meaning the selection happens at runtime. This could explain the drop in performance.

In the spec, it is just an example to show the semantics, but it is not an API definition. Spec defines only SPIRV API. And I find prefetch functionality is the root cause, which seems not working well.

[aacostadiaz]

@jiyang1011 Could you try adding something like 36aa2d4 to use SPIRV or the builtins depending on the compiler version? We might need to disable SPIRV before merging until we can update CI with an IGC version that includes the SPIRV definitions.

[jiyang1011]

@jiyang1011 Could you try adding something like 36aa2d4 to use SPIRV or the builtins depending on the compiler version? We might need to disable SPIRV before merging until we can update CI with an IGC version that includes the SPIRV definitions.

OK

[jiyang1011]

@aacostadiaz

[joeatodd]

Hey @jiyang1011 👋 I'm struggling to compile e.g. pvc_gemm.cpp with this PR. I am using the 2025-03-24 nightly build of DPC++ and version 28698 of IGC. I get IGC: Internal Compiler Error: Segmentation violation. Can you help?

[joeatodd]

Thanks for this PR! I have a few suggestions for improvements 👍

[aacostadiaz]

Hey @jiyang1011 👋 I'm struggling to compile e.g. pvc_gemm.cpp with this PR. I am using the 2025-03-24 nightly build of DPC++ and version 28698 of IGC. I get IGC: Internal Compiler Error: Segmentation violation. Can you help?

I made some changes in https://github.com/aacostadiaz/cutlass-fork/tree/aacosta/spirv_api to make the code compile and run with IGC 28698. There is an issue with the SPIRV functions for MMA, so I had to disable them. I also enabled prefetch to always use the builtins, which restored the performance of the GEMM example. I haven't checked all configurations in the benchmark.

@jiyang1011 Could you take a look and consider adding these changes to this PR? Do you know what the issue is with the SPIRV functions for MMA? Do we need a different version of IGC for it to work?

[jiyang1011]

Hey @jiyang1011 👋 I'm struggling to compile e.g. pvc_gemm.cpp with this PR. I am using the 2025-03-24 nightly build of DPC++ and version 28698 of IGC. I get IGC: Internal Compiler Error: Segmentation violation. Can you help?

I made some changes in https://github.com/aacostadiaz/cutlass-fork/tree/aacosta/spirv_api to make the code compile and run with IGC 28698. There is an issue with the SPIRV functions for MMA, so I had to disable them. I also enabled prefetch to always use the builtins, which restored the performance of the GEMM example. I haven't checked all configurations in the benchmark.

@jiyang1011 Could you take a look and consider adding these changes to this PR? Do you know what the issue is with the SPIRV functions for MMA? Do we need a different version of IGC for it to work?

I think the include chain makes the compilation failure maybe. I reorged it following @joeatodd 's suggestion. 2025-03-24 nightly build of DPC++ and version 28698 of IGC. are OK

[jiyang1011]

hi @aacostadiaz Could you help me debug the CI. I have looked into the info of CI, but I have no clue where the error is

[aacostadiaz]

hi @aacostadiaz Could you help me debug the CI. I have looked into the info of CI, but I have no clue where the error is

It looks like the current version of IGC still doesn't have the SPIRV functions. We will need to disable SPIRV until it is available. This is IGC version 1.6.32224+14 in CI.

cc @rolandschulz