OffloadTest

Experimental Runtime test suite for HLSL

Current Status

Testing Machine	DXC	Clang
Windows DirectX12 Intel GPU
Windows DirectX12 Warp
Windows Vulkan Intel GPU
macOS Apple M1

Prerequisites

Requires the Vulkan 1.4 SDK.

This project requires being able to locally build LLVM and leverages LLVM's build infrastructure. It also requires installing the pyyaml Python package. You can install pyyaml by running:

pip3 install pyyaml

On Windows, the Graphics Tools optional feature is additionally required to run the test suite.

Adding to LLVM Build

Add the following to the CMake options:

-DLLVM_EXTERNAL_OFFLOADTEST_SOURCE_DIR=${workspaceRoot}\..\OffloadTest -DLLVM_EXTERNAL_PROJECTS="OffloadTest"

If you do not have a build of dxc on your path you'll need to specify the shader compiler to use by passing:

-DDXC_DIR=<path to folder containing dxc & dxv>

Enabling clang-tidy

The offload test suite's code is clang-tidy clean for a limited ruleset. If you have clang-tidy installed locally you can enable clang-tidy by adding -DOFFLOADTEST_USE_CLANG_TIDY=On to your CMake invocation. You can also add -DOFFLOADTEST_CLANG_TIDY_APPLY_FIX=On to enable automatically applying the clang-tidy fix-its for any warnings that have automated fixes.

YAML Pipeline Format

This framework provides a YAML representation for describing GPU pipelines and buffers. The format is implemented by the API/Pipeline.{h|cpp} sources. The following is an example pipeline YAML description:

---
Shaders:
  - Stage: Compute
    Entry: main
    DispatchSize: [1, 1, 1]
Buffers:
  - Name: Constants
    Format: Int32
    Data: [ 1, 2, 3, 4, 5, 6, 7, 8]
  - Name: In1
    Format: Float32
    Data: [ 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]
  - Name: In2
    Format: Hex16
    Data: [ 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8]
  - Name: Out1 # Buffer where our output will go
    Format: Float32
    Stride: 4
    ZeroInitSize: 8
  - Name: Expected1 # Buffer which stores the expected result of our test
    Format: Float32
    Stride: 4
    Data: [ 0.0, 1.0 ]
  - Name: Out2 # Buffer where our output will go
    Format: Float16
    Stride: 2
    ZeroInitSize: 4 # ZeroInitSize needs to be 4 bytes minimum
  - Name: Expected2 # Buffer which stores the expected result of our test
    Format: Float16
    Stride: 2
    Data: [ 0x1, 0x2 ]
Results: # Using Result can verify test values without filecheck
  - Result: Test1
    Rule: BufferFloatULP # Rule which can be used to compare Float Buffers; They are compared within a ULP range
    ULPT: 1 # ULP to use
    DenormMode: Any # if DenormMode Field is not Specified, 'Any' is the default; FTZ and Preserve are the other options.
    Actual: Out1 # First buffer to compare
    Expected: Expected1 # Second buffer to compare against first
  - Result: Test2
    Rule: BufferExact # Compares Two Buffers for == equality between each value elementwise
    Actual: Out1
    Expected: Expected1
  - Result: Test3
    Rule: BufferFloatEpsilon # Rule which can be used to compare Float Buffers; They are compared within an epsilon difference
    Epsilon: 0.0008
    Actual: Out1
    Expected: Expected1
DescriptorSets:
  - Resources:
    - Name: Constants
      Kind: ConstantBuffer
      DirectXBinding:
        Register: 0 # implies b0 due to Access being Constant
        Space: 0
      VulkanBinding:
        Binding: 0 # [[vk::binding(0, 0)]]
    - Name: In1
      Kind: Buffer
      DirectXBinding:
        Register: 0 # implies t0 due to Access being RO
        Space: 0
      VulkanBinding:
        Binding: 10
  - Resources:
    - Name: In2
      Kind: Buffer
      DirectXBinding:
        Register: 1 # implies t1 due to Access being RO
        Space: 0
      VulkanBinding:
        Binding: 0 # [[vk::binding(0, 1)]]
...