-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Direct Driver HAL #816
Merged
+8,432
−357
Merged
Direct Driver HAL #816
Changes from 34 commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
22ada20
[WIP] XRT-LITE HAL
makslevental 5a17ba0
add linux kmq shim (doesn't build)
makslevental be04318
compiles
makslevental 488b3d2
start from scratch
makslevental b972a98
bring shim back in-tree
makslevental 12a9844
impl allocator
makslevental b953f91
buffer impl
makslevental d1072ee
executable acche impl
makslevental 89a4a01
command buffer
makslevental f66ed50
e2e
makslevental 1494af7
hack tests for xrt-lite
makslevental f34f5b2
remove xclbin (and XRT) dep
makslevental d9c83cc
removed non-load-bearing vtable functions
makslevental ec92dfb
put shim_debug behind define
makslevental 63e917d
parameterize tests with device-hal
makslevental 9310e77
make xrt-lite default
makslevental 01787d8
remove null comments
makslevental a5e6785
refactor device.cc
makslevental 8aaa1d2
fix iree-benchmark
makslevental 976c418
undo OO
makslevental df0e66b
address comments
makslevental 0045d4f
add missing trace zones
makslevental 7356f7a
remove smart pointers
makslevental dc3d7b9
remove unnecessary sync to device
makslevental 57e0f2c
undo reinterpret_cast
makslevental e27d2c0
remove exceptions
makslevental 8bed954
parameterize n_rows, n_cols
makslevental 1a3efc4
incorporate comments
makslevental 514db1d
make xrt_lite_n_core_rows, xrt_lite_n_core_cols required
makslevental a224bcc
really make xrt-lite default and test building in "cleanroom"
makslevental db3cc6b
add iree-benchmark-module test
makslevental 8974d20
remove WERROR hack and add `run_all_runtime_tests.sh`
makslevental 05ddaec
Delete .github/workflows/ci-linux-cleanroom.yml
makslevental 66db9ce
Update compiler/plugins/target/AMD-AIE/iree-amd-aie/Target/AIETarget.h
makslevental 93a7ba5
Apply suggestions from code review
makslevental File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,12 +4,7 @@ | |
|
|
||
| # AMD AIE Plugin for IREE | ||
|
|
||
| This repository contains an early-phase IREE compiler and runtime plugin for interfacing the AMD AIE accelerator to IREE. | ||
|
|
||
| ## Architectural Overview | ||
|
|
||
|  | ||
|
|
||
| This repository contains an early-phase IREE compiler and runtime plugin for targeting AMD NPUs with IREE. | ||
|
|
||
| ## Developer Setup | ||
|
|
||
|
|
@@ -26,32 +21,30 @@ git clone --recursive [email protected]:nod-ai/iree-amd-aie.git | |
| git clone --recursive https://github.com/nod-ai/iree-amd-aie.git | ||
| ``` | ||
|
|
||
| or if you want a faster checkout | ||
|
|
||
| or, if you want a faster checkout, | ||
|
|
||
| ``` | ||
| git \ | ||
| -c submodule."third_party/torch-mlir".update=none \ | ||
| -c submodule."third_party/stablehlo".update=none \ | ||
| -c submodule."src/runtime_src/core/common/aiebu".update=none \ | ||
| -c submodule."third_party/XRT".update=none \ | ||
| clone \ | ||
| --recursive \ | ||
| --shallow-submodules \ | ||
| https://github.com/nod-ai/iree-amd-aie.git | ||
| [email protected]:nod-ai/iree-amd-aie.git # https://github.com/nod-ai/iree-amd-aie.git | ||
| ``` | ||
|
|
||
| The above avoids cloning entire repo histories, and skips unused nested submodules. | ||
| The above avoids cloning entire repo histories for submodules, and skips a few, currently, unused, | ||
| submodules that are nested in IREE. | ||
|
|
||
| ## Building (along with IREE) | ||
|
|
||
| ### Just show me the CMake | ||
|
|
||
| To configure and build with XRT runtime enabled | ||
|
|
||
| ``` | ||
| cd iree-amd-aie | ||
| cmake \ | ||
| -B $WHERE_YOU_WOULD_LIKE_TO_BUILD \ | ||
| -B <WHERE_YOU_WOULD_LIKE_TO_BUILD> \ | ||
| -S third_party/iree \ | ||
| -DIREE_CMAKE_PLUGIN_PATHS=$PWD \ | ||
| -DIREE_BUILD_PYTHON_BINDINGS=ON \ | ||
|
|
@@ -62,20 +55,20 @@ cmake \ | |
| -DIREE_TARGET_BACKEND_DEFAULTS=OFF \ | ||
| -DIREE_TARGET_BACKEND_LLVM_CPU=ON \ | ||
| -DIREE_BUILD_TESTS=ON \ | ||
| -DIREE_EXTERNAL_HAL_DRIVERS=xrt \ | ||
| -DCMAKE_INSTALL_PREFIX=$WHERE_YOU_WOULD_LIKE_TO_INSTALL | ||
| cmake --build $WHERE_YOU_WOULD_LIKE_TO_BUILD | ||
| -DIREE_EXTERNAL_HAL_DRIVERS=xrt-lite \ | ||
| -DCMAKE_INSTALL_PREFIX=<WHERE_YOU_WOULD_LIKE_TO_INSTALL> | ||
| cmake --build <WHERE_YOU_WOULD_LIKE_TO_BUILD> | ||
| ``` | ||
|
|
||
| ### Instructions | ||
|
|
||
| The bare minimum configure command for IREE with the amd-aie plugin | ||
| The bare minimum configure command for IREE with the amd-aie plugin | ||
|
|
||
| ``` | ||
| cmake \ | ||
| -B $WHERE_YOU_WOULD_LIKE_TO_BUILD \ | ||
| -S $IREE_REPO_SRC_DIR \ | ||
| -DIREE_CMAKE_PLUGIN_PATHS=$IREE_AMD_AIE_REPO_SRC_DIR \ | ||
| -B <WHERE_YOU_WOULD_LIKE_TO_BUILD> \ | ||
| -S <IREE_REPO_SRC_DIR> \ | ||
| -DIREE_CMAKE_PLUGIN_PATHS=<IREE_AMD_AIE_REPO_SRC_DIR> \ | ||
| -DIREE_BUILD_PYTHON_BINDINGS=ON | ||
| ``` | ||
|
|
||
|
|
@@ -88,7 +81,8 @@ Very likely, you will want to use `ccache` and `lld` (or some other modern linke | |
| -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld" | ||
| ``` | ||
|
|
||
| If you don't plan on using any of IREE's frontends or backends/targets (e.g., you're doing work on this code base itself), you can opt-out of everything (except the `llvm-cpu` backend) with | ||
| If you don't plan on using any of IREE's frontends or backends/targets (e.g., you're doing work on this code base itself), | ||
| you can opt-out of everything (except the `llvm-cpu` backend) with | ||
|
|
||
| ``` | ||
| -DIREE_INPUT_STABLEHLO=OFF \ | ||
|
|
@@ -111,75 +105,64 @@ If you're "bringing your own LLVM", i.e., you have a prebuilt/compiled distribut | |
| -DIREE_BUILD_BUNDLED_LLVM=OFF | ||
| ``` | ||
|
|
||
| In this case you will need to supply `-DLLVM_EXTERNAL_LIT=$SOMEWHERE` (e.g., `pip install lit; SOMEWHERE=$(which lit)`). | ||
| In this case you will need `lit` somewhere in your environment and you will need to add to CMake `-DLLVM_EXTERNAL_LIT=<SOMEWHERE>` | ||
| (e.g., `pip install lit; SOMEWHERE=$(which lit)`). | ||
|
|
||
| Note, getting the right/matching build of LLVM, that works with IREE is tough (besides the commit hash, there are various flags to set). | ||
| To enable adventurous users to avail themselves of `-DIREE_BUILD_BUNDLED_LLVM=OFF` we cache/store/save the LLVM distribution for every successful CI run. | ||
| These can then be downloaded by checking the artifacts section of any recent CI run's [Summary page](https://github.com/nod-ai/iree-amd-aie/actions/runs/10713474448): | ||
| See [Bringing your own LLVM](#bringing-your-own-llvm) below for more information on using prebuilt/compiled distributions of LLVM. | ||
|
|
||
| <p align="center"> | ||
| <img src="https://github.com/user-attachments/assets/97fdeff2-41af-4a6d-a072-6ef0a1ec5695" width="500"> | ||
| </p> | ||
| ## Testing | ||
|
|
||
| Lit tests specific to AIE can be run with something like | ||
| Lit tests (i.e., compiler tests) specific to AIE can be run with something like | ||
|
|
||
| ``` | ||
| cd $WHERE_YOU_WOULD_LIKE_TO_BUILD | ||
| ctest -R amd-aie | ||
| cd <WHERE_YOU_WOULD_LIKE_TO_BUILD> | ||
| ctest -R amd-aie --output-on-failure -j 10 | ||
| ``` | ||
|
|
||
| Other tests which run on hardware and requiring XRT are in the `build_tools` subdirectory. | ||
|
|
||
| ## Runtime driver setup | ||
| (the `-j 10` runs `10` tests in parallel) | ||
|
|
||
| To enable the runtime driver, you need to also enable the XRT HAL | ||
| Other tests, which run on device, are in the `build_tools` subdirectory. | ||
| See [build_tools/ci/run_all_runtime_tests.sh](build_tools/ci/run_all_runtime_tests.sh) for an example script that shows how to run all the runtime tests. | ||
|
|
||
| ``` | ||
| -DIREE_EXTERNAL_HAL_DRIVERS=xrt | ||
| ``` | ||
| ## Pro-tips | ||
|
|
||
| Additional IREE-specific flags are explained at [IREE's build instructions](https://iree.dev/building-from-source/getting-started/#quickstart-clone-and-build). To use Ninja instead of Make, and clang++ instead of g++, you can add | ||
| ### Bringing your own LLVM | ||
|
|
||
| When using a pre-built distribution of LLVM, getting the right/matching build, that works with IREE, is tough (besides the commit hash, there are various flags to set). | ||
| To enable adventurous users to avail themselves of `-DIREE_BUILD_BUNDLED_LLVM=OFF` we cache/store/save the LLVM distribution for every successful CI run. | ||
| These can then be downloaded by checking the artifacts section of any recent CI run's [Summary page](https://github.com/nod-ai/iree-amd-aie/actions/runs/10713474448): | ||
|
|
||
| ``` | ||
| -G Ninja \ | ||
| -DCMAKE_CXX_COMPILER=clang++ \ | ||
| -DCMAKE_C_COMPILER=clang | ||
| ``` | ||
| <p align="center"> | ||
| <img src="https://github.com/user-attachments/assets/97fdeff2-41af-4a6d-a072-6ef0a1ec5695" width="500"> | ||
| </p> | ||
|
|
||
|
|
||
| ### Ubuntu Dependencies | ||
| ### Debugging HAL | ||
|
|
||
| XRT requires a number of packages. Here are the requirements for various operating systems | ||
| You can turn on HAL API tracing by adding to CMake: | ||
|
|
||
| ``` | ||
| apt install \ | ||
| libcurl4-openssl-dev \ | ||
| libdrm-dev \ | ||
| libelf-dev \ | ||
| libprotobuf-dev \ | ||
| libudev-dev \ | ||
| pkg-config \ | ||
| protobuf-compiler \ | ||
| python3-pybind11 \ | ||
| systemtap-sdt-dev \ | ||
| uuid-dev | ||
| -DIREE_ENABLE_RUNTIME_TRACING=ON | ||
| -DIREE_TRACING_PROVIDER=console | ||
| // optional but recommended | ||
| -DIREE_TRACING_CONSOLE_FLUSH=1 | ||
| ``` | ||
|
|
||
| ### RH Based Deps | ||
| This will you show you all the HAL APIs that have `IREE_TRACE_ZONE_BEGIN ... IREE_TRACE_ZONE_END` that are hit during a run/execution (of, e.g., `iree-run-module`). | ||
|
|
||
| This is an incomplete list derived by adding what is needed to our development base manylinux (AlmaLinux 8) image. | ||
| You can turn on VM tracing by adding to CMake: | ||
|
|
||
| ``` | ||
| yum install \ | ||
| libcurl-devel \ | ||
| libdrm-devel \ | ||
| libudev-devel \ | ||
| libuuid-devel \ | ||
| ncurses-devel \ | ||
| pkgconfig \ | ||
| protobuf-compiler \ | ||
| protobuf-devel \ | ||
| systemtap-sdt-devel \ | ||
| uuid-devel | ||
| -DIREE_VM_EXECUTION_TRACING_ENABLE=1 | ||
| -DIREE_VM_EXECUTION_TRACING_FORCE_ENABLE=1 | ||
| // optional | ||
| -DIREE_VM_EXECUTION_TRACING_SRC_LOC_ENABLE=1 | ||
| ``` | ||
|
|
||
| This will show you all of the [VM dispatches](https://github.com/iree-org/iree/blob/0e8a5737dfe49a48a4e9c15ba7a7d24dd2fd7623/runtime/src/iree/vm/bytecode/dispatch.c#L661) that actually occur during a run/execution. | ||
| Note, this is roughly equivalent to [passing](https://github.com/nod-ai/iree-amd-aie/blob/737092791dc2428ad71bc172f69804c583b0f60e/build_tools/ci/run_matmul_test.sh#L420) `--compile-to=vm` to `iree-compile`. | ||
|
|
||
| ## Architectural overview (out of date) | ||
|
|
||
|  | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
look ma no XRT