From 1852beab473b678452eda4d8069fd10283dcf6c3 Mon Sep 17 00:00:00 2001 From: Sambhav Jain Date: Thu, 19 Oct 2023 18:08:56 -0700 Subject: [PATCH] [RFC + PR] Use TCP for {LLVM / Torch-MLIR / StableHLO} Green Commit Sync (#11) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Why When bumping LLVM up, it is crucial to be able to test all downstream repos depending on it to ensure they work **in tandem** (and not just in isolation). In the past, LLVM upgrades were simpler because torch-mlir took a hard dependency on mhlo/stablehlo and, in doing so, ensured that the llvm "green commit" (sha1) that torch-mlir and stablehlo were built+tested against was pre-identified. During this time mlir-tcp was developed on a branch of torch-mlir. This meant when upgrades were needed downstream, we’d simply point to torch-mlir@HEAD (sha4) and pick the llvm-project (sha1) and mhlo/stablehlo (sha3) hashes it’d refer to, since these are already tested to work together. This became our set of green commits (llvm@sha1, stablehlo@sha3, torch-mlir@sha4) for downstream integrations (e.g cruise monorepo). image At present the situation is complicated because torch-mlir no longer takes a hard dependency on stablehlo (stablehlo e2e tests [disabled](https://github.com/llvm/torch-mlir/pull/2460)). Here's details from a recent upgrade scenario that motivated this RFC. We picked torch-mlir@HEAD which was right after the llvm bump in https://github.com/llvm/torch-mlir/pull/2511 pointing to https://github.com/llvm/llvm-project/commit/b44b3494f60296db6aca38a14cab061d9b747a0a, but soon realized (when we started building torch-mlir) that the llvm bazel build upstream was broken: ``` ERROR: /root/.cache/bazel/_bazel_root/b89349c08f7224396763d14fe35cba11/external/llvm-project/mlir/BUILD.bazel:5837:18: TdGenerate external/llvm-project/mlir/include/mlir/Dialect/LLVMIR/NVVMOpsInterface.h.inc failed: (Exit 1): mlir-tblgen failed: error executing command ... external/llvm-project/mlir/include/mlir/Dialect/LLVMIR/NVVMOps.td:20:9: error: Could not find include file 'mlir/Dialect/LLVMIR/BasicPtxBuilderInterface.td' include "mlir/Dialect/LLVMIR/BasicPtxBuilderInterface.td" ^ ``` The bazel fixes followed in a subsequent commit at https://github.com/llvm/llvm-project/commit/28b27c1b10ae8d1f5b4fb9df691e8cf0da9be3f6. Hence llvm had to be re-bumped in torch-mlir (https://github.com/llvm/torch-mlir/pull/2517). However, after a bit more work we hit these failing stablehlo tests, which surfaced the fact that stablehlo pointed to by torch-mlir could no longer be used, and we had to separately identify the sha3 of stablehlo that would build cleanly against sha1 of llvm. ``` @stablehlo//stablehlo/conversions/tosa/tests:binary.mlir.test FAILED in 0.7s @stablehlo//stablehlo/tests:print_stablehlo.mlir.test FAILED in 4.7s ``` This meant the burden of identifying the llvm green commit (that works across the board) is shifted further downstream from torch-mlir. Incidentally we are in a great position to leverage mlir-tcp to identify the set of green commits, given it already directly depends on each of these repos. image ## What This PR is an attempt to leverage the mlir-tcp repo as our "proxy" for such downstream integrations, and _I think_ contains everything needed to be able to do that. ## How Specifically, we should now be able to run these from the comfort of `mlir-tcp`: ```shell bazel test --config=clang_linux @llvm-project//mlir/... bazel test --config=clang_linux @stablehlo//... bazel test --config=clang_linux @torch-mlir//... ``` We provide `local_repos.bzl` that allows easier local testing of patches that later need to be upstreamed, and while they're being upstreamed we could land them as patches to our `http_archive` targets. Note: I include a `stablehlo.patch` that allows testing stablehlo from `mlir-tcp`. This is temporary and can be removed once https://github.com/openxla/stablehlo/pull/1810 lands. This PR also enables each of the 3p test suites as GHA workflows (non-merge gating for now, we can change this). These workflows are automatically skipped unless a change is made to `deps.bzl` (which usually means bumping 3p deps), as it would be unnecessary to run them for every PR and `main` commit post-merge. Here's a snapshot from this PR's workflows, having bumped stablehlo commit. image --- .bazelignore | 6 + .github/workflows/bazelBuildAndTestLlvm.yml | 82 ++++++++++ .../workflows/bazelBuildAndTestStablehlo.yml | 82 ++++++++++ .github/workflows/bazelBuildAndTestTcp.yml | 12 +- .../workflows/bazelBuildAndTestTorchmlir.yml | 82 ++++++++++ .gitignore | 14 +- README.md | 14 +- deps.bzl | 90 +++++++---- local_repos.bzl | 36 +++++ stablehlo.patch | 145 ++++++++++++++++++ 10 files changed, 527 insertions(+), 36 deletions(-) create mode 100644 .bazelignore create mode 100644 .github/workflows/bazelBuildAndTestLlvm.yml create mode 100644 .github/workflows/bazelBuildAndTestStablehlo.yml create mode 100644 .github/workflows/bazelBuildAndTestTorchmlir.yml create mode 100644 local_repos.bzl create mode 100644 stablehlo.patch diff --git a/.bazelignore b/.bazelignore new file mode 100644 index 00000000..6e1e2bd6 --- /dev/null +++ b/.bazelignore @@ -0,0 +1,6 @@ +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + +third_party/ diff --git a/.github/workflows/bazelBuildAndTestLlvm.yml b/.github/workflows/bazelBuildAndTestLlvm.yml new file mode 100644 index 00000000..0526a490 --- /dev/null +++ b/.github/workflows/bazelBuildAndTestLlvm.yml @@ -0,0 +1,82 @@ +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + +name: Bazel Build and Test (llvm-project) + +# Only run when llvm-project hash changes (deps.bzl) +on: + pull_request: + branches: + - main + paths: + - 'deps.bzl' + push: + branches: + - main + paths: + - 'deps.bzl' + workflow_dispatch: + +# Ensure that only a single job or workflow using the same +# concurrency group will run at a time. This would cancel +# any in-progress jobs in the same github workflow and github +# ref (e.g. refs/heads/main or refs/pull//merge). +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + + +jobs: + ubuntu-build: + name: ubuntu-x86_64 / llvm-project + runs-on: ubuntu-latest + + steps: + - name: Checkout mlir-tcp + uses: actions/checkout@v3 + + # Continually update cache even if there's a "hit" during + # restore to avoid the cache going stale over time + # https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache + - name: Setup cache for bazel + uses: actions/cache@v3 + with: + path: ~/.cache/bazel + key: llvm-project-bazel-build-cache-${{ runner.os }}-${{ github.sha }} + restore-keys: | + llvm-project-bazel-build-cache-${{ runner.os }} + + # Change bazel cache directory to root ownership + # to allow writing to it from within the docker container. + # If no cache hits, this directory is not present + # so don't run chown (will error otherwise). + - name: Set bazel cache permissions + run: | + if [ -d "${HOME}/.cache/bazel" ]; then + sudo chown -R root:root "${HOME}/.cache/bazel" + fi + + - name: Build docker image + run: | + docker build -f docker/Dockerfile \ + -t mlir-tcp:ci \ + . + + - name: Bazel build and test llvm-project + run: | + docker run --rm \ + -v "$(pwd)":"/opt/src/mlir-tcp" \ + -v "${HOME}/.cache/bazel":"/root/.cache/bazel" \ + mlir-tcp:ci \ + bazel test --config=clang_linux @llvm-project//mlir/... + + # Switch back bazel cache directory to user ownership + # to allow GHA post-cache step to save cache without + # permissions issue. + - name: Switch bazel cache permissions + run: | + if [ -d "${HOME}/.cache/bazel" ]; then + sudo chown -R "$USER":"$USER" "${HOME}/.cache/bazel" + fi diff --git a/.github/workflows/bazelBuildAndTestStablehlo.yml b/.github/workflows/bazelBuildAndTestStablehlo.yml new file mode 100644 index 00000000..f6912db9 --- /dev/null +++ b/.github/workflows/bazelBuildAndTestStablehlo.yml @@ -0,0 +1,82 @@ +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + +name: Bazel Build and Test (stablehlo) + +# Only run when stablehlo hash changes (deps.bzl) +on: + pull_request: + branches: + - main + paths: + - 'deps.bzl' + push: + branches: + - main + paths: + - 'deps.bzl' + workflow_dispatch: + +# Ensure that only a single job or workflow using the same +# concurrency group will run at a time. This would cancel +# any in-progress jobs in the same github workflow and github +# ref (e.g. refs/heads/main or refs/pull//merge). +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + + +jobs: + ubuntu-build: + name: ubuntu-x86_64 / stablehlo + runs-on: ubuntu-latest + + steps: + - name: Checkout mlir-tcp + uses: actions/checkout@v3 + + # Continually update cache even if there's a "hit" during + # restore to avoid the cache going stale over time + # https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache + - name: Setup cache for bazel + uses: actions/cache@v3 + with: + path: ~/.cache/bazel + key: stablehlo-bazel-build-cache-${{ runner.os }}-${{ github.sha }} + restore-keys: | + stablehlo-bazel-build-cache-${{ runner.os }} + + # Change bazel cache directory to root ownership + # to allow writing to it from within the docker container. + # If no cache hits, this directory is not present + # so don't run chown (will error otherwise). + - name: Set bazel cache permissions + run: | + if [ -d "${HOME}/.cache/bazel" ]; then + sudo chown -R root:root "${HOME}/.cache/bazel" + fi + + - name: Build docker image + run: | + docker build -f docker/Dockerfile \ + -t mlir-tcp:ci \ + . + + - name: Bazel build and test stablehlo + run: | + docker run --rm \ + -v "$(pwd)":"/opt/src/mlir-tcp" \ + -v "${HOME}/.cache/bazel":"/root/.cache/bazel" \ + mlir-tcp:ci \ + bazel test --config=clang_linux @stablehlo//... + + # Switch back bazel cache directory to user ownership + # to allow GHA post-cache step to save cache without + # permissions issue. + - name: Switch bazel cache permissions + run: | + if [ -d "${HOME}/.cache/bazel" ]; then + sudo chown -R "$USER":"$USER" "${HOME}/.cache/bazel" + fi diff --git a/.github/workflows/bazelBuildAndTestTcp.yml b/.github/workflows/bazelBuildAndTestTcp.yml index 037b83a3..2c731bb5 100644 --- a/.github/workflows/bazelBuildAndTestTcp.yml +++ b/.github/workflows/bazelBuildAndTestTcp.yml @@ -3,13 +3,15 @@ # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception # Also available under a BSD-style license. See LICENSE. -name: Bazel Build and Test +name: Bazel Build and Test (mlir-tcp) on: pull_request: - branches: [ main ] + branches: + - main push: - branches: [ main ] + branches: + - main workflow_dispatch: # Ensure that only a single job or workflow using the same @@ -23,7 +25,7 @@ concurrency: jobs: ubuntu-build: - name: ubuntu-x86_64 + name: ubuntu-x86_64 / mlir-tcp runs-on: ubuntu-latest steps: @@ -32,7 +34,7 @@ jobs: # Continually update cache even if there's a "hit" during # restore to avoid the cache going stale over time - # https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache + # https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache - name: Setup cache for bazel uses: actions/cache@v3 with: diff --git a/.github/workflows/bazelBuildAndTestTorchmlir.yml b/.github/workflows/bazelBuildAndTestTorchmlir.yml new file mode 100644 index 00000000..5619769a --- /dev/null +++ b/.github/workflows/bazelBuildAndTestTorchmlir.yml @@ -0,0 +1,82 @@ +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + +name: Bazel Build and Test (torch-mlir) + +# Only run when torch-mlir hash changes (deps.bzl) +on: + pull_request: + branches: + - main + paths: + - 'deps.bzl' + push: + branches: + - main + paths: + - 'deps.bzl' + workflow_dispatch: + +# Ensure that only a single job or workflow using the same +# concurrency group will run at a time. This would cancel +# any in-progress jobs in the same github workflow and github +# ref (e.g. refs/heads/main or refs/pull//merge). +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + + +jobs: + ubuntu-build: + name: ubuntu-x86_64 / torch-mlir + runs-on: ubuntu-latest + + steps: + - name: Checkout mlir-tcp + uses: actions/checkout@v3 + + # Continually update cache even if there's a "hit" during + # restore to avoid the cache going stale over time + # https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache + - name: Setup cache for bazel + uses: actions/cache@v3 + with: + path: ~/.cache/bazel + key: torch-mlir-bazel-build-cache-${{ runner.os }}-${{ github.sha }} + restore-keys: | + torch-mlir-bazel-build-cache-${{ runner.os }} + + # Change bazel cache directory to root ownership + # to allow writing to it from within the docker container. + # If no cache hits, this directory is not present + # so don't run chown (will error otherwise). + - name: Set bazel cache permissions + run: | + if [ -d "${HOME}/.cache/bazel" ]; then + sudo chown -R root:root "${HOME}/.cache/bazel" + fi + + - name: Build docker image + run: | + docker build -f docker/Dockerfile \ + -t mlir-tcp:ci \ + . + + - name: Bazel build and test torch-mlir + run: | + docker run --rm \ + -v "$(pwd)":"/opt/src/mlir-tcp" \ + -v "${HOME}/.cache/bazel":"/root/.cache/bazel" \ + mlir-tcp:ci \ + bazel test --config=clang_linux @torch-mlir//... + + # Switch back bazel cache directory to user ownership + # to allow GHA post-cache step to save cache without + # permissions issue. + - name: Switch bazel cache permissions + run: | + if [ -d "${HOME}/.cache/bazel" ]; then + sudo chown -R "$USER":"$USER" "${HOME}/.cache/bazel" + fi diff --git a/.gitignore b/.gitignore index 7c15a388..1c5de2d2 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,10 @@ -/bazel-bin -/bazel-out -/bazel-mlir-tcp -/bazel-testlogs +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + +bazel-bin +bazel-out +bazel-mlir-tcp +bazel-testlogs +third_party/ diff --git a/README.md b/README.md index 74f7bea0..174590f7 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ Tensor Compute Primitives Mid-level intermediate representation for machine learning programs. -![Bazel Build](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestTcp.yml/badge.svg) +[![Bazel Build and Test (mlir-tcp)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestTcp.yml/badge.svg)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestTcp.yml) :construction: **This project is under active development (WIP).** @@ -40,3 +40,15 @@ find . -type f -name "*.cpp" -o -name "*.h" | xargs clang-format -i # buildifer bazel run --config=clang_linux //:buildifier ``` + +When bumping upstream dependencies (LLVM, Torch-MLIR, StableHLO), you may validate the set of "green commits" by running the corresponding third-party tests: +```shell +bazel test --config=clang_linux @llvm-project//mlir/... +bazel test --config=clang_linux @torch-mlir//... +bazel test --config=clang_linux @stablehlo//... +``` + +The following CI workflows are automatically triggered anytime upstream dependencies (`deps.bzl`) are updated: +- [![Bazel Build and Test (llvm-project)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestLlvm.yml/badge.svg)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestLlvm.yml) +- [![Bazel Build and Test (torch-mlir)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestTorchmlir.yml/badge.svg)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestTorchmlir.yml) +- [![Bazel Build and Test (stablehlo)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestStablehlo.yml/badge.svg)](https://github.com/cruise-automation/mlir-tcp/actions/workflows/bazelBuildAndTestStablehlo.yml) diff --git a/deps.bzl b/deps.bzl index 66aea289..41d9ba93 100644 --- a/deps.bzl +++ b/deps.bzl @@ -1,34 +1,72 @@ +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +load( + ":local_repos.bzl", + "local_llvm_repo_path", + "local_stablehlo_repo_path", + "local_torch_mlir_repo_path", + "use_local_llvm_repo", + "use_local_stablehlo_repo", + "use_local_torch_mlir_repo", +) def third_party_deps(): - LLVM_COMMIT = "28b27c1b10ae8d1f5b4fb9df691e8cf0da9be3f6" - LLVM_SHA256 = "1f7a7ca5983801d671901644659c32d028e5e7316418fabcb6159454249aefa3" - http_archive( - name = "llvm-raw", - build_file_content = "# empty", - sha256 = LLVM_SHA256, - strip_prefix = "llvm-project-" + LLVM_COMMIT, - urls = ["https://github.com/llvm/llvm-project/archive/{commit}.tar.gz".format(commit = LLVM_COMMIT)], - ) + if use_local_llvm_repo(): + native.new_local_repository( + name = "llvm-raw", + build_file_content = "# empty", + path = local_llvm_repo_path(), + ) + else: + LLVM_COMMIT = "28b27c1b10ae8d1f5b4fb9df691e8cf0da9be3f6" + LLVM_SHA256 = "1f7a7ca5983801d671901644659c32d028e5e7316418fabcb6159454249aefa3" + http_archive( + name = "llvm-raw", + build_file_content = "# empty", + sha256 = LLVM_SHA256, + strip_prefix = "llvm-project-" + LLVM_COMMIT, + urls = ["https://github.com/llvm/llvm-project/archive/{commit}.tar.gz".format(commit = LLVM_COMMIT)], + ) - TORCH_MLIR_COMMIT = "52abae1526e51ae8c415ca98ce4a56b00782b68b" - TORCH_MLIR_SHA256 = "f9973f3519b4ba98475917eb700f447b65fee88e9dd60c61f174ce38335ccb3b" - http_archive( - name = "torch-mlir-raw", - sha256 = TORCH_MLIR_SHA256, - build_file_content = "# empty", - strip_prefix = "torch-mlir-" + TORCH_MLIR_COMMIT, - urls = ["https://github.com/llvm/torch-mlir/archive/{commit}.tar.gz".format(commit = TORCH_MLIR_COMMIT)], - ) + if use_local_torch_mlir_repo(): + native.new_local_repository( + name = "torch-mlir-raw", + build_file_content = "# empty", + path = local_torch_mlir_repo_path(), + ) + else: + TORCH_MLIR_COMMIT = "52abae1526e51ae8c415ca98ce4a56b00782b68b" + TORCH_MLIR_SHA256 = "f9973f3519b4ba98475917eb700f447b65fee88e9dd60c61f174ce38335ccb3b" + http_archive( + name = "torch-mlir-raw", + sha256 = TORCH_MLIR_SHA256, + build_file_content = "# empty", + strip_prefix = "torch-mlir-" + TORCH_MLIR_COMMIT, + urls = ["https://github.com/llvm/torch-mlir/archive/{commit}.tar.gz".format(commit = TORCH_MLIR_COMMIT)], + ) - STABLEHLO_COMMIT = "77a59815a82b34f7b08ed2d42a711d9920682d0e" - STABLEHLO_SHA256 = "367ac567bc9a543ec3c9bbf16e1304a174b1d42bdb7bdeab2ce8b20134ed68d2" - http_archive( - name = "stablehlo", - sha256 = STABLEHLO_SHA256, - strip_prefix = "stablehlo-" + STABLEHLO_COMMIT, - urls = ["https://github.com/openxla/stablehlo/archive/{commit}.tar.gz".format(commit = STABLEHLO_COMMIT)], - ) + if use_local_stablehlo_repo(): + native.local_repository( + name = "stablehlo", + path = local_stablehlo_repo_path(), + ) + else: + STABLEHLO_COMMIT = "5a8bb985f50a679721292b14f97f270344ac64a3" + STABLEHLO_SHA256 = "abda3e8e029c1409b53b1eea080e5cfb4c4ef6705064d7cd954d8272d059567a" + http_archive( + name = "stablehlo", + sha256 = STABLEHLO_SHA256, + strip_prefix = "stablehlo-" + STABLEHLO_COMMIT, + urls = ["https://github.com/openxla/stablehlo/archive/{commit}.tar.gz".format(commit = STABLEHLO_COMMIT)], + # Note: This patch allows testing stablehlo from mlir-tcp + # TODO: Remove after https://github.com/openxla/stablehlo/pull/1810 lands + patches = ["@//:stablehlo.patch"], + patch_args = ["-p1"], + ) SKYLIB_VERSION = "1.3.0" diff --git a/local_repos.bzl b/local_repos.bzl new file mode 100644 index 00000000..31b25847 --- /dev/null +++ b/local_repos.bzl @@ -0,0 +1,36 @@ +# Licensed under the Apache License v2.0 with LLVM Exceptions. +# See https://llvm.org/LICENSE.txt for license information. +# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +# Also available under a BSD-style license. See LICENSE. + +# Modify this file to get bazel to use a local source checkout for the 3p deps +# (e.g. to quickly test out local changes). +# +# To avoid accidentally checking in modifications to this file, you can ask git +# to ignore changes to this file by running the following: +# +# $ git update-index --assume-unchanged local_repos.bzl + +def use_local_llvm_repo(): + # Change this to return True to have mlir-tcp use the source tree at + # `local_llvm_repo_path()` + return False + +def use_local_torch_mlir_repo(): + # Change this to return True to have mlir-tcp use the source tree at + # `local_torch_mlir_repo_path()` + return False + +def use_local_stablehlo_repo(): + # Change this to return True to have mlir-tcp use the source tree at + # `local_stablehlo_repo_path()` + return False + +def local_llvm_repo_path(): + return "./third_party/llvm-project" + +def local_torch_mlir_repo_path(): + return "./third_party/torch-mlir" + +def local_stablehlo_repo_path(): + return "./third_party/stablehlo" diff --git a/stablehlo.patch b/stablehlo.patch new file mode 100644 index 00000000..d1dd3180 --- /dev/null +++ b/stablehlo.patch @@ -0,0 +1,145 @@ +diff --git a/BUILD.bazel b/BUILD.bazel +index 36d1c5a25b..e127c8cc82 100644 +--- a/BUILD.bazel ++++ b/BUILD.bazel +@@ -1311,3 +1311,12 @@ gentbl_cc_library( + ":vhlo_ops_td_files", + ], + ) ++ ++test_suite( ++ name = "all_tests", ++ tests = [ ++ "//stablehlo/tests:stablehlo_tests", ++ "//stablehlo/testdata:stablehlo_data_tests", ++ "//stablehlo/conversions/tosa/tests:stablehlo_tosa_tests" ++ ], ++) +diff --git a/WORKSPACE.bazel b/WORKSPACE.bazel +index 776cd5ca09..f395aa1164 100644 +--- a/WORKSPACE.bazel ++++ b/WORKSPACE.bazel +@@ -13,6 +13,8 @@ + # limitations under the License. + """Workspace for StableHLO.""" + ++workspace(name = "stablehlo") ++ + load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") + + SKYLIB_VERSION = "1.3.0" +diff --git a/stablehlo/conversions/tosa/tests/BUILD.bazel b/stablehlo/conversions/tosa/tests/BUILD.bazel +index 9304c1ad54..cdbc35e111 100644 +--- a/stablehlo/conversions/tosa/tests/BUILD.bazel ++++ b/stablehlo/conversions/tosa/tests/BUILD.bazel +@@ -27,8 +27,8 @@ expand_template( + substitutions = { + "@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.", + "@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"), +- "@STABLEHLO_TOOLS_DIR@": ".", +- "@STABLEHLO_SOURCE_DIR@": ".", ++ "\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], 'stablehlo')", ++ "\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], 'stablehlo')", + }, + template = "lit.site.cfg.py.in", + ) +@@ -45,6 +45,12 @@ expand_template( + "@llvm-project//llvm:FileCheck", + ], + size = "small", ++ tags = ["stablehlo_tosa_tests"], + ) + for src in glob(["**/*.mlir"]) + ] ++ ++test_suite( ++ name = "stablehlo_tosa_tests", ++ tags = ["stablehlo_tosa_tests"], ++) +diff --git a/stablehlo/conversions/tosa/tests/lit.site.cfg.py.in b/stablehlo/conversions/tosa/tests/lit.site.cfg.py.in +index bc65774850..bfa327d5ae 100644 +--- a/stablehlo/conversions/tosa/tests/lit.site.cfg.py.in ++++ b/stablehlo/conversions/tosa/tests/lit.site.cfg.py.in +@@ -17,4 +17,4 @@ import lit.llvm + lit.llvm.initialize(lit_config, config) + config.llvm_tools_dir = "@LLVM_TOOLS_DIR@" + config.stablehlo_tools_dir = "@STABLEHLO_TOOLS_DIR@" +-lit_config.load_config(config, "@STABLEHLO_SOURCE_DIR@/stablehlo/conversions/tosa/tests/lit.cfg.py") ++lit_config.load_config(config, "@STABLEHLO_SOURCE_DIR@" + "/stablehlo/conversions/tosa/tests/lit.cfg.py") +diff --git a/stablehlo/testdata/BUILD.bazel b/stablehlo/testdata/BUILD.bazel +index 1b34e94532..913ca509fd 100644 +--- a/stablehlo/testdata/BUILD.bazel ++++ b/stablehlo/testdata/BUILD.bazel +@@ -27,8 +27,8 @@ expand_template( + substitutions = { + "@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.", + "@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"), +- "@STABLEHLO_TOOLS_DIR@": ".", +- "@STABLEHLO_SOURCE_DIR@": ".", ++ "\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], 'stablehlo')", ++ "\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], 'stablehlo')", + }, + template = "lit.site.cfg.py.in", + ) +@@ -46,6 +46,13 @@ expand_template( + "@llvm-project//llvm:FileCheck", + ], + size = "small", ++ tags = ["stablehlo_data_tests"], ++ + ) + for src in glob(["**/*.mlir"]) + ] ++ ++test_suite( ++ name = "stablehlo_data_tests", ++ tags = ["stablehlo_data_tests"], ++) +diff --git a/stablehlo/testdata/lit.site.cfg.py.in b/stablehlo/testdata/lit.site.cfg.py.in +index b78557087d..d1da209331 100644 +--- a/stablehlo/testdata/lit.site.cfg.py.in ++++ b/stablehlo/testdata/lit.site.cfg.py.in +@@ -18,4 +18,4 @@ import lit.llvm + lit.llvm.initialize(lit_config, config) + config.llvm_tools_dir = "@LLVM_TOOLS_DIR@" + config.stablehlo_tools_dir = "@STABLEHLO_TOOLS_DIR@" +-lit_config.load_config(config, "@STABLEHLO_SOURCE_DIR@/stablehlo/testdata/lit.cfg.py") ++lit_config.load_config(config, "@STABLEHLO_SOURCE_DIR@" + "/stablehlo/testdata/lit.cfg.py") +diff --git a/stablehlo/tests/BUILD.bazel b/stablehlo/tests/BUILD.bazel +index 968627d42e..50b74ce351 100644 +--- a/stablehlo/tests/BUILD.bazel ++++ b/stablehlo/tests/BUILD.bazel +@@ -130,8 +130,8 @@ expand_template( + substitutions = { + "@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.", + "@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"), +- "@STABLEHLO_TOOLS_DIR@": ".", +- "@STABLEHLO_SOURCE_DIR@": ".", ++ "\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], 'stablehlo')", ++ "\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], 'stablehlo')", + }, + template = "lit.site.cfg.py.in", + ) +@@ -150,6 +150,12 @@ expand_template( + "@llvm-project//llvm:not", + ] + glob(["%s.bc" % src]), + size = "small", ++ tags = ["stablehlo_tests"], + ) + for src in glob(["**/*.mlir"]) + ] ++ ++test_suite( ++ name = "stablehlo_tests", ++ tags = ["stablehlo_tests"], ++) +diff --git a/stablehlo/tests/lit.site.cfg.py.in b/stablehlo/tests/lit.site.cfg.py.in +index 6f916bf766..75537da56d 100644 +--- a/stablehlo/tests/lit.site.cfg.py.in ++++ b/stablehlo/tests/lit.site.cfg.py.in +@@ -18,4 +18,4 @@ import lit.llvm + lit.llvm.initialize(lit_config, config) + config.llvm_tools_dir = "@LLVM_TOOLS_DIR@" + config.stablehlo_tools_dir = "@STABLEHLO_TOOLS_DIR@" +-lit_config.load_config(config, "@STABLEHLO_SOURCE_DIR@/stablehlo/tests/lit.cfg.py") ++lit_config.load_config(config, "@STABLEHLO_SOURCE_DIR@" + "/stablehlo/tests/lit.cfg.py") \ No newline at end of file