Initial import of Jasc. #790
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR imports Jasc, an MLIR-based compiler for JAX kernels, to the
repo. Most functionality works, but some doesn't, and some shortcuts
have been taken. In particular:
* The GPU tests fail, and I haven't investigated why beyond the fact
that it can't run on my machine without GPU. Maybe we need a better
mechanism to choose the backend before we can make GPUs work in OSS.
* I have completely removed support for sparse and the related tests.
* I have removed several tools that aren't supported in OSS.
* The BUILD files are quite hacky and need to be cleaned up. They
currently contain remains of two different approaches I tried, one
with several shared object files and one with a single one. The latter
is the one that is currently used but I *think* that the former can
also work; I just gave up that approach at some point due to a weird
error only to find out that the same error also occurs with the other
approach and is unrelated.
* I have also not made any attempt yet to reduce the diff between this
repository and our internal version but there are a few easy things we
could do to make potential future syncs easier.
* I have not added copyright and license information yet.
What does work, though, is to compile everything and run the tests (some
of which fail):
bazel build //...
bazel test test:*
ftynse
reviewed
Dec 21, 2023
I am using version 6.4.0 because it is the latest current version below 7.0.0, plus it is LTS. Several other minor versions of version 6 have worked as well. Bazel 7.0.0 does not work, probably due to some incompatibility in one of the many dependencies. At least one dependency requires Bazel 5.4.0 or higher.
Otherwise, for some weird reason, the file is expected in `external/`, and putting it there may collide with some Bazel internals.
This sets up the following command, which creates/updates the
`compile_commands.json` file, which is useful for auto-completion and
other tools:
bazel run @hedron_compile_commands//:refresh_all
I tried half a day to make CUDA integrate with bazel -- in vain. We can try again in some future iteration.
The reductions are related to: * abseil's VLOG, * LLVM's dropping of opaque pointers, * random new lines.
I put all new constructs into question again. Turns out that many were not needed and some were even plain unused.
The patches still added things to several dependencies that are actually not needed. This commit removes those. I also created a fresh patch that "squashes" several `diff` sections to the same file into a single section (i.e., I recreated the patch files from scratch).
ingomueller-net
commented
Jan 19, 2024
There was a problem hiding this comment.
@ftynse: Thanks a lot for the helpful comments! I think I addressed them all, plus a few of the TODOs I had listed, plus some general clean-up. Please give it another look.
Edit: I have "resolved" all conversation but you may want to look at some of my comments (and re-open if you aren't happy).
ftynse
approved these changes
Feb 8, 2024
|
Sorry, I forgot about this change. Don't hesitate to ping me by email when I do. |
Thanks for the final round! Addressed and merged :) No worries! I was not in hurry with this PR and had other things on my mind as well... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR imports Jasc, an MLIR-based compiler for JAX kernels, to the repo. Most functionality works, but some doesn't, and some shortcuts have been taken. In particular:
They currently contain remains of two different approaches I tried, one with several shared object files and one with a single one. The latter is the one that is currently used but I think that the former can also work; I just gave up that approach at some point due to a weird error only to find out that the same error also occurs with the other approach and is unrelated.(Update: They BUILD files are still extremely hacky but they are now the closest to minimal I was able to achieve.)I have not added copyright and license information yet.(Update: License information added.)What does work, though, is to compile everything and run the tests (some of which
failare skipped):