-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug-Tracker MPS issues #154
Comments
I wouldn't dismiss the last errors (with Does enabling MPS fallback let you run regular calls to |
I have not tested this any further yet, but I thought that it is strange that these errors in the tests only occur when MPS is enabled and neither on CUDA or CPU, so I feel like there might still be something wrong with the backend here. I will test a bit more once I get my new MacBook in a few days (since then I will be able test on both M1 and AMD GPU for a few days before I return my old one). |
After some more testing I believe it is certainly better to stay away from MPS for now until it is more mature. E.g. just running this basic setup model = inseq.load_model("gpt2", "saliency")
out = model.attribute(
"The developer argued",
generation_args={"max_new_tokens": 10},
) with MPS gives the generation output of: Considering that the first one is basically only empty tokens that get generated it certainly explains the different shapes I got in the error messages above. |
Thanks for testing this @lsickert! Maybe worth raising an issue on HF transformers to understand where's the issue since it's clearly a problem with generation. Would you be willing to take the matter into your hands? |
Yes sure, I can raise it with them, although I am fairly confident that it is probably an underlying PyTorch issue since I managed to trace it back to a place in the gpt2-code where transformers starts calling some tensor methods. But to be sure I will have to investigate a bit more. |
🐛 Bug Report
Even after updating to the newest pytorch version 1.13.1 several issues with the mps-backend still remain when it is enabled in the code. There still seems to be some inconsistency across the different devices depending on the operations that are run, as can be seen below.
The goal of this issue is primarily to collect and highlight these problems.
🔬 How To Reproduce
Steps to reproduce the behavior:
inseq/utils/torch_utils
and changecpu
tomps
in line 229 to enable the mps-backendmake fast-test
to run the testsCode sample
see above
Environment
Screenshots
Running the tests this way generates the following error report:
When run with the environment variable
PYTORCH_ENABLE_MPS_FALLBACK=1
set, the following errors are still occuring:These errors do not occur when running the tests on other backends, implying that there is still some inconsistency between mps and the other torch backends.
📈 Expected behavior
All tests should run consistently across all torch backends.
📎 Additional context
The text was updated successfully, but these errors were encountered: