[Bugfix] Fix illegal memory access in FP8 MoE kernel #6382
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes the illegal memory access error that frequently happens after #6029. It seems like the following lines cause the problem:
The chunk size is
64*1024
, so in most cases we only have one chunk and this line should have no effect. So I have no idea why this causes the issue. This PR simply avoids this branch when we only have one chunk.cc @robertgshaw2-neuralmagic @mgoin @pcmoritz