PR #16938: Add NANOO FP8 support for collaborative communication unit tests #17265
+138
−85
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![copybara-service[bot]](https://avatars.githubusercontent.com/in/44061?s=60&v=4){kind=small}
copybara-service
bot
force-pushed
the
test_675635116
branch
5 times, most recently
from
b209288
to
c4f1a31
Compare
… tests Imported from GitHub PR #16938 This PR adds support for NANOO FP8 data format in the collaborative communication unit tests. - For the context on OCP FP8 and NANOO FP8, please refer to this comment: google/flax#3993 (comment) - The unit tests in this PR are similar to GEMM unit test introduced in the following PR to be able to deal with both OCP and NANOO fp8 formats: #10488 Copybara import of the project: -- 0fc74cc by Wen Chen <[email protected]>: [AMD] Added NCCL support for fp8e4m3fnuz and fp8e5m2fnuz. -- d247af5 by scxfjiang <[email protected]>: refactor tests for collective comm ops -- 6f8c418 by scxfjiang <[email protected]>: rafactor collective comm e2e tests -- 8ecb6ec by scxfjiang <[email protected]>: update: replace str -- 338d3af by scxfjiang <[email protected]>: get rid of macros Merging this change closes #16938 FUTURE_COPYBARA_INTEGRATE_REVIEW=#16938 from ROCm:ci_dev_rccl_nanoo_fp8 338d3af PiperOrigin-RevId: 675635116
copybara-service
bot
force-pushed
the
test_675635116
branch
from
c4f1a31
to
00f8f62
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR #16938: Add NANOO FP8 support for collaborative communication unit tests
Imported from GitHub PR #16938
This PR adds support for NANOO FP8 data format in the collaborative communication unit tests.
Added support of NANOO fp8. google/flax#3993 (comment)
PR #9531: Fp8 matmul support on AMD MI300 #10488
Copybara import of the project:
--
0fc74cc by Wen Chen [email protected]:
[AMD] Added NCCL support for fp8e4m3fnuz and fp8e5m2fnuz.
--
d247af5 by scxfjiang [email protected]:
refactor tests for collective comm ops
--
6f8c418 by scxfjiang [email protected]:
rafactor collective comm e2e tests
--
8ecb6ec by scxfjiang [email protected]:
update: replace str
--
338d3af by scxfjiang [email protected]:
get rid of macros
Merging this change closes #16938
FUTURE_COPYBARA_INTEGRATE_REVIEW=#16938 from ROCm:ci_dev_rccl_nanoo_fp8 338d3af