[PyTorch] Fix autocast deprecation warnings #1277
Open
+78
−34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yaox12
force-pushed
the
xiny/fix_autocast_warning
branch
2 times, most recently
from
1420ea6
to
0987833
Compare
|
/te-ci pytorch |
yaox12
changed the title
yaox12
force-pushed
the
xiny/fix_autocast_warning
branch
from
35d7a31
to
a2212c0
Compare
|
/te-ci pytorch |
ksivaman
reviewed
Oct 22, 2024
ksivaman
reviewed
Oct 22, 2024
yaox12
force-pushed
the
xiny/fix_autocast_warning
branch
from
35ca876
to
39a9fe6
Compare
|
/te-ci pytorch |
|
/te-ci pytorch |
ksivaman
reviewed
Nov 4, 2024
ksivaman
reviewed
Nov 4, 2024
transformer_engine/pytorch/utils.py
Outdated
| """Get PyTorch autocast GPU dtype.""" | ||
| TORCH_MAJOR = int(torch.__version__.split(".")[0]) | ||
| TORCH_MINOR = int(torch.__version__.split(".")[1]) | ||
| if TORCH_MAJOR == 2 and TORCH_MINOR >= 4: |
There was a problem hiding this comment.
Could we do a simpler >= "2.4" similar to the one currently in jit.py. This method isn't future proof and will break for 3+
There was a problem hiding this comment.
I'm good with it.
Furthermore, since we now have packaging as a dependency, how about using packaging.version for a more robust version comparison?
ksivaman
reviewed
Nov 4, 2024
Signed-off-by: Xin Yao <[email protected]>
Signed-off-by: Xin Yao <[email protected]>
Signed-off-by: Xin Yao <[email protected]>
Signed-off-by: Xin Yao <[email protected]>
Signed-off-by: Xin Yao <[email protected]>
yaox12
force-pushed
the
xiny/fix_autocast_warning
branch
from
f43bdad
to
6ba412f
Compare
Signed-off-by: Xin Yao <[email protected]>
yaox12
force-pushed
the
xiny/fix_autocast_warning
branch
from
6ba412f
to
30f8cae
Compare
yaox12
commented
Nov 5, 2024
| TORCH_MAJOR = int(torch.__version__.split(".")[0]) | ||
| TORCH_MINOR = int(torch.__version__.split(".")[1]) | ||
| use_nvfuser = TORCH_MAJOR > 1 or (TORCH_MAJOR == 1 and TORCH_MINOR >= 10) | ||
| use_nvfuser = is_torch_min_version("1.10.0a0") and not is_torch_min_version("2.2.0a0") |
There was a problem hiding this comment.
We stop using nvfuser starting from torch 2.2, so I think this update is needed. Tell me if you have other concerns.
Signed-off-by: Xin Yao <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix
autocastdeprecation warnings.Starting from PyTorch 2.4, use
instead of device-specific APIs.
Closes #1167.
Type of change
Checklist: