Remove calls to deprecated `Tensor.storage()` when using newer PyTorch versions #1230

mrfh92 · 2023-10-06T13:18:34Z

Intended to resolve issue #1229

… for the best!

mrfh92 · 2023-10-06T13:19:14Z

Result of the first try: all test run through expect for

======================================================================
FAIL: test_stride_and_strides (heat.core.tests.test_dndarray.TestDNDarray)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/hopp_fa/heat/heat/core/tests/test_dndarray.py", line 1554, in test_stride_and_strides
    self.assertEqual(heat_int16.strides, numpy_int16.strides)
AssertionError: Tuples differ: (2100, 420, 140, 35, 7, 1) != (4200, 840, 280, 70, 14, 2)

First differing element 0:
2100
4200

- (2100, 420, 140, 35, 7, 1)
+ (4200, 840, 280, 70, 14, 2)

----------------------------------------------------------------------
Ran 428 tests in 47.663s

FAILED (failures=1, errors=2, skipped=5)
ok

ghost · 2023-10-06T13:20:23Z

👇 Click on the image for a new way to code review

Legend

…eprecated

mrfh92 · 2023-10-06T13:22:45Z

merged main (although this is labeled as "bug"-PR) because it is actually not a bug-fix but rather a reaction to some future development in PyTorch and therefore this PR can wait til next release to become "proper" part of Heat

mrfh92 · 2023-10-06T14:46:49Z

The first idea for a workaround mentioned above that resulted in errors only in test_stride_and_strides does not seem to work for certain earlier PyTorch versions. I have tested with torch==2.0.0 on my worksation (result above), but torch==1.8.1 (also on my workstation) and torch==1.12.0 (on HDFML) which resulted in multiple errors over many unittests. When I use torch==2.0.0 on HDFML, I only get errors in test_stride_and_strides again...

TO BE DISCUSSED: Are we going to stay compatible with these earlier PyTorch versions (which potentially means additional work in this PR) or do we plan to jump to torch==2.0.0 with the next Heat-release anyway?

btw: this also is a problem because our AMD-CI runs with a quite old PyTorch version that could not be used anymore...

* test_stride_and_strides * MinMaxScaler Moreover, I have introduced a check whether there are at least two nodes available for the DASO test (that was always failing for 8 processes on a single GPU...)

github-actions · 2023-10-06T15:30:44Z

Thank you for the PR!

mrfh92 · 2023-10-06T15:40:58Z

Still a problem:

/p/project/haf/users/hoppe6/heat/heat/optim/dp_optimizer.py:23: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  tens_a = torch.HalfTensor().set_(torch.HalfStorage.from_buffer(buffer_a, "native"))

mrfh92 · 2023-10-09T08:26:35Z

Decision in PR meeting: introduce version check to ensure backward compatibility

…eprecated

github-actions · 2023-10-09T09:24:46Z

Thank you for the PR!

moreover: introduced decorator for DASO tests (skip if nodes < 2 or no GPUs)

github-actions · 2023-10-09T12:41:02Z

Thank you for the PR!

…at pytorch version 2)

github-actions · 2023-10-09T13:34:33Z

Thank you for the PR!

github-actions · 2023-10-09T13:56:41Z

Thank you for the PR!

codecov · 2023-10-09T14:04:06Z

Codecov Report

Merging #1230 (94e2140) into main (a32efdb) will decrease coverage by 0.58%.
The diff coverage is 32.35%.

@@            Coverage Diff             @@
##             main    #1230      +/-   ##
==========================================
- Coverage   92.32%   91.75%   -0.58%     
==========================================
  Files          77       77              
  Lines       11056    11080      +24     
==========================================
- Hits        10207    10166      -41     
- Misses        849      914      +65

Flag	Coverage Δ
unit	`91.75% <32.35%> (-0.58%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
heat/core/factories.py	`97.10% <ø> (ø)`
heat/core/dndarray.py	`96.97% <50.00%> (-0.24%)`	⬇️
heat/core/memory.py	`91.17% <50.00%> (-5.60%)`	⬇️
heat/core/manipulations.py	`98.61% <50.00%> (-0.22%)`	⬇️
heat/core/communication.py	`95.28% <50.00%> (-0.66%)`	⬇️
heat/optim/dp_optimizer.py	`13.43% <0.00%> (-10.93%)`	⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

mrfh92 · 2023-10-09T14:28:28Z

Current workaround works on HDFML with torch==1.12 and torch==2.0.0 execpt for the following problems:

/p/project/haf/users/hoppe6/heat/heat/optim/dp_optimizer.py:23: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  tens_a = torch.HalfTensor().set_(torch.HalfStorage.from_buffer(buffer_a, "native"))

and similar for lines 24, 33, and 34 of the same file.

Problem: This is related to the float16/half data types used in DASO, in particular to the creation of a custom MPI-op

mrfh92 · 2023-10-09T14:30:04Z

Comment on decreased codecov:
This is mainly due to the fact that I now skipp all DASO-related tests as long as the hardware is inappropriate (i.e., no GPUs or less than 2 nodes), which is the case for our CI-runners. Hence, DASO is not covered anymore.

github-actions · 2023-10-09T14:52:15Z

Thank you for the PR!

github-actions · 2023-10-09T15:04:29Z

Thank you for the PR!

mrfh92 · 2023-10-09T15:12:17Z

Workaround for DASO has successfully been tested on HDFML (2 nodes, 4 GPUs each) with PyTorch==1.12.0 and PyTorch==2.0.0.

github-actions · 2023-10-09T15:13:57Z

Thank you for the PR!

mrfh92 · 2023-10-09T15:15:22Z

Summary:

I have replaced typed storage by untyped storage wherever the deprecation warning appeared. To ensure backward compatibility (untyped storage seems to have been introduced in PyTorch==2.0.0), I use try/except (in the code) and version-checks in the unittests.
DASO-tests are now completely skipped whenever the hardware is inappropriate (no GPUs, less than 2 nodes). Thus, codecov is reduced compared to main, where e.g. wrong input data types for DASO have been tested even on a single CPU-process.
a tolerance in the preprocessing tests has been adapted to avoid occasional random failure due to too tight tolerances

…eprecated

heat/core/version.py

ClaudiaComito

Great @mrfh92 thanks a lot for fixing this. I only have one potential change in the version.py file since we are merging into main.

Thanks a lot!

heat/core/version.py

github-actions · 2023-10-13T09:52:57Z

Thank you for the PR!

Co-authored-by: Claudia Comito <[email protected]>

mrfh92 · 2023-10-13T10:05:50Z

@ClaudiaComito yes, youre right... since it is not directly a bug fix, but rather a hurry-ahead-bug-fix, I would merge into main

github-actions · 2023-10-13T10:10:05Z

Thank you for the PR!

ClaudiaComito

Awesome!

ClaudiaComito and others added 3 commits June 20, 2023 15:22

update version to 1.3.0

c50c1d2

Create .readthedocs.yaml (#1187)

4338622

first try: replaced .storage() by .untyped_storage()... lets hope…

ba012fb

… for the best!

Merge branch 'main' into bugs/1229-_Bug_UserWarning_TypedStorage_is_d…

234c9e9

…eprecated

mrfh92 added interoperability To be discussed Requires discussion in project meeting first bug Something isn't working types labels Oct 6, 2023

removed unittests that were failing after the changes:

16b7b3d

* test_stride_and_strides * MinMaxScaler Moreover, I have introduced a check whether there are at least two nodes available for the DASO test (that was always failing for 8 processes on a single GPU...)

mrfh92 self-assigned this Oct 9, 2023

Merge branch 'main' into bugs/1229-_Bug_UserWarning_TypedStorage_is_d…

bf3c722

…eprecated

adapted unit test_stride_and_strides to match our previours changes

b7d01c5

moreover: introduced decorator for DASO tests (skip if nodes < 2 or no GPUs)

adapted unittests for backward compatibility (untyped storage starts …

6213f1a

…at pytorch version 2)

smaller changes

f30a5ff

experiment in DASO code to replace TypedStorages there...

b8fe979

...

9ba04c1

hopefully fixed also the problem in DASO

c9b2588

mrfh92 marked this pull request as ready for review October 9, 2023 15:17

mrfh92 requested review from ClaudiaComito and mtar October 9, 2023 16:52

Merge branch 'main' into bugs/1229-_Bug_UserWarning_TypedStorage_is_d…

7a21da5

…eprecated

ClaudiaComito added the merge queue label Oct 13, 2023

ClaudiaComito reviewed Oct 13, 2023

View reviewed changes

heat/core/version.py Outdated Show resolved Hide resolved

ClaudiaComito requested changes Oct 13, 2023

View reviewed changes

heat/core/version.py Outdated Show resolved Hide resolved

ClaudiaComito changed the title ~~Bugs/1229 bug user warning typed storage is deprecated~~ Remove calls to deprecated Tensor.storage() when using newer PyTorch versions Oct 13, 2023

Update heat/core/version.py

94e2140

Co-authored-by: Claudia Comito <[email protected]>

ClaudiaComito approved these changes Oct 13, 2023

View reviewed changes

mrfh92 merged commit 5beb254 into main Oct 13, 2023

mrfh92 deleted the bugs/1229-_Bug_UserWarning_TypedStorage_is_deprecated branch October 13, 2023 10:45

mrfh92 mentioned this pull request Oct 13, 2023

[Bug]: UserWarning: TypedStorage is deprecated. #1229

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove calls to deprecated `Tensor.storage()` when using newer PyTorch versions #1230

Remove calls to deprecated `Tensor.storage()` when using newer PyTorch versions #1230

mrfh92 commented Oct 6, 2023

mrfh92 commented Oct 6, 2023

ghost commented Oct 6, 2023 •

edited by ghost

Loading

Legend

mrfh92 commented Oct 6, 2023

mrfh92 commented Oct 6, 2023 •

edited

Loading

github-actions bot commented Oct 6, 2023

mrfh92 commented Oct 6, 2023

mrfh92 commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

codecov bot commented Oct 9, 2023 •

edited

Loading

mrfh92 commented Oct 9, 2023 •

edited

Loading

mrfh92 commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

mrfh92 commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

mrfh92 commented Oct 9, 2023 •

edited

Loading

ClaudiaComito left a comment

github-actions bot commented Oct 13, 2023

mrfh92 commented Oct 13, 2023

github-actions bot commented Oct 13, 2023

ClaudiaComito left a comment

Remove calls to deprecated Tensor.storage() when using newer PyTorch versions #1230

Remove calls to deprecated Tensor.storage() when using newer PyTorch versions #1230

Conversation

mrfh92 commented Oct 6, 2023

mrfh92 commented Oct 6, 2023

ghost commented Oct 6, 2023 • edited by ghost Loading

Legend

mrfh92 commented Oct 6, 2023

mrfh92 commented Oct 6, 2023 • edited Loading

github-actions bot commented Oct 6, 2023

mrfh92 commented Oct 6, 2023

mrfh92 commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

codecov bot commented Oct 9, 2023 • edited Loading

Codecov Report

mrfh92 commented Oct 9, 2023 • edited Loading

mrfh92 commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

mrfh92 commented Oct 9, 2023

github-actions bot commented Oct 9, 2023

mrfh92 commented Oct 9, 2023 • edited Loading

ClaudiaComito left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 13, 2023

mrfh92 commented Oct 13, 2023

github-actions bot commented Oct 13, 2023

ClaudiaComito left a comment

Choose a reason for hiding this comment

Remove calls to deprecated `Tensor.storage()` when using newer PyTorch versions #1230

Remove calls to deprecated `Tensor.storage()` when using newer PyTorch versions #1230

ghost commented Oct 6, 2023 •

edited by ghost

Loading

mrfh92 commented Oct 6, 2023 •

edited

Loading

codecov bot commented Oct 9, 2023 •

edited

Loading

mrfh92 commented Oct 9, 2023 •

edited

Loading

mrfh92 commented Oct 9, 2023 •

edited

Loading