Constant literals are duplicated during the compilation #87
sergeykozub
started this conversation in
xla
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Constant literals are duplicated during the compilation, this may be in issue if they're large.
Currently, the compilation pipeline creates at least 7 copies of constant literals:
This results in high RAM usage during the compilation. Some of those copies are persisted in the caches.
Note that the protobuf implementation doesn't allow sharing the underlying data, so at least 3 copies of the constants will be alive at the same time (points 1-3 in the list above).
Despite the above, we could still somewhat reduce the RAM usage during the XLA compilation by removing unnecessary copies - i.e. by putting the literal data behind a std::shared_ptr (cannot do this for HloModuleProto, unfortunately).
Beta Was this translation helpful? Give feedback.
All reactions