✨[Feature] Add support for truncate_long_and_double
in Dynamo compile
#1964
Labels
feature request
New feature or request
Story: Dynamo Compile Improvements
Issues relating to improvement of the Dynamo compile path
Context
Certain models, including T5 (see #1864, #1740) require INT64 inputs to their embedding layers. 64-bit input types are not compatible with TRT Engines, however, thus a truncation mechanism is necessary to resolve this issue.
Feature Proposal
Add support for
truncate_long_and_double
in Dynamo compile, tracking data types and automatically casting to TRT-allowed types (and back) as necessary.The text was updated successfully, but these errors were encountered: