Edit mixed precision pages in feature guide. #3755
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Dave Welsch <[email protected]>
dwelsch-esi
commented
Jan 28, 2025
| Prerequisites | ||
| ------------- | ||
|
|
||
| Manual mixed precision is supported only on PyTorch models. |
There was a problem hiding this comment.
Is this true? I assumed it is because I didn't see APIs for TF or ONNX.
dwelsch-esi
commented
Jan 28, 2025
| * Change the precision of all the layers in the model of a certain type | ||
| * Change the precision of model input tensors (or only a subset of input tensors) | ||
| * Change the precision of model output tensors (or only a subset of output tensors) | ||
| * A leaf layer |
There was a problem hiding this comment.
How is a leaf layer defined?
dwelsch-esi
commented
Jan 28, 2025
| Not supported. | ||
|
|
||
|
|
||
| The ``apply`` call generates a report detailing how the request was inferred, propagated to other layers, and eventually realized. |
There was a problem hiding this comment.
Where is the report saved?
dwelsch-esi
commented
Jan 28, 2025
| Layer Groups are defined as a group of layers grouped together based on certain rules. | ||
| This helps in reducing search space over which the mixed precision algorithm operates. | ||
| It also ensures that we search only over the valid bit-width settings for parameters and activations. | ||
| Layer Groups are defined based on certain rules. |
There was a problem hiding this comment.
What are the rules and how are they defined?
dwelsch-esi
commented
Jan 28, 2025
|
|
||
| .. image:: ../../images/pareto.png | ||
| :width: 900px | ||
| An example of a Pareto list: |
There was a problem hiding this comment.
How is this list different from the accuracy list in the previous phase?
dwelsch-esi
commented
Jan 28, 2025
|
|
||
| Use Cases | ||
| ========= | ||
| Conversion operations (convert ops) are introduced in the mixed-precision model for transition between ops with different activation bit widths or data types (float vs int). Convert ops contribute to the inference time along with bit-operations of ops. |
There was a problem hiding this comment.
I assume that 'convert op' is jargon for 'conversion operation'. Is that the case?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.