[OVEP] Add a check for type mismatches in QDQ stripping #834
Conversation
|
The casting node is for QuantizeLinear node correct? Could you add the before and after graph after this casting? Will it introduce any performance penalty? |
|
The cast is introduced when rewiring the graph to remove the QuantizeLinear and DequantizeLinear nodes, so that instead of a direct edge from e.g. a f32 input to a f16 output there's a Cast node added between them. The before and after are: |
onnxruntime/core/providers/openvino/qdq_transformations/qdq_scales_fix.cpp
Outdated
Show resolved
Hide resolved
|
The issue that causes the use of a C-style cast for the input arg is that it is provided as const by |
| type_str_to_tensor_data_type_["tensor(uint64)"] = ONNX_NAMESPACE::TensorProto_DataType_UINT64; | ||
| type_str_to_tensor_data_type_["tensor(complex64)"] = ONNX_NAMESPACE::TensorProto_DataType_COMPLEX64; | ||
| type_str_to_tensor_data_type_["tensor(complex128)"] = ONNX_NAMESPACE::TensorProto_DataType_COMPLEX128; | ||
| type_str_to_tensor_data_type_["tensor(string)"] = ONNX_NAMESPACE::TensorProto_DataType_STRING; |
There was a problem hiding this comment.
Does it really make sense to add a possibility to convert fp32->string or to complex types, especially in the context of QDQ stripping?
Anyway, since QuantizeLinear and DequantizeLinear aren't operations per se, but are rather metaoperations, the fact that their input and output types differ seems more like an export bug to me. Let's cover that case only, but with an additional assertion to be on the safe side.
|
Could you open the PR to get review from reviewers with write access? Please also add the Jira ticket. |
When rewiring the graph after eliminating QDQ pairs, the runtime now checks whether the type matches before and after the eliminated nodes and inserts a Cast node if there is a mismatch.
mdvoretc-intel
force-pushed
the
qdq_type_fix
branch
from
19f5230 to
69f09bb
Compare
3 participants
Description
When rewiring the graph after eliminating QDQ pairs, the runtime now checks whether the type matches before and after the eliminated nodes and inserts a Cast node if there is a mismatch.
Motivation and Context
At present, QDQ elimination assumes the floating point type is the same before the QuantizeLinear node and after the following DequantizeLinear, producing errors if the types mismatch.
Jira Ticket :
CVS-175447