Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix quantization for onnxruntime v1.16.0 #1405

Merged
merged 6 commits into from
Sep 21, 2023
Merged

fix quantization for onnxruntime v1.16.0 #1405

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3bea1a2
fix quantization for onnxruntime v1.16.0
echarlaix Sep 21, 2023
b343ce4
Update optimum/onnxruntime/quantization.py
echarlaix Sep 21, 2023
ac550be
Update optimum/onnxruntime/quantization.py
echarlaix Sep 21, 2023
4d6880e
Update optimum/onnxruntime/quantization.py
echarlaix Sep 21, 2023
b20ffa8
skip test for ort v1.16.0
echarlaix Sep 21, 2023
590f993
skip test
echarlaix Sep 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions optimum/onnxruntime/quantization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,10 @@ def compute_ranges(self) -> Dict[str, Tuple[float, float]]:
)

LOGGER.info("Computing calibration ranges")

if parse(ort_version) >= Version("1.16.0"):
return self._calibrator.compute_data()

return self._calibrator.compute_range()

def quantize(
Expand Down Expand Up @@ -351,8 +355,13 @@ def quantize(
has_subgraphs = True
break

if quantization_config.is_static and has_subgraphs:
raise NotImplementedError("Static quantization is currently not supported for models with" " subgraphs.")
if has_subgraphs:
if quantization_config.is_static:
raise NotImplementedError("Static quantization is currently not supported for models with subgraphs.")
if parse(ort_version) == Version("1.16.0"):
raise ValueError(
"ONNX Runtime version v1.16.0 not compatible with quantization for models with subgraphs, please downgrade to 1.15.1 or upgrade to a higher version. Reference: https://github.com/microsoft/onnxruntime/pull/17651"
echarlaix marked this conversation as resolved.
Show resolved Hide resolved
)

quantizer_factory = QDQQuantizer if use_qdq else ONNXQuantizer

Expand Down
Loading