Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add enable_dynamic_shapes To Model Config To Resolve CNN Memory Leaks With OpenVino EP #194

Open
narolski opened this issue Jun 2, 2023 · 0 comments

Comments

@narolski
Copy link

narolski commented Jun 2, 2023

Is your feature request related to a problem? Please describe.
For some models (most notably, CNNs), OpenVino EP for ONNX Runtime produces significant memory leaks due to issues with memory de-allocation: openvinotoolkit/openvino#12307.

Describe the solution you'd like
OpenVino team has proposed to use enable_dynamic_shapes to resolve the issue: openvinotoolkit/openvino#12307. However, it is not currently supported by onnxruntime_backend. I'd like to be able to use this option in model config.

Describe alternatives you've considered
Usage of:

parameters { key: "enable_mem_arena" value: { string_value: "0" } }
parameters { key: "enable_mem_pattern" value: { string_value: "0" } }

did not help for OpenVino EP-based inference for onnxruntime_backend.

@narolski narolski changed the title Add enable_dynamic_shapes To Configuration Options Add enable_dynamic_shapes To Model Config To Resolve CNN Memory Leaks With OpenVino EP Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant