[Bug]: Custom Yolov4 Tiny Model Won't Compile on NPU Driver >= 32.0.100.2408

[michaelmalice]

OpenVINO Version

2024.5

Operating System

Windows System

Device used for inference

NPU

Framework

ONNX

Model used

Yolov4 Tiny Custom

Issue description

Error when compiling model on NPU when the driver version is greater than or equal to 32.0.100.2408. Model compiles on Core Ultra Series 1 NPU on drivers before 32.0.100.2408. Model will not compile on any of the Core Ultra Series 2 drivers.

Step-by-step reproduction

yolov4-tiny-custom.zip

import openvino as ov
core = ov.Core()
model = core.read_model("./yolov4-tiny-custom.xml")
compiled_model = core.compile_model(model, device_name="NPU")

Relevant log output

MTL (Core Ultra Series 1)

Driver Version: 31.0.100.1688
Model compiles

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import openvino as ov
>>> core = ov.Core()
>>> model = core.read_model("./model/object-detection/yolov4-tiny-9-classes-mixed-none-416-v6/yolov4-tiny-custom.xml")
>>> compiled_model = core.compile_model(model, device_name="NPU")
>>>



Driver Version: 31.0.100.2016
Model compiles

Driver Version: 32.0.100.2196
Model compiles

Driver Version: 32.0.100.2267
Model compiles

Driver Version: 32.0.100.2381
Model compiles

Driver Version: 32.0.100.2408
Model failed to compile

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import openvino as ov
>>> core = ov.Core()
>>> model = core.read_model("./yolov4-tiny-custom.xml")
>>> compiled_model = core.compile_model(model, device_name="NPU")
error: ConvertShapeTo4D Pass failed : The dims from range [0, origRank - TARGET_TENSOR_DIM] are not equal to 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\tme\Desktop\object-recognition\venv-t3\lib\site-packages\openvino\runtime\ie_api.py", line 543, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src\inference\src\cpp\core.cpp:107:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:711:
Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:389:
L0 pfnCreate2 result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . Compilation failed
Failed to create executable



LNL

Driver Version: 32.0.100.3053
Model failed to compile

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import openvino as ov
>>> core = ov.Core()
>>> model = core.read_model("./yolov4-tiny-custom.xml")
>>> compiled_model = core.compile_model(model, device_name="NPU")
[ERROR] 14:21:03.040 [vpux-compiler] Got Diagnostic at loc(fused<{name = "main", type = "Func"}>["main"]) : ConvertShapeTo4D Pass failed : The dims from range [0, origRank - TARGET_TENSOR_DIM] are not equal to 1
loc(fused<{name = "main", type = "Func"}>["main"]): error: ConvertShapeTo4D Pass failed : The dims from range [0, origRank - TARGET_TENSOR_DIM] are not equal to 1
[ERROR] 14:21:03.042 [vpux-compiler] Failed Pass ConvertShapeTo4D on Operation loc(fused<{name = "main", type = "Func"}>["main"])
[ERROR] 14:21:03.042 [vpux-compiler] Failed Pass mlir::detail::OpToOpPassAdaptor on Operation loc(fused<{name = "module", type = "Module"}>["module"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\TME\Desktop\object-recognition-20241122\venv-t3\lib\site-packages\openvino\runtime\ie_api.py", line 543, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src\inference\src\cpp\core.cpp:107:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:711:
Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:389:
L0 pfnCreate2 result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . Compilation failed
Failed to create executable

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[avitial]

@michaelmalice thanks for reaching out, I am able to see the error on MTL using the latest available driver. Captured this as a bug, will share more details as soon as we have an update.

Ref. 156655

[bibekyess]

Hi @avitial!
I am also facing the same issue in:
Intel(R) AI Boost, Driver version: 32.0.100.3104, DirectX Version: 12 (FL 1.0: Compute)

My model is not the same as this issue, but the error message is exactly same. The same openvino model can be offloaded well in Intel(R) Arc(TM) Graphics iGPU but not on the NPU.
Thanks!

[avitial]

@bibekyess can you share more details about the model you are using? Perhaps a link to the public repo or share with us the actual model.

[bibekyess]

Hi @avitial Thank you for your message.
I am using this CoDetr model with SwinL backbone.
Reference: https://github.com/Sense-X/Co-DETR