Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracker] All the issue related with ONNX model zoo models #886

Open
pdhirajkumarprasad opened this issue Nov 20, 2024 · 0 comments
Open

Comments

@pdhirajkumarprasad
Copy link

pdhirajkumarprasad commented Nov 20, 2024

We have enabled total 2325 models from ONNX model zoo. This issue is to track all the issues seen in the models.

Passing Summary

TOTAL TESTS = 2325

Stage # Passing % of Total % of Attempted
Setup 2311 99.4% 99.4%
IREE Compilation 2024 87.1% 87.6%
Gold Inference 2024 87.1% 100.0%
IREE Inference Invocation 2015 86.7% 99.6%
Inference Comparison (PASS) 1959 84.3% 97.2%

Fail Summary

TOTAL TESTS = 2325

Stage # Failed at Stage % of Total
Setup 14 0.6%
IREE Compilation 287 12.3%
Gold Inference 0 0.0%
IREE Inference Invocation 9 0.4%
Inference Comparison 56 2.4%

CPU

# issue type issue no #model impacted list of model assignee status
1 failed to legalize operation 'torch.operator' that was explicitly marked illegal 62 NonMaxSuppression, MatMulInteger, Scan, Conv, Loop, QLinearConv, DequantizeLinear, AveragePool, QLinearMatMul
2 One or more operations with large vector sizes (16384 bytes) were found 139
3 expected offsets to be non-negative, but got -1 19501 6
4 'vector.transfer_write' op inferred mask type ('vector<1x1x8xi1>') and mask operand type ('vector<1x8x1xi1>') don't match 19383 9
5 failed to legalize operation 'torch.aten.nonzero' llvm/torch-mlir#3876 8
6 'tensor.reshape' op source and destination tensor should have the same number of elements 3
7 'linalg.generic' op inferred input/output operand #0 has shape's dimension #2 to be greater than or equal to 4, but found 1 5
8 failed to legalize operation 'torch.aten.convolution' that was explicitly marked illegal 3
9 expected type to be 'tensor<?x?x?xf32>' or a rank-reduced version. (size mismatch) 3
10 'tensor.cast' op operand type 'tensor<80x15130xf32>' and result type 'tensor<15130xf32>' are cast incompatible 891 3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant