-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rockchip: update dependencies and add script for model conversion #15699
base: 0.16
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for frigate-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
I think it would be best to target 0.16 with these changes, let me know what you think |
I agree |
Are there any limitations on what model architectures can be converted to rknn format? |
On AArch64 only onnx to rknn is supported. This might change in the future. On x86 other formats are available as well. |
Sorry I mean are there any restrictions for example only yolonas, or also mobilenet and others? |
Sorry, I misunderstood. In theory one can convert any model. Practically it's not a straightforward process. Not all onnx operations are supported. For example, the default yolo-nas model includes post-processing and performes a batched non maximum suppression. The rknn-toolkit2 doesn't support this, so I had to modify the model and do the NMS programmatically in python. |
Proposed change
This PR updates the following rockchip dependencies:
Moreover, the image supports converting models from onnx to rknn format now. This should be useful for Frigate+ users who want to use their own models. Changes:
conv2rknn.py
was addedrknn-toolkit2
was installedType of change
Additional information
Some notes about yolonas:
It seems the super-gradients project is not active anymore after Deci AI was bought by Nvidia. The current pypi release of super-gradients is broken because the download URL changed (see Deci-AI/super-gradients#2057). There are a couple of fixes:
~/.cache/torch/hub/checkpoints/
./.cache/torch/hub/checkpoints/This is relevant for users who try to use this notebook from the frigate repo:
Checklist
ruff format frigate
)