[Detector Support]: Yolov8 use in 0.14 Beta 3

[stevenbrown1862]

Describe the problem you are having

I've been using 0.13 with the Yolov8n model with success. Trying to move across to 0.14 to try out the new UI and having trouble using the same model. The following config (extract) gets Frigate to start but after a few seconds it exits seemingly blaming detection (sorry about the lack of indentation in the paste, rest assured it's indented correctly and I just can't use github).

detectors:
ov:
type: openvino
device: AUTO

model:
width: 416
height: 416
input_tensor: nchw
input_pixel_format: bgr
model_type: yolox
path: /config/yolov8n.xml
labelmap_path: /config/coco_80cl.txt

The following part of the log seems relevant:

2024-06-17 09:38:42.881536981 Process detector:ov:
2024-06-17 09:38:42.883308687 Traceback (most recent call last):
2024-06-17 09:38:42.883374317 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-06-17 09:38:42.883377573 self.run()
2024-06-17 09:38:42.883412652 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-06-17 09:38:42.883415225 self._target(*self._args, **self._kwargs)
2024-06-17 09:38:42.883443932 File "/opt/frigate/frigate/object_detection.py", line 125, in run_detector
2024-06-17 09:38:42.883446712 detections = object_detector.detect_raw(input_frame)
2024-06-17 09:38:42.883486769 File "/opt/frigate/frigate/object_detection.py", line 75, in detect_raw
2024-06-17 09:38:42.883490184 return self.detect_api.detect_raw(tensor_input=tensor_input)
2024-06-17 09:38:42.883522448 File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 155, in detect_raw
2024-06-17 09:38:42.883524995 infer_request.infer(input_tensor)
2024-06-17 09:38:42.883557073 File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 132, in infer
2024-06-17 09:38:42.883559339 return OVDict(super().infer(_data_dispatch(
2024-06-17 09:38:42.883604094 RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:116:
2024-06-17 09:38:42.883607221 Exception from src/inference/src/cpp/infer_request.cpp:66:
2024-06-17 09:38:42.883609966 Exception from src/plugins/intel_cpu/src/infer_request.cpp:367:
2024-06-17 09:38:42.883613283 ParameterMismatch: Failed to set tensor for input with precision: u8, since the model input tensor precision is: f32
2024-06-17 09:38:42.883615091
2024-06-17 09:38:42.883620285
2024-06-17 09:38:42.883622100

Anyone any thoughts ?

Cheers

Version

0.14 Beta 3

Frigate config file

See body of message

docker-compose file or Docker CLI command

N/A

Relevant log output

See body of text

Operating system

Other Linux

Install method

Docker Compose

Object Detector

OpenVino

Any other information that may be helpful

No response

[blakeblackshear]

YOLOv8 support was removed due to licensing issues: #10717

[blakeblackshear]

You should give YOLO-NAS a try instead: https://deploy-preview-11419--frigate-docs.netlify.app/configuration/object_detectors#yolo-nas

[blakeblackshear]

Just to be clear, YOLOv8 might never be supported. Did you try YOLONAS built with the colab notebook in the docs? I get much better than 50ms on the GPU.

[stevenbrown1862]

Yes I used the colab notebook in docs. I noticed at the time that the first step threw some errors about the wrong version of numpy? So anyway following your question I decided to repeat the process to get another copy of the model file and try running with that in case it helps. Anecdotally the inference times seem lower than before but I can convince myself of anything so we shall see. The inference times are less of a concern than the system crashes. I shall report back.

[stevenbrown1862]

System lasted about 30 minutes before crashing with nothing in the logs. I'll stick with 0.13 which works well for me. Will probably try again in a few months after a few kernel updates in case that fixes it.

[xfaustus]

hello @stevenbrown1862, which is your yolonas model shape size? And how many cameras do you have? At which resolution?
I have a Pentium Silver N6000 with 7 cameras a 1280x720 with yolonas exported at shape size 256 and my inference is about 30-35ms.
(Also ignore numphy error, i have the same but i didn't investigate further since conversion worked flawlessly)

[stevenbrown1862]

I only have 4 cameras. As mentioned above, my concern is not really the inference speed but the fact that the system can't go more then a couple of hours without crashing.
My inference speed with yolov8 and 0.13 is around 30ms and more importantly it has gone over a year without crashing. I'm assuming I am an edge case with 0.14 and yolonas and it's something to do with my hardware and os combination.

[jsapede]

what is the impact of the size change from 320 to 256 or lower ?
as i tested inference time is greatly reduced with 128 instead. how does it affecs confidence in detection ?

[xfaustus]

with 320 average inference time was 50ms. Confidence is quite good with 256. I had good result with yolov8 in past (until it was removed due license issue), yolonas should be superior. I had bad experience with yolox, mobilessd net seem superior at least with my case. I have 2 big dogs (~42kg) and main struggling is with them since they easily are detected as person expecially when they stand sit on just back legs.... About 128 i didn't tried but i have the suspect that it will be not enough pixels to correclty detect person at far distance of my property but i will give a try and let you know

[philippedegeer]

Frigate14/Ovenvino/yolonas_s working well for me so far. On my test hardware I get an inference time of 60ms with the 320 model on a modest Celeron 4095 iGPU. With the same setup, Frigate13/Openvino/yolov8 inference time was 55ms, so slightly better while I was expecting yolonas to be a bit faster
I am evaluating if I can transition away from yolov8, which gave me great results, before buying new hardware. So far, anecdotal observations show very good yolonas 320 detection performance. I will report more later

[philippedegeer]

@andrey-skat. I only have done limited custom training so far on YoloNas, using Google Collab as a first test. I have noticed slower training speed but not anything like in your case. I will get back to more custom training after the summer period.

[andrey-skat]

I found a bigger problem, after successful training detection in Frigate doesn't work. No errors, just no events :) I want to check predictions with supergradients library methods directly, but for know I return yolov8 support and use my old model. I don't know how long I can do that, but it easier than struggling with yolonas for now.

update: checked predictions with supergradients, all good. In frigate no detections. Btw, inference speed for 320x320 yolov8 on N100 openvino - 21ms, yolonas - 34ms

[392media]

Is yolonas support being removed from Frigate? I tried to look at the notebook page but it says "Not Found".

https://github.com/frigate/blob/dev/notebooks/YOLO_NAS_Pretrained_Export.ipynb

[blakeblackshear]

That's the wrong url: https://github.com/blakeblackshear/frigate/blob/dev/notebooks/YOLO_NAS_Pretrained_Export.ipynb

[philippedegeer]

Some more news on YoloNAS from this side. I recently purchased a NUC14 with an Intel Core Ultra 7 155H (the one with the Arc iGPU). On YoloNAS-M 256x256 I am seeing an inference speed of 13ms on the iGPU. However, when the sun/wind combination shows up, I can have more than 120 detections per second from my 4 cameras without skipping any. In this case, the iGPU usage never exceeds 50%. This is not consistent with the inference speed and may be linked to the iGPU variable clock speed. If this is the case, I would have expected variations in the inference speed but may be this indicator has some averaging out built in. Or may be someone has another explanation for this.

Apart from that, for my use case (4 Axis cameras pointing to a wild backyard), still great person detection and not a single FP with YoloNAS (s or m) in more than three weeks.