Create profile presets for re-encoding stream to VP9 or AV1 for any platform that can support it

[Xev]

I want to record cameras that provide only H264 and H265 streams, in VP9 now and AV1 soon.

Main reason is space-saving and that most desktops and portable devices have hardware acceleration support for VP9, and will have for AV1 soon.
That means that if I setup in config for detection role to use regular h264 stream, and set up recording role with the same stream but with preset for ffmpeg for re-encoding, then I should have recordings on hard drive in format that I can play in browsers on any desktop or phone.

I don't want to wait eons for camera manufacturers to start selling cameras that support VP9 or AV1 codec, and especially I don't want to wait even more years on top of that for those cameras to get reasonably priced.

Main fault of h265 which modern cameras readily provide is that most browsers don't have license to decode it, which means even if I correctly feed frigate h265 stream, I won't be able to browse recordings in frigate's web interface, rendering those recordings useless.

Currently I am forced to store h264 recordings on hard drive, which take a lot of space if they are in good quality.
With how good Frigate got with support for graphic cards, I assume based on my rudimentary knowledge that a lot of models (of intel cpus, amd cpus, nvidia gpus, amd gpus, intel arc) already support hardware decoding and encoding of VP9, and linux kernels and ffmpegs version had support for it for ages.
Which means for anyone that wants to save space, it should be as easy to purchase for example Nvidia GPU starting from 900 series or intel cpu starting from 10th gen, install it and add preset to config.

Drivers in linux kernels and ffmpeg versions that can support gpus and cpus that support hardware encoding of AV1 should start appearing soon, even if they are not already there.
Then it will be as simple as switching profile preset in config from VP9 to AV1, and everything should still work flawlessly?

Why wait for industry to play catch up, when we can go ahead of curve by ourselves?

[nglessner]

My wish list:

camera streams to frigate: H264
frigate live streaming, object detection, etc: H264
frigate 24/7 recording: transcoding to H265, AV1 or VP9 using intel quick sync

[moandcompany]

Your wishlist for re-encoding is probably best handled through go2rtc's features Docs

The goal for re-encoding of recordings can effectively be done through transcoding of the actual camera stream to your desired codec (i.e. H265) via ffmpeg as used by go2rtc.

[nglessner]

Thanks @moandcompany . I tried this configuration (and many variations of it) with little luck. Oddly, the h264_2mbps custom profile with the added #hardware tag on the end of the stream definition did show gpu usage (10-15% range). The h264 and h265 software profiles worked, but as expected with cpu only. Neither of the custom hardware acceleration profiles worked. The log would say "invalid data found when processing input".

At this point I'm expected the issue is either related to the N305's capabilities or that the camera is a Reolink doorbell camera (I have tried both rtsp and rtmp streams). It's also possible that I have something configured incorrectly as well. Does the igpu driver on the proxmox host need to match the version listed in vapi on the docker container?

[NickM-27]

no, the versions don't need to match.

[nglessner]

Oh and is there a better source for ffmpeg settings than their site? Something that cross references hardware and platforms? I've tried looking, but came up empty.

[NickM-27]

I learned off of their site along with searching different things

[moandcompany]

Since you're trying to use QuickSync , perhaps the following doc page would be useful:
https://trac.ffmpeg.org/wiki/Hardware/QuickSync#Transcode

Here is one of their examples using qsv for a hardware accelerated hevc/h265 -> hardware accelerated h264 transcode.

ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.mp4 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080' -c:v h264_qsv output.mp4

[mike2307]

Did anyone had success with configuring go2rtc for transcoding to AV1 or VP9 and can provide a reference config?
Thanks in advance :)

[Arol450]

ffmpeg:
hwaccel_args: preset-intel-qsv-h264

go2rtc:
streams:
Cars_live:
- ffmpeg:rtsp://:@192.168.2.44/Streaming/Channels/1

cameras:
Cars:
ffmpeg:
output_args:
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v av1_qsv -global_quality:v 31 -an
inputs:
- path: rtsp://localhost:8554/Cars_live
input_args: -loglevel error -hwaccel qsv -qsv_device /dev/dri/renderD128 -hwaccel_output_format qsv -c:v h264_qsv
roles:
- record
- path: rtsp://:@192.168.2.44/Streaming/Channels/102
roles:
- detect
detect:
enabled: true
width: 640
height: 480
motion:
threshold: 30
contour_area: 10
improve_contrast: true
live:
stream_name: Cars_live