[Config Support]: audio no longer working with go2rtc and tapo

[ivanjx]

Describe the problem you are having

my camera outputs pcm audio and as expected i cant play the audio in the browser (unless it is on webrtc live streaming). i tried using the generic audio aac preset but the recordings are still muted. exporting the footage also shows no audio.

Version

latest dev

Frigate config file

go2rtc:
  streams:
    cam1:
      - tapo://x:[email protected]
  webrtc:
    candidates:
      - 192.168.20.1:8555

ffmpeg:
  hwaccel_args: ' '
  output_args:
    record: preset-record-generic-audio-aac
  input_args: preset-rtsp-restream

cameras:
  cam1:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam1
          roles:
            - detect
            - record

Relevant Frigate log output

no errors

Relevant go2rtc log output

no errors

Frigate stats

No response

Operating system

Debian

Install method

Docker Compose

docker-compose file or Docker CLI command

irrelevant

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

go2rtc streams:

{
   "consumers": [
      {
         "bytes_send": 39859432,
         "format_name": "rtsp",
         "id": 1,
         "medias": [
            "video, sendonly, H264, H265",
            "audio, sendonly, MPEG4-GENERIC"
         ],
         "protocol": "rtsp+tcp",
         "remote_addr": "127.0.0.1:47082",
         "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.4\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=control:trackID=0\r\n",
         "senders": [
            {
               "bytes": 39360731,
               "codec": {
                  "codec_name": "h264",
                  "codec_type": "video"
               },
               "id": 3,
               "packets": 2333,
               "parent": 2
            }
         ],
         "user_agent": "FFmpeg Frigate/0.15.0-0b6ab24"
      },
      {
         "format_name": "probe",
         "id": 16,
         "medias": [
            "video, sendonly, ALL",
            "audio, sendonly, ALL",
            "audio, recvonly, ANY"
         ],
         "protocol": "http",
         "receivers": [
            {
               "childs": [
                  21
               ],
               "codec": {
                  "codec_name": "ANY",
                  "codec_type": ""
               },
               "id": 20
            }
         ],
         "remote_addr": "127.0.0.1:54690",
         "senders": [
            {
               "codec": {
                  "codec_name": "h264",
                  "codec_type": "video"
               },
               "id": 17,
               "parent": 2
            },
            {
               "codec": {
                  "codec_name": "pcm_alaw",
                  "codec_type": "audio",
                  "sample_rate": 8000
               },
               "id": 19,
               "parent": 18
            }
         ],
         "user_agent": "python-requests/2.32.3"
      }
   ],
   "producers": [
      {
         "bytes_recv": 41608080,
         "format_name": "tapo",
         "id": 2148319232,
         "medias": [
            "video, recvonly, H264",
            "audio, recvonly, PCMA/8000",
            "audio, sendonly, PCMA/8000"
         ],
         "protocol": "http",
         "receivers": [
            {
               "bytes": 39360731,
               "childs": [
                  3,
                  17
               ],
               "codec": {
                  "codec_name": "h264",
                  "codec_type": "video"
               },
               "id": 2,
               "packets": 2333
            },
            {
               "childs": [
                  19
               ],
               "codec": {
                  "codec_name": "pcm_alaw",
                  "codec_type": "audio",
                  "sample_rate": 8000
               },
               "id": 18
            }
         ],
         "remote_addr": "192.168.20.21:8800",
         "senders": [
            {
               "codec": {
                  "codec_name": "ANY",
                  "codec_type": ""
               },
               "id": 21,
               "parent": 20
            }
         ],
         "url": ""
      }
   ]
}

export:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'cam1_yf1asf.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.1.100
  Duration: 00:00:24.04, start: 0.000000, bitrate: 2803 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, progressive), 2688x1520, 2802 kb/s, 20.05 fps, 20 tbr, 90k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

ffmpeg command:

"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-0b6ab24 -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/cam1 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/cam1@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale=2688:1520 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"

[NickM-27]

The aac preset works fine, the issue here is that go2rtc only sends aac by default which is why the docs recommend setting up audio transcoding in go2rtc.

[ivanjx]

makes sense but why the live audio works? doesnt that mean go2rtc still sends pcm audio to the browser but somehow not to frigate?

[NickM-27]

No, the config only applies as the default query. Live views only support a specific type of audio so it overrides the default query that is set in the config

[ivanjx]

so if i do:

go2rtc:
  streams:
    cam1:
      - tapo://x:[email protected]
      - ffmpeg://cam1#audio=aac

frigate will pick up the aac or do i need to specify it somewhere?

[NickM-27]

You'll want to use the aac or copy recording output args but yes

[ivanjx]

go2rtc:
  streams:
    cam1:
      - tapo://x:[email protected]
      - ffmpeg://cam1#video=copy#audio=aac
  webrtc:
    candidates:
      - 192.168.20.1:8555

ffmpeg:
  hwaccel_args: ' '
  output_args:
    record: preset-record-generic-audio-copy
  input_args: preset-rtsp-restream

still not working. do i need to wait before the new hls playlist to be generated after 1 hour or so?

[ivanjx]

sorry about my last deleted comment, please ignore that.
so to make audio working again, i have to use my camera's own rtsp stream again. tapo:// used to work until last week or so combined with ffmpeg://cam1#video=copy#audio=aac. this is my working config now:

go2rtc:
  streams:
    cam1:
      - ffmpeg:rtsp://rtspuser:[email protected]:554/stream1#video=copy#audio=aac
      - tapo://tapouser:[email protected]

[ivanjx]

it seems that restarting my camera somehow restored the audio recording with tapo protocol. weird that live worked fine but not recording. everything is fine now.