Stream Live Under 1 Second

[ofekkazes]

Hi,

Thank you for your package, i am only now checking it out. up until now I used libcamera and LCCV (as a wrapper) to take videos from the raspberry pi.

I am trying to get a live stream from the picam into my qt qml app as a preview, that will be under 1 second.
The reason for the qt qml app is that it is used for many other stuff, however every attempt I tried was over 4 seconds.

• I tried with the HLS stream - it got ~4.5 seconds of delay - however this is the way HLS is built.
• I tried with node-rtmp server and got a delay of ~10 seconds.
• I tried with tcp output, with ffmpeg as a middleware and nginx, with or without compression, and then receive the stream straight with rtmp in QML and got 10 seconds delay. I also tried with gstreamer in the QML side however could not get it below the 10 seconds mark.

What per your expertise would get the least amount of latency?
This Qt QML app is on the same raspberry pi, connected to the display.

Just to note, this is my QML app for the test:

import QtQuick 2.15
import QtQuick.Controls 2.15
import QtMultimedia 5.15

ApplicationWindow {
    visible: true
    width: 800
    height: 600
    title: "RTSP/RTMP Video Stream"

    MediaPlayer {
        id: mediaPlayer
        autoPlay: true
        source: "gst-pipeline:udpsrc port=1234 ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! autovideosink sync=false"
        //source: "udpsrc port=5000 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false"
        //source: "rtmp://127.0.0.1/live/stream"
        //source: "gst-pipeline: udpsrc port=5000 buffer-size=1024 ! decodebin max-size-buffers=1 max-size-time=0 max-size-bytes=0 ! videoconvert ! autovideosink" // Replace with your RTSP or RTMP URL
        videoOutput: videoOutput
    }

    VideoOutput {
        id: videoOutput
        anchors.fill: parent
        fillMode: VideoOutput.PreserveAspectFit
    }

    Button {
        text: "Play"
        anchors.bottom: parent.bottom
        anchors.left: parent.left
        onClicked: mediaPlayer.play()
    }

    Button {
        text: "Pause"
        anchors.bottom: parent.bottom
        anchors.right: parent.right
        onClicked: mediaPlayer.pause()
    }
}

[iizukanao]

Low latency and stable video playback are often a trade-off, and you need to find the right balance based on your application's requirements.

For example, with the following setup, I was able to reduce latency to around 0.4 seconds:

• Raspberry Pi 4 Model B with Camera Module 3 and a USB microphone
• Using picam with the --rtspout option and node-rtsp-rtmp-server on the sender side
• Using VLC media player 3.0.21 on macOS as the receiver
• Both sender and receiver are on the same local network
• In VLC settings:
  • Set "Network caching" to 0
  • Uncheck "Use RTP over RTSP (TCP)" (to connect via RTSP over UDP)
• Connect to rtsp://<Pi_IPAddress>:80/live/picam in VLC

The key idea here is to minimize the receiver-side buffer as much as possible. However, this comes at the cost of stability—any small network delay can cause playback to temporarily stop.

If stable playback is more important for your use case, try increasing the buffer size. The optimal buffer setting depends on your specific environment, so I recommend experimenting to find the best balance.

Hope this helps!