Create a low-latency (fast) video camera with a Raspberry Pi and camera module. Stream the video in realtime to a Mac, PC or other Pi for use in media projects in Open Broadcaster Software (OBS) and other media tools. Optionally scale up by creating a fleet of cameras that broadcast video to one or more machines.
Raspberry Pi UDP Camera is not a webcam, however it may be used similarly via obs-gstreamer (for OBS), GStreamer projects in general and other software (todo for other software, untested, solutions welcome).
This project was created with a need to use multiple cameras in a live Twitch stream. During a pandemic, webcams have been considerably more expensive, while Raspberry Pi camera modules offer arguably higher quality video than some webcams. With Pis and camera modules handy, let's share.
Using balena.io allows for quick deployment and monitoring of Docker-driven apps to devices such as Raspberry Pi (among others). It also allows for mass deployment, monitoring and updates, which had a use case when this project was created (multiple low-latency cameras for the HotSparkLab Twitch stream).
Use the following link to quickly setup the app in balena.io. See Slow Setup below for full details of what's included and various options.
Expand
Here's a full run-through from scratch if not using the deploy button above, including descriptions of available options and where they live.
Setup a belana.io account.
On the Applications dashboard, choose Create application.
The camera fleet of one or more Pis will be configured with defaults that will apply to all potential camera devices. Select Fleet configuration on the left navigation.
Under CUSTOM CONFIGURATION VARIABLES, add the following variables and values. Memory will adapt to what's available on the Pi:
| Variable Name | Value |
|---|---|
| BALENA_HOST_CONFIG_start_x | 1 |
| BALENA_HOST_CONFIG_gpu_mem_256 | 192 |
| BALENA_HOST_CONFIG_gpu_mem_512 | 256 |
| BALENA_HOST_CONFIG_gpu_mem_1024 | 448 |
Select Environment Variables in the left menu. Add the following required environment variables and values that tell the camera which computer to broadcast to.
| Environment Variable | Value | Description |
|---|---|---|
| STREAM_DESTINATION_IP | Example: 192.168.0.10 | This is the local IP address of the computer that will receive the UDP video stream. |
| STREAM_DESTINATION_PORT | Example: 5001 | This is the the port of the receiving computer to send to. Note that this will need to be unique per camera if the computer is receiving multiple video streams. As this is in the fleet environment variables as the default, it can be overridden per device as needed. |
Additional optional environment variables may be set as needed to customize camera/raspivid settings.
| Environment Variable | Default Value | Description |
|---|---|---|
| WIDTH | 1280 | Available camera image width as supported by raspistill (see 'Version 1.x' and 'Version 2.x' depending on Pi camera version used). Choose an option that pairs with IMAGE_HEIGHT. |
| HEIGHT | 720 | Available camera image height as supported by raspistill (see 'Version 1.x' and 'Version 2.x' depending on Pi camera version used). |
| ROTATION | 0 | Supported image rotation values: 0, 90, 180 or 270 |
| FLIP_HORIZONTAL | Flip the image horizontally. If enabling, use a value of -hf. |
|
| FLIP_VERTICAL | Flip the image vertically. If enabling, use a value of -vf. |
|
| FRAMES_PER_SECOND | 30 | A framerate of 2 to 30 frames per second is supported. |
| INTRA_REFRESH_PERIOD | 60 | This option specifies the number of frames between each I-frame. Larger numbers will reduce the size of the resulting video. Smaller numbers make the stream less error-prone. |
| BITRATE | 2500000 | Bits per second. 2500000 would be 2.5Mbits/s. Setting to high will result in stuttering video (untested in this project so far, 2500000 was smooth with Ethernet connection). |
| METERING_MODE | average | The following metering modes are supported: average, spot, backlit or matrix. See -mm details here for full details. |
| EXPOSURE_MODE | fixedfps | The following exposure modes are supported: auto, night, nightpreview, backlight, spotlight, sports, snow, beach, verylong, fixedfps, antishake, fireworks. See -ex details here for full details. |
lone the Raspberry Pi UDP Camera repository.
git clone https://github.com/hotsparklab/raspberry-pi-udp-camera.git
Follow these instructions to install the balena.io CLI, used to upload the project locally to the balena.io application.
Login via the CLI (web authentication option when asked is recommended)
balena login
List available projects.
balena apps
Take note of the app name that was created for Raspberry Pi UDP Camera and push to the app.
balena push your-app-name-here
🦄 A successful deploy will end with an ascii art unicorn named Charlie.
Expand
In the balena.io dashboard, choose Devices in the left navigation.
Click the Add Device button at the top.
In the modal window, set an Application Name and choose the device type, Raspberry Pi 3 for example (several Pis supported, tested on Pi 3 and Zero Wireless), then choose Create new application. Note: Wireless setup can also be configured here. Wireless connections could result in added latency or dropped frames when viewing the camera depending on the connection.
After downloading the application for the device, it can be burned to a MicroSD card using balenaEtcher. Ensure the camera is installed properly to the Pi. Insert the card. Plug-in via Ethernet on same switch or use wireless if configured as such. Power the device and wait for a couple minutes. Connection status can be tracked on the Balena.io dashboard.
GStreamer is a robust, open source multimedia framework, in this use case great for the purpose of receiving and displaying video broadcasted from the Pi. The obs-gstreamer plugin bridges the gap between GStreamer and OBS, allowing configuration and display within OBS for a new "GStreamer source" (kinda like VLC Player, more robust and without a UI).
Expand
For the obs-gstreamer to work, GStreamer runtime and the GStreamer development SDK will need to be installed to the PC. Note: There's an experimental Mac and Linux obs-gstreamer plugin, untested here. Post an issue or PR to inform us how it works if you use it.
Download the MinGW 64-bit (or 32-bit if needed) runtime and development installers from GStreamer's website and install.
An environment variable will need to be added to Windows PATH so that obs-gstreamer knows where to look for GStreamer. Otherwise, it won't show as an option in OBS.
Open Control Panel and search for environment variables on the top-right.
Choose Edit environment variables for your account.
Add a new environment variable with variable name of GSTREAMER_1_0_ROOT_X86_64 and variable value of C:\gstreamer\1.0\x86_64 (or wherever you installed GStreamer if different).
Hit the OK button, then the parent window's OK button to finish up.
obs-gstreamer will enable a new configurable "GStreamer source" in OBS.
Visit the obs-gstreamer page and choose Go to download.
Download and extract the latest obs-gstreamer.zip.
copy obs-gstreamer.dll into C:\Program Files\OBS\plugins (64-bit) or C:\Program Files (x86)\OBS\plugins (32-bit), altered if OBS was installed elsewhere.
Now for the fun part. Get live, low-latency video from the Pi to show in OBS as a video source.
The Pi with camera will send video to the streaming computer via UDP. To do that, the Pi will need to know the streaming computer's local IP address.
In Windows, type cmd in the Windows Start search bar and hit enter to launch Windows terminal.
Type, ipconfig and press Enter.
The IP address will be towards the top and looking similar to, `192.168.0.X (just not the .255 one listed). Note that address.
Back to the balena.io for a moment, visit the application's Environment Variables page. Update STREAM_DESTINATION_IP value to the noted IP address above.
If multiple Pis are going to be running the application for the same streaming computer, make sure each device specifies a unique STREAM_DESTINATION_PORT, e.g. 5001, 5002, etc.
Launch OBS.
With a scene active, under Sources, choose +.
Select GStreamer Source.
In the pipeline field enter the following, replacing YOUR_PI_PORT with the port specified in belana.io for that device (or fleet port is fine if there's only one device).
udpsrc port=YOUR_PI_PORT ! h264parse ! avdec_h264 ! video.
If/when the Pi is booted and running, the camera source will be displayed in the scene and can be scaled and filtered to taste. Note: Pi status can be monitored in the balena.io dashboard with latest logs shown.
🚨🎉🚨🎉
There hasn't been luck successfully playing this formatted UPD video stream in VLC Player on Mac or PC yet (or any low-latency video stream from a Pi to VLC Player). Prove me wrong. Post an issue or PR with the solution. Let's add instructions for it as it would be very accessible to OBS users with VLC Player already installed (if simple and possible).
While untested here, UDP video streams have been known to be used as webcams for a receiving computer through third-party applications. A Google search brought up several results to explore. Which solution works for you? Post an issue or a PR and share!
Many more holes would be in the wall with far fewer brain cells active today if it wasn't for fzwoch's obs-streamer project. Also thanks to great comments from Dregu and crossan007 in this issue and the balena.io team for their feedback on live streaming topics. Thank you!