How to produce compressedVideo

[charlielito]

Hi! A couple of months ago you announced https://foxglove.dev/blog/announcing-h264-support-in-foxglove
There you said

For ROS 2, there is a community-contributed open source image transport package that uses ffmpeg to encode H.264 video.
To experiment with video in your existing image pipeline, you can adapt this package to publish foxglove_msgs/msg/CompressedVideo messages.

Nonetheless, this doesn't seem trivial to do. Do you have any out-of-the-box tooling so the users can create a rosbag with your compressedVideo messages from a normal ROS2 Image message topic?

[jtbandes]

There is an experimental tool which converts foxglove.CompressedImage to foxglove.CompressedVideo: https://github.com/foxglove/mcap-videoify

[defunctzombie]

Do you have any out-of-the-box tooling

We do not - video support tends to depend very much on what hardware you have, what libraries you want to use, etc. The ffmpeg transport package approach is viable and it would be good to make a variant that publishes our CompressedVideo messages but it is not something we've done yet.

[charlielito]

For anyone wondering how to test this, these are the steps we did to create the rosbag with the foxglove messages for compressedVideo:

1. We have to compile this repo: https://github.com/ros-misc-utilities/ffmpeg_image_transport. In the readme are the instructions on how to install the dependencies with rosdep. We have a Fork of that package modified to output directly foxglove compressedVideo msgs you should use https://github.com/kiwicampus/ffmpeg_image_transport/tree/feature/foxglove_compressed_video.

2. Install image_common because there is image_transport needed to do the decoding. If your image topics have QoS best_effort we encourage you instead to compile it from source from this branch https://github.com/ros-perception/image_common/tree/rolling because there was a bug when setting the QoS. If you are using another ros distro incompatible, look at what we changed in this PR Add QoS option reliability to republisher qos params ros-perception/image_common#296 to fix it.

Finally just run the encoding node with the ffmpeg params suits your case

ros2 run image_transport republish raw in:=/camera/color/image_raw --ros-args -p "qos_overrides./camera/color/image_raw.subscription.reliability:=best_effort" --ros-args -p "ffmpeg_image_transport.preset:=ultrafast" --ros-args -p "ffmpeg_image_transport.qmax:=10"

This should produce an /out/ffmpeg topic with already the foxglove messages you need to record the rosbag