Support for Time-Series Database and H.264 Encoding

[vachelyang]

Description

In ROS 2, besides images and videos, there are many other large data types such as point clouds, logs, etc. These types of data are sequential and have stable frequencies. To manage these data types more efficiently, there is a need for a more user-friendly database management system or accelerated compression methods. Specifically, it would be beneficial to support:

Integration with a time-series database (TSDB) to manage sequential data efficiently.
Utilizing H.264 encoding to compress fixed-frequency messages, leveraging hardware-accelerated H.264 encoding available on many machines.

Related Issues

This feature request is not dependent on any other existing features but would significantly enhance the usability and efficiency of handling large, sequential data in ROS 2. It could be part of a larger effort to improve data management and compression in ROS 2.

Completion Criteria

The feature can be considered complete when:

Support for a time-series database (TSDB) integration is implemented and documented.
H.264 encoder support for compressing fixed-frequency messages is implemented and documented.
The new features are tested and verified to work with various large data types such as point clouds and logs.
User documentation and examples are provided to demonstrate how to use these features.
Implementation Notes / Suggestions
For the TSDB integration:

Consider popular TSDB options such as InfluxDB, TimescaleDB, or others that are compatible with ROS 2.
Ensure that the integration allows for efficient storage, retrieval, and querying of sequential data.
For H.264 encoding:

Use open-cv or ffmpeg H.264 encoding available on common platforms.
Ensure that the implementation can handle various data types and frequencies without significant performance overhead.
Provide options for users to enable or disable H.264 encoding based on their hardware capabilities and requirements.
Testing Notes / Suggestions
All features in this project need tests. Please consider the following test cases:

Verify that the time-series database integration works with different types of sequential data (e.g., point clouds, logs) and supports efficient querying.
Test the performance and compression efficiency of the H.264 encoder with different data types and frequencies.
Ensure compatibility with hardware-accelerated H.264 encoding on various platforms.
Include stress tests to verify the stability and performance of the new features under heavy load.
Thank you for considering these suggestions. Implementing these features would greatly enhance the usability and efficiency of handling large, sequential data in ROS 2.

[MichaelOrlov]

@vachelyang Thanks for the proposal and feature requests.
It is very unlikely that we will add support for the time-series database (TSDB) as a core package for the rosbag2.
We already have support for the MCAP and SQLite3 database. It is just too much effort to add (TSDB) backend and support it further along with the existent MCAP and SQLite3 backends.

Feel free to add TSDB suport via stoarage plugins. Anyone can write their own storage plugins. Please see https://github.com/ros2/rosbag2/blob/rolling/docs/storage_plugin_development.md for details.

As regards the H.264 compression, that is a good proposal that makes sense to consider. However, it doesn't fit well in our current design for compression plugins. Since by current design we assume that compression plugin will serve all messages in all topics. We might need to adjust the API to allow apply compression only for specified topics.

cc: @defunctzombie to consider adding support for the H.264 compression to the MCAP plugin and visualization tools.

[defunctzombie]

tl;dr; everything you ask for is already possible and does not require any changes or features in the ROS 2 recording infrastructure.

Regarding h.264 - you can already accomplish this today within ROS2 by putting your h.264 data into supported messages. The approach you take will depend on the tooling or libraries you want to use.

For tools like Foxglove, we have defined a message format for storing h.264 and playing it back.
Heres a blog post on the topic https://foxglove.dev/blog/announcing-h264-support-in-foxglove
and it mentions leveraging this package to do the video encoding: https://github.com/ros-misc-utilities/ffmpeg_image_transport but you can use any approach that produces the correctly formatted h.264 data. For example, here's another package specific to using nvidia https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_compression which can give you hardware encoding.

As for TSDB support, my recommendation is to do this processing on your end. Rather than tying ROS 2 packaging to Influx, Timescale, whatever-new-hotness-tomorrow, you can pick the one appropriate for your company and use-case and write an workflow that will import the data from mcap into whatever datastore you want.