Reduce Fast-RTPS RSS usage by relaxing RTPS specification [8613]

[alsora]

This issue is linked with ros2/rmw#172

Long story short, the delta in memory for adding an additional participant to a single process system increases a lot with the number of participants in the system, regardless of how they will communicate.

I looked a bit into when Fast-RTPS allocates memory and I noticed that there are 2 relevant contributions to this phenomenon, both required in order to implement EDP as described in the RTPS specs.

• Each participant has SEDPbuiltinPublicationsWriter, SEDPbuiltinPublicationsReader, SEDPbuiltinSubscriptionsWriter and SEDPbuiltinSubscriptionsReader that have to match with the ones of all the other discovered participants.

• Each participant creates ReaderProxyData and WriterProxyData objects for any discovered publisher and subscriber.

These memory allocations are invariant with respect to the topology of the DDS graph, as they basically assume that eventually it could be fully connected.

I made some experiments using a simple ROS2 application (creating 1 publisher node and N subscriber nodes in the same process).
In the following table STANDARD denotes the current Fast-RTPS master, NO_PROXY denotes a modified system where only ProxyData for the needed endpoints will be created, NO_MATCH_NO_PROXY denotes a system where, additionally, SEDPbuiltin structures match only between a subset of nodes that will actually have to communicate each other.

The measurements have been obtained with SECURITY=OFF on a standard laptop.

System	1 node	2 nodes	3 nodes	9 nodes	10 nodes	19 nodes	20 nodes
STANDARD	19Mb	21.7Mb	25.5Mb	64.9Mb	73.1Mb	192.4Mb	206.8Mb
NO_PROXY	19Mb	21.5Mb	24.5Mb	57.6Mb	65.5Mb	129.4Mb	140Mb
NO_MATCH_NO_PROXY	19Mb	21.4Mb	24Mb	42.8Mb	46Mb	73.5Mb	76.9Mb

The reduction in memory from each of these contributions is relevant.

I know of the existence of Fast-RTPS static discovery, however it has a different goal in mind, i.e. reducing the network usage.

I was thinking at a new EDP implementation that lies in between Simple and Static ones.
It could be based on a blacklist of topic names and participant names, without making any assumption on where a publisher/subscriber is located.

This would simplify the development of an application with respect of using Static EDP.

I would like to know your thoughts about this.

[alsora]

Hi, I wanted to update this PR with some of the last experiments I did.

The experiments have been done using the iRobot's benchmark application. This simulates a real ROS2 system with 10 nodes, 13 topics, 13 publishers, 16 subscribers.
https://github.com/irobot-ros/ros2-performance/tree/master/performances/benchmark

Here you find a Fast-RTPS branch where I have brutally applied the blacklist/whitelist changes mentioned above.
I have hardcoded the relationships between the nodes in order to try the idea. In the future it could be an XML to be parsed.
https://github.com/alsora/Fast-RTPS

X86_64 evaluations
ROS2 Master branch + FastRTPS Master branch

These are the results I got. Each line is a different change and they are applied incrementally.

ROS2 system	mallinfo Heap in use	Physical RAM RSS	Virtual RAM VRT
Standard	470 Mb	116 Mb	2.76 Gb
EDP blacklist	331 Mb	85 Mb	2.76 Gb
1 type listener	323 Mb	83 Mb	2.76 Gb
Proxy blacklist	308 Mb	76 Mb	2.76 Gb
RCLCPP changes and no param server	186 Mb	52 Mb	2.64 Gb
FastRTPS PKG_SIZE=10.5Kb	46 Mb	51 Mb	2.61 Gb
FastRTPS SECURITY=OFF	24 Mb	37 Mb	2.59 Gb

The links in the table redirect to the commit or issues where the applied change is described