PoC: Add load simulator

[viktorerlingsson]

WHAT is this pull request doing?

WIP/PoC - Just an idea!

Adds a load simulator to lavinmqperf. Creates nr of queues, then pubs and consumes from those queues at a specified rate and message size with a specified number of publishers/consumers per queue. Could be helpful for testing what kind of resources are used in specific scenarios. (For example customers wanting to know what plan they should be using)

Mostly copied stuff from throughput. Should definitely be refactored if we continue with this idea.

How to run:

bin/lavinmqperf load-simulation --uri=amqp://guest:guest@localhost:5673 --time 30 --rate 1000 --queues 10 --connections 1 --consumers 1 --publishers 1 --size 100

-- queues (-q) is number of queues to be created and published/consumed from/to
--rate (-r) is publish rate per queue per second
--connections (-c) is number of connections per queue
--consumers (-y) is number of consumers per queue
--publishers (-x) is number of publishers per queue
--size (-s) is message size

Lavinmqperf will report progress every second like this:

Publish rate: 10814 msgs/s Consume rate: 11128 msgs/s. Expected rate: 10000 msgs/s. Using 9.6MiB memory.

Expected rate is how many messages 'should' be published/consumed every second based on rate multiplied by nr of queues.

and after the run is finished:

Summary:
Average publish rate: 9360.7 msgs/s
Average consume rate: 9814.6 msgs/s

Per queue:
Average publish rate: 936.07 msgs/s
Average consume rate: 981.46 msgs/s

If the load could be handled succesfully, the average publish and consume rates per queue should match up with the rate you specified with --rate.

HOW can this pull request be tested?

Run lavinmqperf load-simulation. Example:

bin/lavinmqperf load-simulation --uri=amqp://guest:guest@localhost:5673 --time 30 --rate 1000 --queues 10 --connections 1 --consumers 1 --publishers 1 --size 100

[github-actions]

Main benchmark
'Average publish rate: 424305.0 msgs/s'
'Average consume rate: 424377.8 msgs/s'

PR benchmark
'Average publish rate: 401302.9 msgs/s'
'Average consume rate: 399304.2 msgs/s'

Keep in mind, these numbers are not representative of LavinMQ's peak performance.
It is rather an indication of how the changes of this pull request affects the performance of the main branch.

[carlhoerberg]

Have you checked out https://github.com/rabbitmq/RabbitTestTool/tree/main/benchmark ? it's pretty cool for testing pretty advanced topoligies.

[viktorerlingsson]

Have you checked out https://github.com/rabbitmq/RabbitTestTool/tree/main/benchmark ? it's pretty cool for testing pretty advanced topoligies.

No, had not seen that before. Interesting! Seems like a competent tool.

This simpler load testing in lavinmqperf might still be interesting though? As a simpler tool that we will have easily available since it comes with lavin. (It already pretty much works as is, just needs to be polished a bit)