Question: What is the maximum number of argumentfile can be used in pabot?

[ask-almas]

Hi,

I'm trying to run the same test multiple times in parallel with different arguments and found about this argumentfile option.I would like to know what is the maximum number of argumentfile can be used in pabot? Is there any limit to to this argument?

Thank you!

[joonaskuisma]

Understanding --argumentfile[N] in Pabot

Hi @ask-almas !

I explored Pabot's code and tested the --argumentfile[N] functionality, where N is a sequentially increasing positive integer.

Technically, there doesn't seem to be an upper limit, but in practice, system memory may become a limiting factor at some point—especially if the test suites are already large. This is because when results are merged into a single log.html file using Rebot, memory usage can increase significantly.

From a CPU perspective, there shouldn't be any issues, as the number of concurrently running subprocesses is determined by the --processes argument. This means the maximum number of parallel executions is limited by the number of logical processors available on your machine.

However, note that if you run:

pabot --processes 10 tests

and the tests directory contains 10 suites, all 10 suites will run in parallel. If there are more than 10 suites, the excess ones will be queued until a core becomes available.

Now, consider running the same tests directory with 10 different --argumentfile arguments. This results in 100 test suite executions in total, but only 10 can run simultaneously.

Key Takeaway

If you want to execute a single test suite multiple times without queuing—ensuring all executions truly run in parallel—then your CPU (or the --processes argument) defines the upper limit for the number of --argumentfile instances you can use.

Hope this clarifies things! 🚀

[joonaskuisma]

Additionally, if the number of --argumentfile instances is large, the resulting log file can quickly become cluttered.

I tested this with my own dataset containing eight suites and a total of 24 test cases. Running it with 12 different --argumentfile arguments resulted in a massive and confusing log.html, displaying 24 × 12 = 288 passed tests. 😄 The "Statistics by Suite" table, in particular, became quite messy...

Alternative Approach

A more manageable solution in such cases might be to run Pabot 12 times as separate commands. This way, you get 12 individual logs that are easier to interpret rather than one overwhelming report.

Hope this helps! 🚀