Fix parallel parsing of many files in get_results_as_dataframe

[non-det-alle]

Hello,

When working with 1000+ simulation result files, the parallel_parsing option of the CampaignManager.get_results_as_dataframe() utility causes the main process to get stuck trying to open and load all the files at once, with no display of progress. If the files are big, this could eventually lead to memory issues and crashes.

This happens because the program is trying to build at once the full list of inputs to be fed to pool.imap_unordered. As a simple fix, I suggest to change the list into a generator. Generators are lazy iterators, so each file is only opened when the next element is requested by pool.imap_unordered, and closed when the task is done. As an added benefit, this also fixes the tqdm progress bar, which otherwise appears only when the list is completely loaded.

Best,
Alessandro

[pagmatt]

Great improvement, thanks @non-det-alle