Multiprocessing

[nkrah]

Enable GATE 10 to split a simulation into multiple parallel processes.
THIS IS WORK IN PROGRESS

First implemented items:

• split run timing intervals
• adapt dynamic objects (run-based)
• spawn processes via Pool
• write output into a separate subfolder per process

Still missing:

• merge actor output from different processes

[dsarrut]

why ? I thought we just divide ALL time_interval by the number_of_processes

[nkrah]

Yes, but letting the user define the total number of processes rather than the process per run is more intuitive and will not require an API change if we implement a more advanced splitting scheme in the future. So I think it's better this way.

[nkrah]

Yes, but letting the user define the total number of processes rather than the process per run is more intuitive and will not require an API change if we implement a more advanced splitting scheme in the future. So I think it's better this way.

…

On Oct 11 2024, at 11:00 am, David Sarrut ***@***.***> wrote: @dsarrut commented on this pull request. In opengate/managers.py (#486 (comment)): > output = se.run_engine() return output - def run(self, start_new_process=False): + def generate_run_timing_interval_map(self, number_of_processes): + if number_of_processes % len(self.run_timing_intervals) != 0: why ? I thought we just divide ALL time_interval by the number_of_processes — Reply to this email directly, view it on GitHub (#486 (review)), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AIFQFYM2QBJ7YWBUP4ELK2TZ26HTVAVCNFSM6AAAAABPVS5KBSVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGNRSGMYTOOBUHE). You are receiving this because you authored the thread.

[nkrah]

I figured out a flexible mechanism to merge data back into one single actor output (if data is mergeable: true for images, not true yet for ROOT).
We will need a new type of method, common to all actors, namely FinalizeSimulation(), to be triggered from the Simulation after all processes have finished. Writing the combined output (from the processes) to disk will be done in FinalizeSimulation(). The EndOfSimulation(), where writing currently takes place, is called inside the process and therefore before combining the output. We can also add an option to not store intermediate, i.e. per process, output on disk if not needed. For example: images are accessible directly via memory and can be merged that way. No need to access data from disk.

Note: FinalizeSimulation() will not have access to engines because they do not exist any more outside of the subprocess.

[nkrah]

New:
The following actors now work in multiprocessing (local machine):

• SimulationStatisticsActor: data is merged in memory and accessible after the simulation; written to disk if requested

• Actors with ROOT output: root files (from subdirectories per process) are merged into new root file in main output folder structure. Event IDs are automatically incremented. RunIDs are recreated as per the original simulation.

Works with test019_phsp_actor -> created a new variant of the test.

Still need to create variants of other tests that use ROOT output to check.