Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate logging from all processes in distributed run. #3590

Open
sriharshakandala opened this issue Feb 3, 2025 · 2 comments
Open

Duplicate logging from all processes in distributed run. #3590

sriharshakandala opened this issue Feb 3, 2025 · 2 comments

Comments

@sriharshakandala
Copy link
Member

There are duplicate printouts of some logging statements during distributed runs.
E.g.: https://buildkite.com/clima/climaatmos-dot-jl-cpu-scaling/builds/47#0194c851-8a69-485e-807f-50b817598dc2
Here, for example, all processes print the configuration file name. This is particularly noticeable for large distributed runs.
It is sufficient if this information is printed out from one process (for example, the root process).

@sriharshakandala
Copy link
Member Author

cc: @nefrathenrici

@nefrathenrici
Copy link
Member

nefrathenrici commented Feb 3, 2025

These are duplicated because the logger is only activated in ClimaComms.init, since it requires MPI_comm_rank to silence non-root processes. Any log statements before will be printing by all MPI procs.

@nefrathenrici nefrathenrici marked this as a duplicate of #3591 Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants