MPIO support for Basis classes #274
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dreamer2368
changed the title
dreamer2368
changed the title
dreamer2368
force-pushed
the
mpi-io
branch
2 times, most recently
from
06d0818 to
55acb08
Compare
dreamer2368
requested review from
ckendrick,
dylan-copeland,
chldkdtn and
siuwuncheung
…the exisiting file.
…mains for backward compatibility.
|
Reflecting @dylan-copeland 's comments, function signatures of database are changed:
|
ckendrick
reviewed
Mar 18, 2024
|
Reflecting @ckendrick 's comment, parallel hdf5 is set optional. |
dylan-copeland
approved these changes
Mar 19, 2024
ckendrick
approved these changes
Mar 19, 2024
andersonw1
pushed a commit
that referenced
this pull request
Apr 2, 2024
* BasisWriter::writeBasis create/open a file and closes the file at the end. * stylization. * HDFDatabase::putIntegerArray - overwrites if the dataset exists. * enforce single time interval in Options. * HDFDatabase::putIntegerArray does not allow overwrite. * BasisWriter::writeBasis always create the file, which will overwrite the exisiting file. * add a header and stylization. * remove increase time interval test, as time interval is fixed to 1. * add an error message for a guidance. * remove test_SVD from ci workflow. * SVD::increaseTimeInterval - allow the initial time interval. * minor fix in test_IncrementalSVDBrand. * reflecting the comments. * removed the concept of time interval in BasisReader. time argument remains for backward compatibility. * BasisWriter: removed the concept of time interval. * minor fix in BasisReader. * SVD: removed the concept of time intervals. * BasisGenerator: removed the concept of time interval. * add test_SVD.cpp for resolving conflict. * stylization. * changed function signature of BasisGenerator::takeSample. * rebased to resolve conflict. * changed function signature for BasisReader::getSpatialBasis. * changed function signature for BasisReader::getTemporalBasis. * changed function signature for BasisReader::getSingularValues. * changed function signature for BasisReader::getSnapshotMatrix. * unit test with fapl_mpi. * parallel integer array writing example. * add timing for H5DWrite. * 2d integer array parallel writing example. * hdf5 parallel integer array reading example. * add d_dim as const member variable of BasisGenerator/Reader. * BasisReader: CSV format is never used. Enforce HDF5 format from now on. * create/open_parallel function. * stylization * test_HDFDatabase: test for selective parallal I/O. * parallel I/O routines within basis classes. * HDFDatabaseMPIO class initial loading. * rebase from generator-fix2 * deployed HDFDatabaseMPIO to basis classes. * test for partial getSpatialBasis. * test routien for scaling. * add hdf database test in ci workflow. * update doxygen description * reflecting comments 1 * reflecting comments 2. changed interface. * minor fix * reflecting comments * hdf5 parallel is optional. set compile-time if statements. * minor fix
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR intends to support multi-path input/output (MPIO) of HDF5 format, thereby enabling MPI I/O of a single file.
Currently the I/O only supports 'file-per-processor' parallel I/O, where each processor reads/writes its own file. While this guarantees the I/O scalability most of the time, this fixates the number of processors for the entire workflow of basis generation and continual use.
Single file I/O with multiple processors, of course, can cause a significant overhead. This, however, has been optimized within HDF5 library up to some extent. As long as the file system is appropriately set up (e.g. Lustre stripe count), the parallel I/O on a single file can maintain reasonable time cost (from HDF5 group post).
If used carefully, this option will provide users with the freedom of changing number of processors throughout their own basis generation/use workflow.
List of implementation:
HDFDatabaseMPIOclassHDFDatabaseMPIOas an option inBasisGenerator,BasisReader, andBasisWriter(optional) removing the concept of time interval in basis data formats.Comparison of scalability for
5000x1000snapshot matrix, betweenHDFDatabaseandHDFDatabaseMPIO:BasisGenerator::endSamples(SVD + basis writing)BasisGenerator::writeSnapshots(snapshot writing only)The comparison is done on LC quartz debug node, with lustre stripe count of 36.