-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Subfile Read Issue #62
Open
yzanhua
wants to merge
22
commits into
master
Choose a base branch
from
subfile-read
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Commits on May 30, 2023
-
Configuration menu - View commit details
-
Copy full SHA for ba718a1 - Browse repository at this point
Copy the full SHA ba718a1View commit details -
Configuration menu - View commit details
-
Copy full SHA for f7049e4 - Browse repository at this point
Copy the full SHA f7049e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for d905666 - Browse repository at this point
Copy the full SHA d905666View commit details -
Configuration menu - View commit details
-
Copy full SHA for 493ea35 - Browse repository at this point
Copy the full SHA 493ea35View commit details -
Configuration menu - View commit details
-
Copy full SHA for 63963d0 - Browse repository at this point
Copy the full SHA 63963d0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5d1fed5 - Browse repository at this point
Copy the full SHA 5d1fed5View commit details -
Configuration menu - View commit details
-
Copy full SHA for ebd19c2 - Browse repository at this point
Copy the full SHA ebd19c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0e11f4f - Browse repository at this point
Copy the full SHA 0e11f4fView commit details -
Configuration menu - View commit details
-
Copy full SHA for d21c041 - Browse repository at this point
Copy the full SHA d21c041View commit details -
fix: correct nldset and nmdset for subfiles
when opening a subfile, fp->nldset and fp->nmdset are not read from subfiles. Instead, they are read from the master file. This commit fix this issue and read from the subfile correctly.
Configuration menu - View commit details
-
Copy full SHA for eeca154 - Browse repository at this point
Copy the full SHA eeca154View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9436082 - Browse repository at this point
Copy the full SHA 9436082View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3601bbc - Browse repository at this point
Copy the full SHA 3601bbcView commit details -
Configuration menu - View commit details
-
Copy full SHA for b21a80c - Browse repository at this point
Copy the full SHA b21a80cView commit details -
fix: subfile read: mismatch of nsubfiles and nproc
It is possible that, for example, a file is created with 8 subfiles. But when openning and reading the file, we only use 4 processes. In the original implementation before this fix, the info of 8 subfiles is not saved. Only the first fp->ngroup subfiles will be opened for read, where fp->ngroup is a number bounded by the number of processes (i.e. <= 4 in this case). In this fix, we use fp->nsubfiles to store the number of subfiles for an opened file. All fp->nsubfiles subfiles will be opened for read.
Configuration menu - View commit details
-
Copy full SHA for c32690c - Browse repository at this point
Copy the full SHA c32690cView commit details -
fix: possible hangs during dataset read.
MPI_FILE_set_view is a collective call. Befroe this fix, not all processes call this function during dataset read, introducing possible hangs. A subfile read test case may trigger this issue more easiliy. This commit fix this issue.
Configuration menu - View commit details
-
Copy full SHA for 6c0a053 - Browse repository at this point
Copy the full SHA 6c0a053View commit details -
testcase: subfile_dread tests multiple scenarios
Test the following: 1. nsbufile > nproc 2. nsubfile == nproc 3. nsubfile < nproc For each of the above, test: 1. read pattern same as write pattern (row wise) 2. read pattern is row wise, but each process read a different row than it writes. (read from one subfile that process is not responsible for) 3. read pattern is column wise (read from several subfiles) 4. read all dataset. This means a total of 12 scenarios are tested. Also, we test each scenario using 1 to 12 number of processes. This makes sures Log VOL also works for odd number of processes.
Configuration menu - View commit details
-
Copy full SHA for 817fd11 - Browse repository at this point
Copy the full SHA 817fd11View commit details -
Configuration menu - View commit details
-
Copy full SHA for c15c360 - Browse repository at this point
Copy the full SHA c15c360View commit details
Commits on Jun 5, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3b3a0a2 - Browse repository at this point
Copy the full SHA 3b3a0a2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4e61d18 - Browse repository at this point
Copy the full SHA 4e61d18View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2f85ab9 - Browse repository at this point
Copy the full SHA 2f85ab9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f368c6 - Browse repository at this point
Copy the full SHA 7f368c6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 815918f - Browse repository at this point
Copy the full SHA 815918fView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.