Compound subsetting through H5Tconvert and field selection #118
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mattjala
force-pushed
the
cmpd_subset_conversion_pipeline
branch
from
bb3f05e
to
9979a64
Compare
mattjala
force-pushed
the
cmpd_subset_conversion_pipeline
branch
from
b67a1b0
to
fc05d2e
Compare
src/rest_vol_dataset.c
Outdated
| "compound member names too long for URL"); | ||
|
|
||
| /* Copy name to query string without null byte */ | ||
| memcpy(cmpd_query_ptr, member_name, strlen(member_name)); |
There was a problem hiding this comment.
Looks like this and the reference in the line below should be url_encoded_member_name rather than member_name.
| /* If another member name will follow, add sepator */ | ||
| if (j < num_cmpd_members - 1) { | ||
| *cmpd_query_ptr = COMPOUND_MEMBER_SEPARATOR; | ||
| cmpd_query_ptr++; |
There was a problem hiding this comment.
It's likely worth adding another check for going past URL_MAX_LENGTH here before adding the separator and below before adding the NUL terminator just to be better assured that we aren't running off the buffer end.
jhendersonHDF
approved these changes
Apr 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Alternative implementation of #107. Compound reads are handled through
H5Tconvert. The previous issue with this - thatH5Tconvertwould overwrite the unused fields - is fixed by using the background buffer.Compound writes are more complicated, since HSDS expects to receive a packed buffer containing only the data for the fields that are actually being written to.
H5Tconvertleaves blank spaces in the buffer, so it was necessary to create the packed buffer throughH5Dgatherand a new callback.This depends on HDFGroup/hsds#298, which fixes some issues with writing to field selections.