Buffered Reads #60
Merged
Buffered Reads #60
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bbockelm
force-pushed
the
buffered_reads_v1
branch
4 times, most recently
from
aa4feee to
ef3558a
Compare
bbockelm
force-pushed
the
buffered_reads_v1
branch
from
ef3558a to
435b5d2
Compare
|
@jhiemstrawisc - while the CERN xrootd repositories are down right now (causing an old version of xrootd to be pulled and the tests to fail) which will cause the unit tests to fail, I think this is ready to review! |
This keeps a 2-entry cache inside the S3File object. These cache entries round up small reads and allow prefetching in the case of linear reads. The goal is to establish pipelining when doing linear reads, allowing data to be read to the S3File cache entry before it is requested.
- Missing imports that weren't necessary on Mac OS X. - Signed comparision issues. - Content-length was wrong type -- should really be off_t.
bbockelm
force-pushed
the
buffered_reads_v1
branch
from
435b5d2 to
96e8748
Compare
|
Going to self-merge this one -- it's got reasonable tests and I'd like to move the work forward. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces buffered reads -- reads go through a new cache layer, including prefetch.
This is built on top of the streaming reads patch and should be merged after that one.