GH-45263: [MATLAB] Add ability to construct RecordBatchStreamReader from uint8 array
#45274
+144
−42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rdBatchStreamReader MATLAB class.
…ytes constructor functions.
…roxy object is supplied to the RecordBatchStreamReader constructor.
github-actions
bot
added
Component: MATLAB
awaiting committer review
kou
reviewed
Jan 16, 2025
github-actions
bot
added
awaiting changes
github-actions
bot
added
awaiting merge
|
+1 |
github-actions
bot
added
awaiting changes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
To enable more workflows using the IPC Stream format in the MATLAB interface, this pull request adds the ability to construct a
RecordBatchStreamReaderfrom a MATLABuint8array.This is helpful, for example, to enable Arrow-over-HTTP workflows in conjunction with the MATLAB
webreadfunction (which can return auint8array from an HTTP request).This is a followup issue to #44923.
What changes are included in this PR?
static"construction function"arrow.io.ipc.RecordBatchStreamReader.fromBytes(bytes).static"construction function"arrow.io.ipc.RecordBatchStreamReader.fromFile(filename).arrow.io.ipc.RecordBatchStreamReaderconstructor to no longer directly accept afilenameas an input. Instead, aarrow.io.ipc.RecordBatchStreamReadercan now only be directly constructed from alibmexclass.proxy.Proxyinstance. This mirrors the design of other MATLAB classes which wrapProxyinstances in the MATLAB interface. To constructRecordBatchStreamReaderobjects from an Arrow IPC Stream file on disk, users can instead use the newstatic"construction function"arrow.io.ipc.RecordBatchStreamReader.fromFile(filename).Are these changes tested?
Yes.
arrow/matlab/test/arrow/io/ipc/tRecordBatchStreamReader.mto be parameterized over thefromFileandfromBytes"construction functions".RecordBatchStreamReaderconstructor is called directly with an input that is not alibmexclass.proxy.Proxyinstance.Are there any user-facing changes?
Yes.
arrow.io.ipc.RecordBatchStreamReaderobjects from an Arrow IPC Stream file on disk using the newstatic"construction function"arrow.io.ipc.RecordBatchStreamReader.fromFile(filename).arrow.io.ipc.RecordBatchStreamReaderobjects from an in-memory MATLABuint8"bytes" array using the newstatic"construction function"arrow.io.ipc.RecordBatchStreamReader.fromBytes(bytes).This PR includes breaking changes to public APIs.
This PR changes the signature of the public
arrow.io.ipc.RecordBatchStreamReaderconstructor to no longer directly accept afilenameas an input. Instead, aarrow.io.ipc.RecordBatchStreamReadercan now only be directly constructed from alibmexclass.proxy.Proxyinstance. This mirrors the design of other MATLAB classes which wrapProxyinstances in the MATLAB interface. To constructRecordBatchStreamReaderobjects from an Arrow IPC Stream file on disk, users can instead use the newstatic"construction function"arrow.io.ipc.RecordBatchStreamReader.fromFile(filename).Future Directions
static"construction function"arrow.io.ipc.RecordBatchStreamReader.fromBytes(bytes)in an example to demonstrate how to read an Arrow IPC Stream from an HTTP endpoint as part of apache/arrow-experiments.Notes
RecordBatchStreamReaderfromuint8array #45263