Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MATLAB] Add IPC RecordBatchStreamReader MATLAB class #44923

Closed
sgilmore10 opened this issue Dec 3, 2024 · 1 comment
Closed

[MATLAB] Add IPC RecordBatchStreamReader MATLAB class #44923

sgilmore10 opened this issue Dec 3, 2024 · 1 comment
Assignees
@kevingurney
Labels
Component: MATLAB Type: enhancement
Milestone
19.0.0

Comments

@sgilmore10
Copy link
Member

sgilmore10 commented Dec 3, 2024 •
edited
Loading

Describe the enhancement requested

To enable support for the IPC Streaming format in the MATLAB interface, we should add a RecordBatchStreamReader class.

This is a followup issue to #44922

Component(s)

MATLAB
@github-project-automation github-project-automation bot moved this to Backlog in Arrow MATLAB Dec 3, 2024
@sgilmore10 sgilmore10 mentioned this issue Dec 3, 2024
Merged
sgilmore10 added a commit that referenced this issue Dec 4, 2024
@sgilmore10 @kevingurney @kou
GH-44922: [MATLAB] Add IPC RecordBatchStreamFileWriter MATLAB class (…
2b5a802 
…#44925)

### Rationale for this change

To enable support for the IPC Streaming format in the MATLAB interface, we should add a `RecordBatchStreamWriter` class.

### What changes are included in this PR?

Added `arrow.io.ipc.RecordBatchStreamWriter` class.

**Example Usage:**

```matlab
>> city = ["Boston" "Seattle" "Denver" "Juno" "Anchorage" "Chicago"]';
>> daylength = duration(["15:17:01" "15:59:16" "14:59:14" "19:21:23" "14:18:24" "15:13:39"])';
>> matlabTable = table(city, daylength, VariableNames=["City", "DayLength"]);
>> recordBatch1 = arrow.recordBatch(matlabTable(1:4, :))
>> recordBatch2 = arrow.recordBatch(matlabTable(5:end, :));

>> writer = arrow.io.ipc.RecordBatchStreamWriter("daylight.arrow", recordBatch1.Schema);
>> writer.writeRecordBatch(recordBatch1);
>> writer.writeRecordBatch(recordBatch2);
>> writer.close();
```

### Are these changes tested?

Yes. I Parameterized the test cases in `test/arrow/io/ipc/tRecordBatchWriter.m` to test the behavior of both `arrow.io.ipc.RecordBatchFileWriter` AND `arrow.io.ipc.RecordBatchStreamWriter`.

### Are there any user-facing changes?

Yes. Users can now use `arrow.io.ipc.RecordBatchStreamWriter` to serialize `RecordBatch`es/`Table`s to the Arrow IPC Streaming format.

### Future Directions

1. #44923
* GitHub Issue: #44922

Lead-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Kevin Gurney <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
@kevingurney kevingurney changed the title [MATLAB] Add IPC RecordBatchStreamFileReader MATLAB class [MATLAB] Add IPC RecordBatchStreamReader MATLAB class Dec 4, 2024
@kevingurney kevingurney self-assigned this Dec 4, 2024
@github-actions github-actions bot mentioned this issue Dec 18, 2024
Merged
kevingurney added a commit that referenced this issue Dec 23, 2024
@kevingurney @sgilmore10
GH-44923: [MATLAB] Add IPC RecordBatchStreamReader MATLAB class (#4…
8b75373 
…5068)

### Rationale for this change

To enable support for the IPC Streaming format in the MATLAB interface, we should add a `RecordBatchStreamReader` class.

This is a followup to #44922 

### What changes are included in this PR?

1. Added a new `arrow.io.ipc.RecordBatchStreamReader` MATLAB class.

### Are these changes tested?

Yes.

1. Added new MATLAB test suite `arrow/matlab/test/arrow/io/ipc/tRecordBatchStreamReader.m`.

### Are there any user-facing changes?

Yes.

1. Users can now create `arrow.io.ipc.RecordBatchStreamReader` objects to read `RecordBatch` objects incrementally from an Arrow IPC Stream file.

### Notes

1. Thank you @ sgilmore10 for your help with this pull request!
* GitHub Issue: #44923

Lead-authored-by: Kevin Gurney <[email protected]>
Co-authored-by: Kevin Gurney <[email protected]>
Co-authored-by: Sarah Gilmore <[email protected]>
Signed-off-by: Kevin Gurney <[email protected]>
@kevingurney kevingurney added this to the 19.0.0 milestone Dec 23, 2024
@kevingurney
Copy link
Member

Issue resolved by pull request 45068
#45068

@github-project-automation github-project-automation bot moved this from Backlog to Done in Arrow MATLAB Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kevingurney kevingurney

Labels
Component: MATLAB Type: enhancement
Projects
Arrow MATLAB
Archived in project
Milestone
19.0.0
Development

No branches or pull requests
2 participants
@kevingurney @sgilmore10