Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow streaming batches of data from a DataFrame #75

Merged
merged 10 commits into from
Sep 4, 2023
Merged

Allow streaming batches of data from a DataFrame #75

merged 10 commits into from
Sep 4, 2023

Commits on Sep 1, 2023

  1. Implement executeStream method 

    This returns a new RecordBatchStream class that acts similarly
to ArrowReader but record batches can be retrieved asynchronously.
    @adamreeve
    adamreeve committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    677504d View commit details
    Browse the repository at this point in the history

  2. Use C data interface for reading stream data

    @adamreeve
    adamreeve committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    0d8e9bf View commit details
    Browse the repository at this point in the history

  3. Support reading Arrow dictionary data with executeStream

    @adamreeve
    adamreeve committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    0f04ba8 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into execute_stream

    @jimexist
    jimexist authored Sep 1, 2023
    Configuration menu
    Copy the full SHA
    d8a302c View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2023

  1. Update datafusion-jni/src/dataframe.rs

    @jimexist
    jimexist authored Sep 2, 2023
    Configuration menu
    Copy the full SHA
    9f2ba2d View commit details
    Browse the repository at this point in the history

  2. Update datafusion-jni/src/stream.rs

    @jimexist
    jimexist authored Sep 2, 2023
    Configuration menu
    Copy the full SHA
    4b61a36 View commit details
    Browse the repository at this point in the history

  3. Update datafusion-jni/src/stream.rs

    @jimexist
    jimexist authored Sep 2, 2023
    Configuration menu
    Copy the full SHA
    4ed42c7 View commit details
    Browse the repository at this point in the history

  4. Apply suggestions from code review

    @jimexist
    jimexist authored Sep 2, 2023
    Configuration menu
    Copy the full SHA
    d1212c6 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2023

  1. Merge remote-tracking branch 'upstream/main' into execute_stream

    @adamreeve
    adamreeve committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    efde49b View commit details
    Browse the repository at this point in the history

  2. Fixes for upgraded JNI and datafusion

    @adamreeve
    adamreeve committed Sep 3, 2023
    Configuration menu
    Copy the full SHA
    3663dd6 View commit details
    Browse the repository at this point in the history