Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add source trait #803

Merged
merged 5 commits into from
Oct 12, 2023
Merged

feat: add source trait #803

merged 5 commits into from
Oct 12, 2023

Conversation

jordanrfrazier
Copy link
Collaborator

@jordanrfrazier jordanrfrazier commented Oct 10, 2023 •
edited
Loading

Adds the source trait for sources to implement. The idea being they'll be created as async functions and we can just call next on the returned stream -- no need to make them a whole ReadPipeline or anything we think.

  • Adds the sparrow-sources crate
  • Adds the sparrow-interfaces crate
  • Refactors batch into sparrow-batch.

@cla-bot cla-bot bot added the cla-signed Set when all authors of a PR have signed our CLA label Oct 10, 2023
@jordanrfrazier jordanrfrazier changed the title draft: Add source trait feat: add source trait Oct 11, 2023
@github-actions github-actions bot added the enhancement New feature or request label Oct 11, 2023
@jordanrfrazier jordanrfrazier marked this pull request as ready for review October 11, 2023 04:14
bjchambers
bjchambers approved these changes Oct 11, 2023
View reviewed changes
crates/sparrow-arrow/src/lib.rs Show resolved Hide resolved
crates/sparrow-batch/src/batch.rs Show resolved Hide resolved
crates/sparrow-batch/src/batch.rs Show resolved Hide resolved
crates/sparrow-batch/src/batch.rs Show resolved Hide resolved
crates/sparrow-batch/src/batch.rs Show resolved Hide resolved
crates/sparrow-sources/src/in_memory.rs Show resolved Hide resolved
crates/sparrow-sources/src/in_memory.rs Outdated Show resolved Hide resolved
crates/sparrow-sources/src/in_memory.rs Outdated Show resolved Hide resolved
crates/sparrow-sources/src/error.rs Outdated Show resolved Hide resolved
crates/sparrow-interfaces/src/error.rs Outdated Show resolved Hide resolved
@jordanrfrazier
Copy link
Collaborator Author

TODO: add task for adding machete to ci

@jordanrfrazier jordanrfrazier mentioned this pull request Oct 11, 2023
Open
@bjchambers bjchambers added this pull request to the merge queue Oct 12, 2023
Merged via the queue into main with commit 13be46f Oct 12, 2023
13 checks passed
@bjchambers bjchambers deleted the partitioned/source-trait branch October 12, 2023 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bjchambers bjchambers bjchambers approved these changes

Assignees
No one assigned
Labels
cla-signed Set when all authors of a PR have signed our CLA enhancement New feature or request sparrow
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jordanrfrazier @bjchambers