-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: compile and execute partitioned query #806
Merged
Merged
+2,211
−694
Commits on Oct 12, 2023
-
draft: feat: compile and execute partitioned query
The main change here is having the `sparrow-execution` tests compile from a logical plan, created within a session, and execute things. Changes Include - Fix naming `field_ref` vs. `fieldref` (matches Python) - Tweak pipeline scheduling so that only transforms are grouped together. - Tweak pipelines so that each may output to multiple consumers - Introduces a `SourceTasks` so we can poll all inputs on a single tokio task. - Fixes to how the scheduler detects "completion" (empty queue isn't sufficient). TODO - [x] Actually execute the compiled query - [x] Connect to sources - [x] Move compilation / execution into session. - [x] Look at compiling out `tracing::trace` level statements. - [x] Look at running the query in the sessions tokio runtime - [ ] Move logical expr building to session (out of sparrow-logical) - [ ] Replace `ScalarValue` with arrows scalar? Literal too? - [ ] Look at suspending workers when queue is empty / waking on push - [ ] Look at a trait for logical functions (type-checking, compilation)
Configuration menu - View commit details
-
Copy full SHA for d2d887c - Browse repository at this point
Copy the full SHA d2d887cView commit details
Commits on Oct 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 2faf0cc - Browse repository at this point
Copy the full SHA 2faf0ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for b9797e2 - Browse repository at this point
Copy the full SHA b9797e2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e65b9b - Browse repository at this point
Copy the full SHA 5e65b9bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ced7840 - Browse repository at this point
Copy the full SHA ced7840View commit details
Commits on Oct 16, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 9eb4afa - Browse repository at this point
Copy the full SHA 9eb4afaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 76acd0e - Browse repository at this point
Copy the full SHA 76acd0eView commit details -
Configuration menu - View commit details
-
Copy full SHA for f0cf075 - Browse repository at this point
Copy the full SHA f0cf075View commit details
Commits on Oct 17, 2023
-
Configuration menu - View commit details
-
Copy full SHA for c31dc1b - Browse repository at this point
Copy the full SHA c31dc1bView commit details -
Configuration menu - View commit details
-
Copy full SHA for e6a85dd - Browse repository at this point
Copy the full SHA e6a85ddView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9a86aa4 - Browse repository at this point
Copy the full SHA 9a86aa4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9683be4 - Browse repository at this point
Copy the full SHA 9683be4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 14e7641 - Browse repository at this point
Copy the full SHA 14e7641View commit details -
Configuration menu - View commit details
-
Copy full SHA for 648e7d8 - Browse repository at this point
Copy the full SHA 648e7d8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 297594f - Browse repository at this point
Copy the full SHA 297594fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 378bfe2 - Browse repository at this point
Copy the full SHA 378bfe2View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.