-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
draft: feat: compile and execute partitioned query
The main change here is having the `sparrow-execution` tests compile from a logical plan, created within a session, and execute things. Changes Include - Fix naming `field_ref` vs. `fieldref` (matches Python) - Tweak pipeline scheduling so that only transforms are grouped together. - Tweak pipelines so that each may output to multiple consumers - Start introducing a `SourceTasks` so we can poll all inputs on a single tokio task. - Fixes to how the scheduler detects "completion" (empty queue isn't sufficient). Likely follow-ups needed: - Use `park` and `unpark` so the worker threads don't spin TODO - [x] Actually execute the compiled query - [ ] Replace `ScalarValue` with arrows scalar? Literal too? - [ ] Move logical expr building to session (out of sparrow-logical) - [ ] Look at compiling out `tracing::trace` level statements. - [ ] Look at running the query in the sessions tokio runtime
- Loading branch information
1 parent
13be46f
commit 24ccecb
Showing
44 changed files
with
1,275 additions
and
524 deletions.
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.