Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: initial logical to physical compile #792

Merged
merged 6 commits into from
Oct 5, 2023
Merged

feat: initial logical to physical compile #792

merged 6 commits into from
Oct 5, 2023

Conversation

bjchambers
Copy link
Collaborator

This converts logical plans consisting of a tree of reference counted expressions into a physical execution plan.

Also renames the scan operation to read and table_id to source_id.

Work remaining in follow-ups:

  • Type-checking of the expressions in the physical plan.
  • Simplification pass to rewrite expressions within each step using arithmetic rules.
  • Projection pushdown pass to (a) move projections towards sources, allowing us to eliminate unnecessary columns, (b) combine projection steps.
  • Predicate pushdown pass to move select steps closer to sources.

@bjchambers
draft: feat: initial logical to physical compile
9e7af75 
This converts logical plans consisting of a tree of reference counted
expressions into a physical execution plan.

Also renames the scan operation to `read` and `table_id` to `source_id`.

Work remaining in follow-ups:

- Type-checking of the expressions in the physical plan.
- Simplification pass to rewrite expressions within each step using
  arithmetic rules.
- Projection pushdown pass to (a) move projections towards
  sources, allowing us to eliminate unnecessary columns, (b) combine
  projection steps.
- Predicate pushdown pass to move `select` steps closer to sources.
@cla-bot cla-bot bot added the cla-signed Set when all authors of a PR have signed our CLA label Oct 4, 2023
@bjchambers bjchambers changed the title draft: feat: initial logical to physical compile feat: initial logical to physical compile Oct 5, 2023
@github-actions github-actions bot added the enhancement New feature or request label Oct 5, 2023
jordanrfrazier
jordanrfrazier approved these changes Oct 5, 2023
View reviewed changes
crates/sparrow-backend/src/exprs/expr_lang.rs Outdated Show resolved Hide resolved
crates/sparrow-backend/src/exprs/expr_pattern.rs Outdated Show resolved Hide resolved
@bjchambers bjchambers added this pull request to the merge queue Oct 5, 2023
Merged via the queue into main with commit da03696 Oct 5, 2023
32 checks passed
@bjchambers bjchambers deleted the ben/logical-to-physical branch October 5, 2023 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jordanrfrazier jordanrfrazier jordanrfrazier approved these changes

Assignees
No one assigned
Labels
cla-signed Set when all authors of a PR have signed our CLA enhancement New feature or request sparrow
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bjchambers @jordanrfrazier