Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support uses of BACK that cause correlated references: SQLGlot conversion #234

Merged
merged 128 commits into from
Feb 17, 2025
Merged

Support uses of BACK that cause correlated references: SQLGlot conversion #234

merged 128 commits into from
Feb 17, 2025

Conversation

knassre-bodo
Copy link
Contributor

@knassre-bodo knassre-bodo commented Jan 28, 2025 •
edited
Loading

Child PR of #269 that is part of addressing #141. Adds the SQLGlot conversion step. With these changes, all SQLGlot correlation queries are functional except for:

  • TPC-H queries: 5 & 22
  • Correl queries: 1, 2, 3, 6, 8, 9, 15, 17, 18, 19, 20

These remainders need to be handled via the decorrelation cases in the comments of #141. Specifically:

  • Singular: Correl 8
  • Aggregation: TPC-H Q5, TPC-H Q22, Correl 1, Correl 2, Correl 3, Correl 18, Correl 19
  • Semi-Singular: Correl 9, Correl 17, Correl 20
  • Semi-Aggregation: Correl 6

knassre-bodo and others added 30 commits January 13, 2025 11:59
@knassre-bodo
Starting function list documentation
deeb914
@knassre-bodo
Adding datetime functions and bad boolean tests
5d6c513
@knassre-bodo
Merge branch 'main' into kian/function_docs
170492e
@knassre-bodo
Adding remaining functions including agg/window
8b8c098
@knassre-bodo
Adding toc
cb83f1f
@knassre-bodo
Adding toc
ef39e6e
@knassre-bodo
Fixing typo [RUN CI]
3c79543
@knassre-bodo
Started DSL documentation
3db13d1
@knassre-bodo
Adding calc, contextless, and back
8dc17b3
@knassre-bodo
Added TOC and TODOs
044c217
@knassre-bodo
Added TOC and TODOs
0252cee
@knassre-bodo
Changing highlighting
98c722b
@knassre-bodo
Merge branch 'kian/function_docs' into kian/dsl_docs
aa86d31
@knassre-bodo
Addded more examples
b03a5ec
@knassre-bodo
Added WHERE, ORDER_BY, and TOP_K documentation
b569bfb
@knassre-bodo
Fixing typo
7607fea
@knassre-bodo
Added extra example
215735b
@knassre-bodo @hadia206
Update pydough/unqualified/unqualified_node.py
0b8e1fb 
Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
a5190f8 
Revision 2

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
a0d43f3 
Revision 3

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
24c1e4a 
Revision 4

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
8009e05 
Revision 5

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
5d87b59 
Revision 6

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
a3ee536 
Revision 7

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
9945752 
Revision 8

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
4c6aa79 
Revision 9

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
e60dcc3 
Revision 10

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
a23c395 
Revision 11

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
2515683 
Revision 12

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo @hadia206
Update documentation/functions.md
d34c64c 
Revision 13

Co-authored-by: Hadia Ahmed <[email protected]>
@knassre-bodo knassre-bodo marked this pull request as ready for review February 10, 2025 20:18
amullerbodo
amullerbodo reviewed Feb 16, 2025
View reviewed changes
Copy link
Collaborator

@amullerbodo amullerbodo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no major things to share here.

pydough/sqlglot/sqlglot_relational_visitor.py
seen_cols.add(expr)
else:
raise ValueError(
f"Unsupported expression type for column merging: {new_column.__class__.__name__}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you able to provide column name here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A column name doesn't necessarily exist in this case (depends on what type new_column is and this is the catch-all for all unsupported types). This case should never occur by construction of the _is_mergeable_column API which prevents anything except these cases.

vineetg3
vineetg3 approved these changes Feb 17, 2025
View reviewed changes
Copy link
Contributor

@vineetg3 vineetg3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@knassre-bodo knassre-bodo mentioned this pull request Feb 17, 2025
Merged
Base automatically changed from kian/corrleated_backref to kian/correlation_omnibus February 17, 2025 20:26
@knassre-bodo knassre-bodo merged commit 0a28b20 into kian/correlation_omnibus Feb 17, 2025
3 checks passed
@knassre-bodo knassre-bodo deleted the kian/correlated_backref_2 branch February 17, 2025 20:28
knassre-bodo added a commit that referenced this pull request Feb 19, 2025
@knassre-bodo
Add support for uses of BACK that cause correlated references (#269)
947ca93 
Resolves #141. See issue for more details. Child PRs deal with the components of the issue, ancillary changes, and additional bugfixes:
- #232
- #234 
- #251 
- #254 
- #272
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amullerbodo amullerbodo amullerbodo left review comments

@vineetg3 vineetg3 vineetg3 approved these changes

@hadia206 hadia206 Awaiting requested review from hadia206

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@knassre-bodo @vineetg3 @amullerbodo