Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for BodoSQL context in PyDough #228

Open
knassre-bodo opened this issue Jan 21, 2025 · 0 comments
Open

Add support for BodoSQL context in PyDough #228

knassre-bodo opened this issue Jan 21, 2025 · 0 comments
Labels
documentation Improvements or additions to documentation effort - high major issue that will require multiple steps or complex design enhancement New feature or request user feature Adding a new user-facing feature/functionality

Comments

@knassre-bodo
Copy link
Contributor

knassre-bodo commented Jan 21, 2025

Goals of this issue:

  • Add the ability to create a PyDough database context that is attatched to a BodoSQL context (either local, Snowflake, or Iceberg catalog). E.g. pydough.active_session.load_database_context("BodoSQL", context=bc)
    • This may require creating a DB API 2.0 compliant wrapper around the BodoSQL context, which can simply raise an exception for everything except for cursor which fails for eveyrthing except execute and fetchall for the time being.
    • Alternatively, a refactor can be done to the DatabaseConnection API to work around this by allowing either a DB API 2.0 connection or the BodoSQL context, and all methods of the context switch on whether it is the connection or the BodoSQL context.
  • Add any necessary changes to SQLGlot transformation to work with a BodoSQL context (the Snowflake dialect should work?)
  • Add the ability to use the Pydough BodoSQL context to execute a query in BodoSQL and return the answer as a pandas DataFrame.
  • Add any necessary documentation
  • Add e2e testing of BodoSQL with local catalog & filesystem-based Iceberg catalog (will need PyDough metadata).
  • Add non-CI testing of BodoSQL with a Snowflake catalog & non-filesystem-based Iceberg catalog (requires using secrets to configure the catalogs). If it is possible to have a CI that accesses these secrets without exposing them to users, create a special workflow that runs these tests that triggers nightly or with special permissions. Otherwise, mark these tests so they are always skipped unless the secrets are present, which can only be done in local development.
@knassre-bodo knassre-bodo added documentation Improvements or additions to documentation effort - medium mid-sized issue with average implementation time/difficulty enhancement New feature or request user feature Adding a new user-facing feature/functionality effort - high major issue that will require multiple steps or complex design and removed effort - medium mid-sized issue with average implementation time/difficulty labels Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation effort - high major issue that will require multiple steps or complex design enhancement New feature or request user feature Adding a new user-facing feature/functionality
Projects
None yet
Development

No branches or pull requests

1 participant