Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export datafusion_physical_optimizer, only use datafusion crate in the examples #14305

Merged
merged 4 commits into from
Jan 27, 2025

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jan 26, 2025

Which issue does this PR close?

Rationale for this change

DataFusion is largely intended to be used via the datafusion crate. It is split into multiple subcrates largely for project organization rather than for users to use subcrates

The examples currently use many of the datafusion subcrates (e.g. datafusion_common) directly. This is non ideal because

  1. It may confuse users who try to follow the examples and lead them incorrectly towards using the subcrates directly
  2. It may potentially mask issues where a struct is not publicly exported

Thus I think our examples should only use the DataFusion crate

What changes are included in this PR?

  1. Update the examples to use (only) datafusion
  2. Re-export ApplyOrder in datafusion::optimizer to make it easier to use
  3. Re-export datafusion_physical_optimizer as datafusion::physical_optimizer for consistency

Are these changes tested?

Yes, by CI

Are there any user-facing changes?

  1. Better examples
  2. Users can use datafusion::optimizer::ApplyOrder in addition to what they currently do use datafusion::optimizer::optimizer::ApplyOrder

@github-actions github-actions bot added the optimizer Optimizer rules label Jan 26, 2025
@alamb alamb marked this pull request as ready for review January 26, 2025 13:23
@github-actions github-actions bot added the core Core DataFusion crate label Jan 26, 2025
@alamb alamb changed the title Only use datafusion crate in the examples Export datafusion_physical_optimizer, only use datafusion crate in the examples Jan 26, 2025
Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @alamb. The description and changes make sense to me

@berkaysynnada berkaysynnada merged commit 0eebc0c into apache:main Jan 27, 2025
29 checks passed
@alamb alamb deleted the alamb/only_datafusion_in_examples branch January 27, 2025 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate optimizer Optimizer rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants