Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interface layer between Parquet and Hadoop Configuration #2812

Closed
asfimport opened this issue Sep 14, 2023 · 0 comments
Closed

Add interface layer between Parquet and Hadoop Configuration #2812

asfimport opened this issue Sep 14, 2023 · 0 comments

Comments

@asfimport
Copy link
Collaborator

asfimport commented Sep 14, 2023

Parquet relies heavily on a few Hadoop classes, such as its Configuration class, which is used throughout Parquet's reading and writing logic. If we include our own interface for this, this could potentially allow users to use Parquet's readers and writers without the Hadoop dependency later on.

In order to preserve backward compatibility and avoid breaking downstream projects, the constructors and methods using Hadoop's constructor should be preserved for the time being, though I would favour deprecation in the near future.

This is part of an effort that has been discussed on the dev mailing list.

Reporter: Atour Mousavi Gourabi / @amousavigourabi
Assignee: Atour Mousavi Gourabi / @amousavigourabi

Related issues:

PRs and other links:

Note: This issue was originally created as PARQUET-2347. Please see the migration documentation for further details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant