Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support add columns with all nulls #3044

Open
Tracked by #3016
wjones127 opened this issue Oct 24, 2024 · 2 comments
Open
Tracked by #3016

Support add columns with all nulls #3044

wjones127 opened this issue Oct 24, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@wjones127
Copy link
Contributor

wjones127 commented Oct 24, 2024

User should be able to write:

dataset.add_columns([pa.field("new_col1", pa.int32()), pa.field("new_col2", pa.string())])

and it would add the new_col1 and new_col2 to the end of the schema. The values would all be null.

After finishing #3016, this can just be a metadata-only operation. Would just add to the schema, and not modify the data files.

@wjones127 wjones127 added the enhancement New feature or request label Oct 24, 2024
@westonpace
Copy link
Contributor

Is this feature the only motivation for #3016? All null data files should be quite small, regardless of the data type so they might not be so bad.

@wjones127
Copy link
Contributor Author

Not at all. Motivation for #3016 is being able to create a schema and omit columns when inserting later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants