We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The file_writer_options can control the page size of lance file. But now it can't be set in dataset and fragment API.
file_writer_options
Add it into lance.write_dataset as:
lance.write_dataset
dat = { "foo": [1, 3], "bar": ["one", "three"], } file_writer_options = {"data_cache_bytes": "4096", "max_page_bytes": "4096"} ds = lance.write_dataset(dat, tmp_path, file_writer_options=file_writer_options)
Add it into LanceFragment.create as:
LanceFragment.create
df = pd.DataFrame({"a": [1, 2, 3, 4, 5]}) file_writer_options = {"data_cache_bytes": "4096", "max_page_bytes": "4096"} frag = LanceFragment.create(tmp_path, df, file_writer_options=file_writer_options)
The text was updated successfully, but these errors were encountered:
No branches or pull requests
The
file_writer_options
can control the page size of lance file.But now it can't be set in dataset and fragment API.
Add it into
lance.write_dataset
as:Add it into
LanceFragment.create
as:The text was updated successfully, but these errors were encountered: