Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: enrich panic context when BooleanBuffer fails to create #6810

Merged
merged 2 commits into from
Nov 29, 2024

Conversation

tisonkun
Copy link
Member

TRIVIAL AS IS

Today I encounter this panic while I have no idea whether my data set is too large or there is other logical errors.

@github-actions github-actions bot added the arrow Changes to the arrow crate label Nov 28, 2024
@tisonkun
Copy link
Member Author

tisonkun commented Nov 28, 2024

Besides, I encountered this issue when iterating StreamReader. I wonder if we should return an error on data too large rather than panic here.

Run with RUST_BACKTRACE=full to include source snippets.
Backtrace (most recent call first):
  File "rust:library/core/src/panicking.rs", line 148, in core::panicking::panic
2024-11-28T14:34:31.156844151+08:00   File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-buffer-53.2.0/src/buffer/boolean.rs", line 56, in arrow_buffer::buffer::boolean::BooleanBuffer::new
  File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-data-53.2.0/src/data.rs", line 1904, in arrow_data::data::ArrayDataBuilder::build_impl::{{closure}}
2024-11-28T14:34:31.156853077+08:00   File "rust:library/core/src/option.rs", line 1542, in core::option::Option<T>::or_else
2024-11-28T14:34:31.156856229+08:00   File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-data-53.2.0/src/data.rs", line 1900, in arrow_data::data::ArrayDataBuilder::build_impl
  File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-data-53.2.0/src/data.rs", line 1945, in arrow_data::data::ArrayDataBuilder::build_aligned
2024-11-28T14:34:31.156864860+08:00   File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-ipc-53.2.0/src/reader.rs", line 316, in arrow_ipc::reader::create_primitive_array
  File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-ipc-53.2.0/src/reader.rs", line 85, in arrow_ipc::reader::create_array
  File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-ipc-53.2.0/src/reader.rs", line 151, in arrow_ipc::reader::create_array
2024-11-28T14:34:31.156885018+08:00   File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-ipc-53.2.0/src/reader.rs", line 597, in arrow_ipc::reader::read_record_batch_impl
  File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-ipc-53.2.0/src/reader.rs", line 1327, in arrow_ipc::reader::StreamReader<R>::maybe_next
  File "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-ipc-53.2.0/src/reader.rs", line 1386, in <arrow_ipc::reader::StreamReader<R> as core::iter::traits::iterator::Iterator>::next
  File "rust:library/core/src/iter/traits/iterator.rs", line 4102, in <&mut I as core::iter::traits::iterator::Iterator>::next
...
2024-11-28T14:34:31.158210232+08:00 
2024-11-28T14:34:31.158243667+08:00 The application panicked (crashed).
2024-11-28T14:34:31.158250295+08:00   assertion failed: total_len <= bit_len
2024-11-28T14:34:31.158254971+08:00 in /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-buffer-53.2.0/src/buffer/boolean.rs, line 56
2024-11-28T14:34:31.158260049+08:00 thread: api_thread

@tustvold tustvold merged commit 283fb4b into apache:main Nov 29, 2024
26 checks passed
@tisonkun tisonkun deleted the patch-1 branch November 29, 2024 12:27
@alamb alamb added the documentation Improvements or additions to documentation label Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants