Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ByteVector Stack overflow exception with only 8KB of data. Mayba copy to ByteVector of Vec<u8> #13

Open
apps4uco opened this issue Apr 28, 2022 · 0 comments

Comments

@apps4uco
Copy link

Thanks for the crate. I really like the philosophy of invertible Codecs.

I ran into a problem parsing many small elements, of a stack overflow exception.

The test case below produces a stack overflow with only 8KB of data.

I believe that this is due to the fact that the library tries to avoid allocations by not consolidating arrays in ByteVector's and so they remain on the stack.

I think the fix would be to monitor the length of the ByteVector and when it goes above a threshold replace the current tree of ByteVectors of arrays (on the stack) with a single one which contains a Vec (on the heap).

Error:

thread 'codec::test::small_vec_repeat_should_work' has overflowed its stack
fatal runtime error: stack overflow

Test Case

  #[test]
    fn small_vec_repeat_should_work() {
        let mut b=byte_vector::empty();

        for i in 0..1000 {
            let small_array=[0u8;8];
            let small_vector=byte_vector::from_slice_copy(&small_array);
            b=byte_vector::append(&b,&small_vector);
            println!("len {}",b.length())
        }

        let vec=b.to_vec().unwrap();
        println!("Vec Len {:?}",vec.len())
    }

Workaround

I made a really nasty workaround of every time I append ByteVectors, I write a Vec of the result and then create a new ByteVector with that vector.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant