Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide API for PooledBlockAllocator management #771

Open
honourosis opened this issue Mar 13, 2024 · 1 comment
Open

Provide API for PooledBlockAllocator management #771

honourosis opened this issue Mar 13, 2024 · 1 comment

Comments

@honourosis
Copy link

The current PooledBlockAllocator implementation relies on a single PooledBlockAllocatorProvider instance. The current PooledBlockAllocator implementation has an upper limit on the number of free blocks that can be reused, but there is no way to free unused blocks.

Let's take for instance this scenario:

  1. We need to serialize a very large file, for example more than 100 Mb
  2. Using PooledBlockAllocator with blockSize = 32768 results in the creation of 2048 blocks
  3. Once the serialization is complete, we don't need 2048 blocks because all the following serializations will only use a few dozen blocks
  4. As a result, we have a lot of pre-allocated blocks that are never used

It would be nice to have some API to explicitly control the PooledBlockAllocator. Also, this problem can be solved by using some heuristics based on "allocation pressure". For example, if we have 2048 blocks, but have used no more than 100 blocks in the last 1 minute, we can be sure that reducing half of the free blocks would be safe.

@linlin-s
Copy link
Contributor

Hello, thanks for submitting this request. We will look into this as soon as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants