Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mmap() may not reserve all required memory #6

Open
jsteemann opened this issue Dec 2, 2016 · 0 comments
Open

mmap() may not reserve all required memory #6

jsteemann opened this issue Dec 2, 2016 · 0 comments

Comments

@jsteemann
Copy link
Contributor

The mmap() calls performed by CuckooFilter and InternalCuckooMap reserve a file-backed range of virtual memory. There is a seek and write to the very end of the memory-mapped file, but on some file systems that will create only a sparse file. The file is not pre-allocated at least, so there is no guarantee that when there is enough disk space when there is a read or write to the memory-mapped file.

For example, mmapping a range of 32 MB memory backed by a file may work, but the file system may only reserve a few pages of disk space. It may allocate the other space lazily, but this can fail. If it does, there is access to memory not backed by any file and this will normally fail at runtime with SIGBUS or so.

Failure can probably be reproduced by creating the memory-mapped files on a too small disk partition and then writing to the full memory range. This should fail somewhere in the middle, when the file system needs to provide the actual space but runs out of it.

I think a fix for this is to pre-allocate disk space for the memory-mapped file so it is guaranteed that later accesses to the file's memory will succeed. This could be done using fallocate(), posix_fallocate() or simply write().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant