Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
lmdb: Platform-specific default map size
The current code works fine on x86 platforms since x86-64 uses at least 48-bit of virtual address space. On other 64-bit platforms like aarch64 or riscv64, the minimum allowed virtual address is 39-bit [1] [2]. Current 2**40 allocation will fail: ``` zict/tests/test_lmdb.py:53: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <zict.lmdb.LMDB object at 0x40043bfc10> directory = '/tmp/test_lmdb-sfepfnw8' def __init__(self, directory: str): import lmdb # map_size is the maximum database size but shouldn't fill up the # virtual address space map_size = 1 << 40 if sys.maxsize >= 2**32 else 1 << 28 # writemap requires sparse file support otherwise the whole # `map_size` may be reserved up front on disk writemap = sys.platform.startswith("linux") > self.db = lmdb.open( directory, subdir=True, map_size=map_size, sync=False, writemap=writemap, ) E lmdb.Error: /tmp/test_lmdb-sfepfnw8: Operation not supported zict/lmdb.py:43: Error ``` Switching to 2**37 on aarch64 and riscv64 should fix the issue. [1]: https://www.kernel.org/doc/html/v5.8/arm64/memory.html [2]: https://www.kernel.org/doc/html/latest/riscv/vm-layout.html
- Loading branch information