Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider switching away from mmap #224

Closed
nazar-pc opened this issue Sep 27, 2023 · 1 comment
Closed

Consider switching away from mmap #224

nazar-pc opened this issue Sep 27, 2023 · 1 comment

Comments

@nazar-pc
Copy link
Contributor

I read Are You Sure You Want to Use MMAP in Your
Database Management System?
recently and sicne ParityDb seems to be using mmap actively, it is probably worth reconsidering its usage here as well given how few benefits it brings and how many drawbacks it has.

@arkpar
Copy link
Member

arkpar commented Oct 5, 2023

I'm familiar with this article. mmap-related issue listed there have a comparitevly lower impact for ParityDb, mostly bceause of the use case. ParityDb does not strive to be an all-purpose key-value store. Its main application is the blockchain.

mmap problems listed in the article:
Problem #1: Transactional Safety: Not that important because writes are serialized. In a blockchain node there's usually just one thread that needs to write to the database and observing partial transactions is OK (or resolved on the user level)
Problem #2: I/O Stalls: The library has a sync API anyway. I've benchmarks io_uring a while ago and it showed to have no benefit when used synchronously.
Problem #3: Error Handling: This is handled by the WAL.
Problem #4: Performance Issues: This one describes OS-related issues with page tables/TLB. It could be relevant in the future, but with the number or threads and data that we typically have, it does not seem to be an issue.

An early version of ParityDb used pwrite/pread based IO for everything. Switching to mmaps resulted in a considerable boost for our benchmarks.

@arkpar arkpar closed this as completed Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants