Enable single-tree transactional concurrency #56
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This introduces a third ActiveState that is managed on a per-committing-transaction basis. Rather than wait to release the locks on trees until after the transaction is fully synchronized, these changes release the locks after the file is fsynced but before the transaction log is synced. Before the transaction log is synced and the locks are released, the write state is cloned. After the transaction is confirmed, the state is published (taking care to ensure an older state doesn't overwrite a newer state since the OS controls waking threads). This works, but doesn't really offer parallelism due to the inability to batch the fsync operations. To complete this work, I need to figure out a good way to let the transaction manager issue the fsync commands, but be smart enough to only do it once per tree per batched commit.
The limiting factor of the last commit was that each write operation on each tree still had its own fsync operation. This change moves the responsibility of fsyncing to the transaction manager, which supports batching multiple transactions together already. Now the transaction manager keeps a unique list of paths that need to be synchronized as part of the batch. This allows multiple pending transactions for a single tree to pile up while waiting on the current batch to be fsynced. This is safe because when opening a file with a transaction manager, the file is automatically rolled back to the last confirmed transaction id. The transaction manager ensures each tree is fsynced before writing the transaction log entry and fsyncing the transaction. All new data written by pending transactions will be rolled back if a power loss occurs while the next batch is being fsynced unless the transaction log is fully synced. These changes allowed a significant speedup to BonsaiDb's commerce benchmark, and the new bottleneck is the reliance on two fsyncs.
Now returns the current read sequence rather than write.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs khonsulabs/bonsaidb#250